Post-mortem: "stream it!" — and the one crash that taught me to read the threading API

So after the first version worked, i thought to properly use streaming for less latency would be a good idea. deepseek v4 flash seems to have made it, and here is the second and last post-mortem for this project. Again not written or edited by me.

The easy version of this story is "we made TTS stream into an audio engine with lower latency." The honest version is "the feature worked on the first try, and then a one-line mistake crashed the Objective-C runtime and forced me to understand how threading actually works in mulle-objc." This is the second version.

The mission #

mulle-speak synthesizes text to speech with pocket-tts-raven (a C API, mono float32 @ 24 kHz) and plays it through MulleAudio (an Objective-C wrapper over miniaudio).

My first cut was buffered: synthesize the entire utterance into one big buffer, then hand that buffer to the audio engine to play. Simple, correct, and — because pocket-tts runs much faster than realtime — it even sounded fine. The latency wasn't ugly, it was just wrong in principle: time-to-first audio should be the model's first chunk, not the whole sentence.

So the plan was obvious and, on paper, clean:

1. A producer thread runs ptt_stream_read and pushes each audio chunk into a lock-free ring buffer.
2. miniaudio's audio thread pulls from that same ring buffer live.
3. Playback starts the moment the first chunk lands.

Two threads, one ring buffer, done. What could go wrong?

The crash #

The producer thread was a raw mulle_thread:

1static mulle_thread_rval_t
2   produce( void *arg)
3{
4   ...
5   [dataSource appendFrames:chunk count:n];   // ⚠️ ObjC message on a raw thread
6   return 0;
7}
8
9mulle_thread_create( produce, args, &thread);

Compiles clean. Runs... and then, deep in the runtime:

mulle-objc-runtime/src/mulle-objc-class-search.c:848: mulle_objc_class_search_method:
  Assertion `mulle_objc_class_is_current_thread_registered( cls)' failed.

Abgebrochen (Core Dump). A dead, silent assertion from inside the runtime's method lookup. No helpful message. The program didn't even tell me which line.

This is the single most important thing I learned on this project, and it's not specific to mulle — it's true of essentially every ObjC/runtime with per-thread registration:

You cannot message an Objective-C object from a thread that the Objective-C runtime doesn't know about.

mulle_thread is a pure C thread primitive. It creates an OS thread, runs your function, and joins. It knows nothing about autorelease pools, garbage collection, thread-local class caches, or the per-thread GC/ABA bookkeeping that the mulle-objc runtime maintains. When the producer called [dataSource appendFrames:...], the runtime went to look up the method in the cached, current-thread class table — and found no registration for this thread → assertion.

The rule that falls out of this #

You now have two very different kinds of thread, and they mix fine at the OS level but not at the ObjC level:

So the decision is really:

• If the thread's work is all plain C (calling into a C API, moving bytes, joins, condition vars) → mulle_thread is fine and simpler.
• If the thread needs to send ObjC messages (retain/release objects, call methods, use Foundation containers, log with NSLog) → you must use an NSThread (or the mulleThreadWith* factory), which does the registration bookkeeping and gives you an autorelease pool.

It's not a performance judgment — a registered NSThread is still a real thread. It's a "does the runtime think this thread exists?" judgment.

The mulle-objc API I should have reached for #

To message ObjC from a thread in mulle, the path is NSThread with an object function that CARRIES an id argument. From the header:

1typedef int   MulleThreadObjectFunction_t( NSThread *, id);

1+ (instancetype) mulleThreadWithObjectFunction:(MulleThreadObjectFunction_t) f
2                                        object:(id) obj;
3+ (instancetype) mulleThreadWithTarget:(id) target
4                              selector:(SEL) sel
5                                object:(id) arg;

You provide an object-function whose first param is an NSThread * and whose second is the id you passed (the runtime holds a registration on this — it created the class cache for this thread, and an autorelease pool). The thread returns an int status; you mulleJoin it from the caller. Target/object are retained and released by NSThread.

That is the idiomatic replacement for what I tried to do by hand. If I needed my producer to actually message dataSource (and I had originally wanted to — appendFrames:/markEndOfStream: are ObjC methods), the correct code is:

 1static int
 2   produce_objc( NSThread *self, id context)
 3{
 4   StreamingFeed *feed = context;
 5   for(;;) {
 6      chunk = ptt_stream_read(...);
 7      [feed appendFrames:chunk count:n];   // ✅ runtime-registered thread
 8   }
 9   [feed markEndOfStream];
10   return 0;
11}
12
13NSThread *thr = [NSThread mulleThreadWithObjectFunction:produce_objc
14                                                 object:feed];
15[thr mulleStart];

Registered ✓, pool ✓, can message ✓.

What I actually did (and why it's also legitimate) #

I had a choice to make:

1. Reach for NSThread + object function (the "right" band-aid), or
2. Remove the ObjC from the hot path and keep the raw C thread.

The producer thread's only "ObjC" was two message sends — appendFrames: and markEndOfStream: — and underneath them it was already just calling into a C ring buffer. That made the cleanest fix obvious.

The cleanest fix, and the one that most tidily matches the "C-only hot path" philosophy that mulle_thread encourages, was option 2: make the producer 100% C, and let the ObjC wrapper live only on the registered/main side.

I did this:

1. Extracted pure-C functions operating on the ring context instead of using ObjC methods from the worker:

   MulleSpeakStreamAppendFrames( ctx, frames, count);
   MulleSpeakStreamMarkEnd( ctx);
   

   These take an opaque context (the same StreamContext * that backs the ring) and do acquire_write/commit_write. No msg_send anywhere.
2. Captured the context on the main (registered) thread and passed the raw pointer through the args struct to the pure-C thread.
3. Left the ObjC convenience methods (appendFrames:/markEndOfStream:) for consumers that already have a registered thread.

So the producer stays a mulle_thread because it genuinely has no reason to talk to the ObjC runtime — the boundary of the two worlds is drawn exactly at the ring buffer.

That is the other valid half of the lesson, and it's worth naming:

You can also keep a plain C thread if you move the ObjC boundary so the thread never has to call into the runtime. Design the concurrency so the workers only touch C, and hand ObjC interaction to a registered thread (or the main thread).

Both are legitimate. Pick based on how much of your worker is actually ObjC.

So, to answer your question directly #

"if you want to message ObjC you pretty much need to use NSThread; can't use mulle_thread alone."

Correct. A plain mulle_thread is a C thread with no ObjC registration, no autorelease pool, no class cache on the thread — so an ObjC message dispatch from it is undefined/crashy in the best of runtimes. If the work needs to message ObjC, you use NSThread / mulleThreadWithObjectFunction: / mulleThreadWithTarget:selector:object: so the runtime registers the thread.

The only way to keep mulle_thread for ObjC-side work is the one I chose: make the work pure C and move the ObjC messaging to a registered thread, passing the worker thennothing but opaque C data. Then mulle_thread is the better fit (less overhead, no pool machinery) — precisely because it's not touching ObjC.

So the two idioms map cleanly onto two job shapes:

Lessons / checklist #

1. Start streaming at the first chunk, not after the whole render. The ring-buffer + producer/consumer split is the right shape and was the right idea the whole time.
2. Message-passing to ObjC requires a runtime-registered thread. A raw C thread asserting in class-search is the whole story in one line.
3. Two ways to fix it: (a) use an NSThread ([muleThreadWithObjectFunction:]), or (b) keep the C thread but strip the ObjC out of it and hand it opaque data (pure-C feed helpers + captured context).
4. Read the threaded API before writing threaded code. The header documents exactly which factory registers the thread and which doesn't — 30 seconds there beats a core dump later.

May your background threads always know which universe they're in.