Post-mortem: strapping an ONNX text-to-speech engine onto an Objective-C app

· nat's blog

I let opencode - deepseek v4 flash code this project and here is it's unabridged post-mortem.

mulle-speak is a tiny Objective-C executable that turns a line of text into speech. Synthesis comes from pocket-tts-raven — the lightweight, CPU-only C++ runtime for Kyutai's Pocket TTS — driven through its plain C API. The resulting audio plays back through MulleAudio, an Objective-C wrapper over miniaudio.

The synthesis and the playback were the easy 20%. The other 80% was a single, recurring problem: how to make a dependency with its own deep, opinionated, CMake-driven sub-build part of a build system that expects simple libraries. This is a post-mortem of that problem and the one pattern that actually solved it.

The easy part: the C API and the playback #

The pocket-tts C API is clean and streaming:

ptt_create                       create the engine once
ptt_stream_start(text, voice)    begin synthesizing
loop ptt_stream_read -> float*   pull audio chunks
ptt_free_audio(chunk)            free each chunk
ptt_stream_end / ptt_destroy     teardown

Audio is always mono float32 at 24 kHz — the same format miniaudio wants, so the playback side was nearly a one-liner. MulleAudio already exposes

audioVoiceWithBytes:length:
    sampleEncoding:MulleAudioSampleEncodingFloat
    numberOfChannels:1
    sampleRateHz:24000
    options:MulleAudioVoiceOptionVoiceAny

Feed it the buffer, playVoice:, wait, free. The wrapper class I wrote around the C engine (MulleSpeak) plus a + speak factory to locate models/ and voices/ reviewed clean and never caused a bug. That whole feature was a couple of hours.

The real story is what it took to get pocket-tts-raven built and packaged in the first place.

The problem: you don't build the dependency, its sub-build builds it #

pocket-tts-raven is not a small library. It's an application with a deep build of its own, and that build pulls in its own dependencies via CMake FetchContent:

pocket-tts-raven
├── onnxruntime        (prebuilt tarball)
├── dr_libs            (git, pinned to a specific commit)
└── sentencepiece      (git)
    └── abseil-cpp     (git, via sentencepiece's own FetchContent)

At first I tried to make the host build drive this from the outside. That was the mistake, and it failed in a cascade of edge cases.

The cascade of fetch-policy edge cases #

Shallow clone vs. a pinned commit. dr_libs is pinned to a specific SHA, but FetchContent fetches with git clone --depth 1 --no-single-branch and then runs git checkout <sha>. A shallow clone only carries the branch tips, not the tree objects for an arbitrary pinned commit, so the checkout fails:

fatal: unable to read tree (cd99e...)
CMake step for dr_libs failed: 1

The standard escape hatch is to feed FetchContent a local, full clone via FETCHCONTENT_SOURCE_DIR_<NAME>, or disable shallow fetching entirely. Which brings us to the next wall.

Overrides leak between nesting levels. sentencepiece itself pulls abseil-cpp with its own nested FetchContent. If you set FETCHCONTENT_SOURCE_DIR_ABSEIL_CPP to point at an existing source directory, FetchContent treats it as pre-existing and skips add_subdirectory — so the absl::* CMake targets are never created, and sentencepiece fails to link against targets that "don't exist". The override that fixed dr_libs silently broke Abseil one level down.

The takeaway crystallized quickly: a dependency's build is a tree, not a list. Each node has its own fetch policy, its own pinning, and its own idea of what "available" means. There is no single knob that fixes a whole tree from the outside — every knob you turn at one level leaks into the next.

The fix that mattered: make the dependency own its build #

Instead of continuing to drive pocket-tts-raven's CMake from the host build — which meant inheriting all of its fetch policy and re-fighting every level — I flipped the model:

Wrap the uncooperative dependency in its own build step, then consume the resulting artifact as an opaque blob.

In mulle-sde this is a first-class feature called craftinfo: you can override how a specific dependency builds by giving it a build script.

mulle-sde dependency craftinfo --os linux \
    set pocket-tts-raven BUILD_SCRIPT pocket-tts-raven-build.linux

The script does three things:

  1. Builds pocket-tts-raven once, using its own CMake, inside the dependency's own directory — where its nested FetchContent tree lives and behaves as designed. (The only external nudge is a local, full dr_libs clone to dodge the shallow-clone/pinned-commit problem, since that one is a genuine incompatibility, not a policy choice.)
  2. Installs the built artifacts into the host's dependency prefix — the libpocket_tts.so, its libonnxruntime.so, and pocket_tts.h — under the names the host linker expects.
  3. Registers the header via the build system's include/import mechanism so app code can #include <pocket-tts-raven/pocket_tts.h>.

That single decision collapsed three separate walls. I stopped arguing with two levels of nested FetchContent and instead said: you build yourself, I consume the result. The dependency became an opaque artifact with a clean boundary, and the host build went back to being about the host's own code.

The broader, tool-agnostic version of the rule:

When a dependency is an application with its own opinionated sub-build, don't inherit that build — isolate it behind a script and treat its output as an artifact.

Whether it's craftinfo in mulle-sde, a vendored wrapper, a container step, or a FetchContent "overlay", the shape is the same: the boundary is between their build and your build, and you want that boundary explicit.

Two subtle traps that cost real time #

1. The SONAME is what matters, not the filename you copied #

The engine .so records a DT_NEEDED on the versioned libonnxruntime.so.1. If your install step only copies the unversioned libonnxruntime.so, the runtime loader complains and the link fails with an undefined reference to OrtGetApiBase@VERS_1.23.2.

Lesson: ship every sibling the SONAME references — libonnxruntime.so, .so.1, .so.1.23.2 — and install the engine under both the host's expected name and the SONAME name the loader will ask for. Filenames you choose, SONAMEs the library chooses.

2. Multi-level build = multi-level namespace collisions #

The host expects the library under one name (pocket-tts-raven), the SONAME is another (pocket_tts), and the runtime loader needs both visible in the same place. Similarly, when I hand-copied files into the build system's managed dependency directory, I broke its ownership tracking and it then couldn't manage that directory later.

Lesson: let the build script do the copying into the managed prefix (so ownership stays consistent), and don't graft files into managed directories by hand as an experiment.

The real post-mortem lesson #

The headline of this whole exercise is embarrassingly simple in hindsight:

The easy part was the code. The build of the dependency was the work.

Before this project I would have reached for "just add the dependency and let FetchContent / the package manager sort it out." That works for flat, simple dependencies. It breaks the moment a dependency is itself an application with a deep, opinionated sub-build: then their fetch policy, their pinning, and their multi-level target graph become your problem, one wall at a time.

The reusable move — worth stealing even outside mulle-sde — is to own the boundary: wrap the uncooperative build in a script, let it build itself in its own habitat, and consume the artifact. That single decision turned a frustrating, three-wall cascade into a working, self-contained package:

bin/mulle-speak
lib/libpocket_tts.so   (+ libonnxruntime.so)
share/mulle-speak/models/   # fetched separately (~165 MB, license-encumbered)
share/mulle-speak/voices/

And it's why the final product is a one-liner for the user:

1mulle-sde craft
2mulle-sde run -- "Hello from mulle-speak!"

If you only remember three things #

  1. Bind the happy path first. The synthesis + playback was the easy, boring 20%; building the dependency was the real effort.
  2. Don't inherit a bad dependency tree's build. Wrap the uncooperative dependency in its own build step and consume the artifact. In mulle-sde that's craftinfo + a BUILD_SCRIPT — in any toolchain, it's "own the boundary between their build and yours."
  3. Respect the artifact contract. Ship SONAMEs, not just filenames, and let your build system's own mechanism install into its managed prefix — don't hand-graft files into it.

May your own dependencies be shallow in the ways you expect.

last updated: