My AI podcasts now speak Mandarin: 169 episodes, and what it took to build them
All four of my AI podcast series now have Mandarin editions: 169 episodes and about 35 hours of audio. Here is what it took to build that capability: seven speech engines and 11 speech-recognition models compared, about 330 clips judged by ear, and 14 kinds of problem found and fixed.
All four of my AI podcast series now have Mandarin editions. That is 169 episodes and about 35 hours of audio, with the same two hosts, Maya and Leo, now speaking Mandarin.
The four series are Mastering Language Models, Agentic Coding Capability, Agentic Safety & Alignment, and a standalone collection of single episodes. Together the Mandarin episodes hold more than 10,000 dialogue turns and over half a million Chinese characters.
This post is about the capability behind them: what I compared, what I checked by ear, and how many problems I found and fixed along the way. The work took about three months. I am not naming the models I run in production or describing how the pieces connect.
What I compared
I compared seven text-to-speech engines: Chatterbox, CosyVoice3, Fish Audio, GPT-SoVITS, IndexTTS2, Kokoro and VoxCPM2.
To check the audio automatically, I needed speech recognition I could trust. I compared 11 open Mandarin speech-recognition models on 168 clips of my own podcast that I had labelled by ear, about 25 minutes of audio. On the clips I had confirmed correct, the most accurate model got about 2.4% of characters wrong; the least accurate got 14.3% wrong.
Accuracy alone was not the answer. The most accurate models sometimes silently "fixed" words my voice had mispronounced, so the mistake disappeared from the transcript. A quality checker has to hear what was said, not what should have been said.
What I checked by ear
My first Mandarin renders passed an automated check and still failed when I listened. Some problems never show up in a transcript, so a human ear stayed in the loop.
I judged about 330 clips by ear:
| What I listened for | Clips |
|---|---|
| Clips judged on my listening pages | 173 |
| A blind sample to calibrate the automated checks | 79 |
| Other blind listening tests | 80 |
I also listened to 8 full episodes end to end before choosing the production voice.
Of the 262 clips I marked ok or defect, 47 were defects, about 18%. The rule I worked by: machines check everything, and my ear should get only what they cannot decide.
What I found and fixed
Over the project I found and fixed 14 distinct kinds of problem in the Mandarin audio and scripts. I am not listing them here, because the list is a map of how the pipeline works.
The work took more than 100 Mandarin-specific code changes. More than 2,200 automated tests passed on the latest run, and that test gate must pass before any commit.
The pipeline now runs seven kinds of automated check on each new episode. The final repair pass checked 7,792 dialogue turns across 124 episodes, selected about 1,000 of them for re-rendering, and merged all 124 episodes.
Making the checks quieter mattered as much as making them stricter. One fix to a noisy check cut the episodes waiting for my ear from 28 to 6.
What this capability is
I now trust a Mandarin episode from this pipeline enough to put my name on it, and I want my listening time to go only to the spots that need a human. I expect the same approach to carry to the next language.