AIToday
Audio & SpeechZenn AI/MLPublished: Nov 21, 2026, 04:00 JST

Gemini TTS read the tone prompt aloud: engineer trims 15s with ffmpeg

Send on LINE
Gemini TTS read the tone prompt aloud: engineer trims 15s with ffmpeg

3 Key Points

  1. What happened

    He sent a 10-line script plus a 'play a capable secretary' tone instruction to Gemini's TTS model (voice Laomedeia) in one call; the resulting 62.7-second audio opened with about 15 seconds of the instruction being read aloud, and after cutting it the script portion ran 47.7 seconds.

  2. Why it matters

    He caught the problem by running the audio through faster-whisper transcription, which is already part of his pipeline as an automatic check that the audio matches the script — so a leaked instruction would lower the match score and be noticed.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

The experiment grew out of a small, repeated workflow: an engineer who holds daily meetings with AI executive personas turns each session into a roughly one-minute video. Until now the voice was made by hand — extracting audio from a video generated with Gemini — so script-to-voice was never a single step. The proposal to call the TTS API directly came from his CTO-role AI, and the appeal was precisely that it could go from script to voice in one pass.

Two design choices shaped what happened next. Rather than calling the model ten times, once per line, he sent all ten lines in a single call, on the view that a per-line approach tends to shift the voice's tone from call to call; per-line timing would instead be recovered afterward from transcription. And because the model received the tone instruction alongside the script, it treated both as text to be read — which is how roughly 15 seconds of instruction ended up at the head of a 62.7-second file, leaving 47.7 seconds of script after the trim.

The transcription step is what makes the failure detectable rather than silent: it was already slated to serve as an automatic 90%-or-better match check against the script, and a leaked instruction lowers that score. Open questions remain — the whole flow is still a draft awaiting his sign-off, and the split-field fix is untested.

FAQ
How did he notice the instruction had been read aloud?
He ran the generated audio through faster-whisper transcription and saw the instruction text at the start of the transcript.
What voice and model did he use?
He used Gemini's text-to-speech (TTS) model with the voice named Laomedeia, a higher-pitched girl's voice, and sent a tone instruction to act as a capable secretary.
How does he plan to fix it?
He intends to send the tone and role instructions in a dedicated instruction field and put only the lines to be spoken in the reading field — though he has not tried this yet.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleUS tech giants shift AI race to agentic action, led by Amazon and Google