We turned EmbeddingGemma 2 into a speech-to-text model
EmbeddingGemma 2 is repurposed into a 288× on-device English transcriber, but its ~5% WER trails leading speech models and multilingual support remains unproven.
Embedding Gemma Transcribe repurposes Google's EmbeddingGemma 2, a multimodal embedding model, into an on-device speech-to-text system for Apple Silicon Macs. It freezes the audio encoder and trains a small reader to map short audio slices into the model's text embedding space, selecting the nearest of roughly 229,000 English words; a separate tiny model adds punctuation and capitalization.
The author reports 288× real time—one hour of audio in 12.6 seconds on a base M3 MacBook Air with 8 GB—but a word error rate near 5%, versus 2–3% for leading speech models, and says only the reader was trained, for under two hours on 8×H100 using openly licensed English speech.
Key takeaways
- The base EmbeddingGemma 2 model was not trained for speech-to-text; only the reader is trained, on openly licensed English speech, for under two hours on 8×H100.
- Transcription works by mapping short audio slices into a shared text embedding space and choosing the closest English word from about 229,000 candidates; a small model adds punctuation and capitals.
- The author reports 288× real time on a base M3 MacBook Air with 8 GB, transcribing an hour of audio in 12.6 seconds via the Neural Engine.
- Accuracy is about 5% word error rate, compared with 2–3% for leading speech models; the author expects a better-trained reader to close the gap.
- The comparison table includes IBM Granite Speech 5.0 TurboCTC at roughly 150× and FluidAudio's Parakeet benchmarks on M4/M5 Pro Macs, which the author labels a rough guide rather than a like-for-like ranking.
- The Mac app is free, requires macOS 14+ and Apple Silicon, downloads a 715 MB model on first launch, and processes audio locally; dictation is triggered by holding Right Option.
What people think
Analysis details
7 comments analyzed. Analyzed .
The sample selects active discussion branches and includes available parent comments. It may omit parts of the full thread. The analysis included 7 of 12 usable stored comments because of the input limit. Selected themes do not measure community opinion or how common a view is.
Parallelism and chunking for fast transcription
A commenter asks whether fast transcription requires playing audio at 200x speed; a reply suggests it likely uses significant parallelism, splitting audio into overlapping chunks to preserve word transitions and deduplicating words.
Apple on-device transcription benchmark and testability
One commenter reports Apple's on-device model ran about 3x slower than Apple's reported results on a more powerful computer, with similar error rate, and shares a CLI benchmark; they later note they cannot test the new model because it is not provided and they do not want to extract it from the app bundle.
Whisper.cpp as an alternative to Apple's model
A commenter questions whether Apple's transcription is worth using, saying early voice-memo transcription was laughably bad in their experience, and that they defaulted to whisper.cpp.
Accuracy and training limitations of the Embedding Gemma 2 experiment
The experiment author says accuracy is good but below cloud models, decent for everyday use, and WER can improve with more varied training data; a commenter also asks about transcription quality.
Use as a decision model
A commenter asks whether the transcription experiment can be turned into a decision model.
Open-source release and TTS extension plans
The experiment author says they will open-source the whole thing soon and plans to release a TTS version.