The numbers.
Share of words wrong. Lower is better.
Public benchmark
AASR-Bench: 407 dictations full of fillers and self-corrections. Mostly synthetic voices.
| Size | Words wrong | |
|---|---|---|
| Typd, full | 1.5 GB | 8.4% |
| Wispr Flow, in the cloud | – | 8.6% |
| Typd, light | 0.6 GB | 8.8% |
| Whisper large-v3-turbo + Qwen3.5-9B | 7.8 GB | 12.9% |
| Parakeet v3 + Qwen3.5-9B | 6.8 GB | 14.3% |
| Parakeet v3, no cleanup | 0.7 GB | 26.9% |
Real people
Common Voice: 193 unscripted recordings, against the volunteers’ own transcripts.
| Typd, full | 8.6% |
|---|---|
| Wispr Flow | 9.1% |
My own dictations
70 of them, one speaker: me. Typd has heard my voice before, so this one favours it.
| Typd, full | 4.2% |
|---|---|
| Wispr Flow | 6.1% |
Where Typd still loses
715 clips of meetings and conversation, with human transcripts. Typd cleans up far better but misses more words. That’s what I’m training next.
| Words missed | Self-corrections fixed | |
|---|---|---|
| Typd, full | 11.3% | 84% |
| Whisper large-v3 | 7.8% | 70% |
| Parakeet v2 | 7.6% | 30% |
How
- Wispr Flow heard each clip through a virtual audio cable and its own app. Its text was only scored, never used to train Typd.
- Typd ran on the same audio, with the same model files as the app.
- Fillers left in count as errors. Capitals and punctuation don’t.
- Not tested: noisy rooms, or languages other than English.
Got a different result? Tell me.