Learned aliases beat rules for spelling names

No shipping dictation tool solves "the same name comes out spelled differently every time" with hand-written matching rules. The tools that work use three layers. The first is in-decoder biasing, used only where the model accepts a hint list. The second is deterministic replacement of misspellings already seen. The third is a fuzzy step that is phonetic or LLM-based and decides whether a new variant is the known term. Learning comes from diffing the text field the app just pasted into, read through the macOS Accessibility (AX) API. Wispr Flow documents this behavior and VoiceInk's open-source code implements it. No vendor claims it works in terminals. For a Parakeet TDT model, which takes no prompt, the evidence points one way: post-ASR correction is the main lever. FluidAudio's acoustic vocabulary rescoring, the obvious in-model alternative, measurably over-fires on personal lists and has an open bug that corrupts its scores. The research literature has converged on "retrieve, then decide": a cheap, high-recall similarity retriever proposes about 10 candidates, and a separate model decides whether to replace, with an explicit "none of these" option. Airboard's architecture already has that shape. Its weak point is the retriever: exact Metaphone-key equality over word windows is brittle by construction. The fix with the most support is to store every observed misspelling as a counted alias of its term, score new spans by graded phonetic or character similarity with spaces removed, and gate replacements on evidence that the span is not ordinary English.

Every leading app stacks the same three layers

Across the apps, vocabulary is applied in three places, and the mix depends mostly on whether the ASR model accepts hints. Wispr Flow runs ASR in the cloud ("Transcription always occurs on the cloud") and pairs it with fine-tuned Llama post-processing. Its correction rules are whole-word, keep the saved capitalization, and leave "don't" alone when a rule maps Don to Dawn (Wispr Data Controls; Teach Flow your words). Wispr's engineering blog describes "context-conditioned ASR models (conditioned on speaker qualities, surrounding context, and individual history)" within a 700 ms end-to-end budget. It does not say whether dictionary words reach the ASR or only the LLM (Technical challenges). Superwhisper sends vocabulary "alongside your audio as recognition hints" whose "effect varies by model". It runs its replacements "programmatically" with no AI, and it tells users to add a separate replacement for every misspelling variant they see (Superwhisper Vocabulary). That advice concedes that exact-match tables never converge when variants keep changing. MacWhisper and Talon offer only user-authored tables. MacWhisper has a global find-and-replace with "only replace separate words" (MacWhisper Support). Talon has a words_to_replace.csv for words the engine "hears as a weird word", plus an "additional words" list that gets "top priority in recognition" (Talon usage notes). Spokenly passes "keyterms" only to cloud models that support them; only the first 50 apply in live mode, and GPT-4o models "can be inconsistent" at following them (Spokenly Dictionary docs). Aqua Voice, Willow, Monologue, Typeless and Otter advertise dictionaries or automatic learning but do not document where they apply them. Otter caps vocabulary at 100 names plus 100 terms on Pro (Otter pricing).

The most useful single reference is VoiceInk, an open-source local Mac app that supports Parakeet. Its pipeline is "transcribe → filter → format → word-replace → AI enhance". With Parakeet, the ASR receives no vocabulary (prompt: nil). The vocabulary reaches only the LLM step, which is told to use it "as the spelling authority … replace similar-sounding or phonetically close transcription mistakes … Do not force a replacement when the text clearly means something else" (VoiceInk repo). Its deterministic layer lets a single rule hold several comma-separated wrong variants. Rules are sorted longest-first and matched case-insensitively with Unicode word boundaries (WordReplacementService.swift). An open-source design proposal for another Whisper+Parakeet app states the constraint directly: a post-decode pass of exact "heardAs" replacement plus Double Metaphone matching with a length-scaled edit-distance threshold "is the only mechanism supporting Parakeet, which lacks prompt input". It runs that phonetic pass only on terms with recorded mis-hearings or enough usage (waveform #70).

Platform vendors split the same way. Apple offers developers contextualStrings, a hint list of up to 100 phrases (Apple Developer Forums). It also offers SFCustomLanguageModelData, an on-device custom language model built from weighted phrase counts and X-SAMPA pronunciations; its WWDC example weights "Call \(name)" by how often the user called that contact (WWDC23 10101 notes). Every cloud API exposes a runtime bias list: Deepgram keyterms, AssemblyAI keyterms_prompt, Google PhraseSets, Azure phrase lists and AWS custom vocabulary. OpenAI's own cookbook offers the clearest head-to-head result for post-processing. Prompting Whisper with correct spellings was inconsistent, and long lists made it worse. GPT-4 post-processing with the same list fixed every product name in their test and was "notably more scalable" (OpenAI Cookbook).

Learning means diffing the pasted field, and terminals are a blind spot

Wispr is the only commercial app that documents its learning mechanism: "Flow monitors the text box where it pastes text to detect any edits you made to transcribed words." When the user changes a word's spelling, the new spelling is added to the dictionary (Wispr Data Controls). Auto-add keeps proper nouns, trade terms, acronyms, variant-spelled given names and email addresses. It skips paths, URLs, code symbols and everyday words (Teach Flow your words). VoiceInk's source shows the full machinery. After pasting, it records a baseline snapshot of the focused field and the pasted range. It watches the field for 60 seconds or until focus changes, then takes a final snapshot. It recovers the corrected span by anchoring on the unchanged text on either side. It reads text through kAXStringForRangeParameterizedAttribute, then kAXValueAttribute, then WebKit text markers. For Chromium and Electron apps it sets AXManualAccessibility on the target app (AutoLearnAXTextReader.swift). Candidate pairs then go to an LLM reviewer with two mandatory gates: "Phonetic evidence: the changed source and destination spans must recognizably resemble two renderings of the same spoken term" and "No semantic rewrite". The reviewer also rejects case-only edits and well-known brands, treats multiword names as one term, and does "batch canonicalization", folding spelling variants of one entity into a single form (AutoLearnAIReviewer.swift). Each of those rules reads like a fix for a failure the author hit.

The older lineage reaches the same gates. Microsoft's dictation patent separates a correction from a change of mind using three signals: acoustic similarity between the original and the replacement, a short delay before the edit, and a small edit (many successive changed words mean editing, not correcting). It learns a pronunciation only after it has appeared N times (US8019602). Dragon learns from explicit acts rather than arbitrary later edits: correcting through the Correction menu, "Spell that", "Train That", and a batch optimizer that refuses to run until enough dictation audio has accumulated (Nuance: Acoustic and Language Model Optimizer). Google's on-device research supplies the best numbers on what correction learning is worth. Name recall rose from 2.4% at baseline to 48.6% with TTS-synthesized audio of the name list and biasing, to 64.4% when users corrected only the names, and to 73.5% when they corrected every error. That took about 50 utterances per user (Sim et al. 2019). Apple's system Dictation, by contrast, does not document any learning from edits.

The AX approach has a known coverage hole, and it matches Airboard's. Wispr's IDE documentation admits Flow "can mistake other fields for terminals or miss a terminal pane". Its file tagging requires VS Code's Screen Reader Optimized mode (Wispr IDE doc). No vendor documents which apps auto-learn works in, and none claims terminal support. The other learning channels are screen context and explicit commands. Superwhisper reads selected text, active-window text, and clipboard contents copied within 3 seconds of dictation (Superwhisper llms-full.txt). FreeFlow screenshots the window and asks a cloud LLM to "pull out any key info like names with spelling" (HN). Wispr also has an explicit "Correct a misspelling" rule action (Wispr help). Demand for edit-learning is real: a MacParakeet user asked for exactly this, citing Wispr (macparakeet #1200).

In-decoder biasing trades away the precision dictation needs

The strongest in-model technique for Parakeet-family models is NVIDIA's CTC-WS (CTC-based word spotter). It runs a dynamic-programming word spotter over CTC log-probabilities against a context graph. It splices the detected terms into the greedy transcript and keeps them only if they beat a CTC alignment score of the original words. On a 100-term benchmark it raised Transducer F-score from 0.44 to 0.87: recall went from 0.28 to 0.85, and precision fell from 0.98 to 0.89. It runs at about twice the cost of greedy decoding (arXiv 2406.07096). Giving each term alternative spellings raised F further, to 0.904. It requires a CTC head, which Parakeet TDT 0.6B lacks. GPU-PB / TurboBias is the head-free alternative: token-level shallow fusion with an Aho-Corasick boosting tree, applied only where the decoder already wants to emit a non-blank token. In greedy mode it gains only 8–10 points of F (arXiv 2508.07014). On parakeet-tdt-0.6b-v2 it reaches greedy F 81.5 at precision 91.5% (TurboBias 2.0). One user found WER on parakeet-tdt-0.6b-v2 rose from 12.06% to 22.61% as the boost weight increased (NeMo #14500). Deep biasing methods (CLAS, TCPGen) give the largest rare-word gains, including a 46.8% relative cut in rare-word error. They require training a bias encoder into the model (arXiv 2109.00627), so they do not apply to a frozen CoreML checkpoint. Their transferable lesson is that the model learned a "no-bias" option by training with bias phrases randomly dropped.

FluidAudio's custom vocabulary is a CTC-WS-style rescorer. For TDT 0.6B it loads a separate 97.5 MB CTC 110M encoder, runs a second encoder pass, and roughly doubles peak memory. It also adds a "spotter-anchored acoustic rescue" that can replace a word "even if string similarity to the transcript is low" (FluidAudio CustomVocabulary.md). Its similarity gate is deliberately most permissive for lists of 10 terms or fewer. Eight unspoken short terms produced 12 false positives in one minute of ordinary speech under default settings. One user tested Parakeet v3 with a 51-term personal list on 500 real dictations, and the defaults changed 278 of 500 transcripts. The errors included "Maya → EMEA" 114 times and "twist → Tuist" 9 times. Even the most conservative setting produced false replacements on 9.3% of a Common Voice set (FluidAudio #967). Issue #991, open since 2026-10-05, reports that the CTC spotter's 240,000-sample input buffer is never zeroed, so "CTC log-probs [are] often independent of audio". The reporter saw correct output in only 6 of 60 runs with an empty vocabulary. The same code is in Airboard's pinned 0.15.5 (FluidAudio #991). Short dictations are mostly padding, so the bug should hit Airboard's use case hard (inference). FluidAudio's own docs concede that over-firing and real recall "come from the same mechanisms … and neither string similarity nor token length separates them".

Cloud vendors repeat the warning. Google says boost "can also increase the likelihood of false positives" (Google Cloud adaptation). Deepgram recommends 20–50 keyterms and has removed weights from its newer models (Deepgram Keyterm). AWS stopped supporting its SoundsLike and IPA pronunciation fields (AWS custom vocabularies). AssemblyAI calls a keyterm list "a thumb on the scale … including in places where the model wasn't unsure at all". In its example, correctly heard audio of "Donahue" became "Donoghue" in 3 of 3 runs because that variant was on the list (AssemblyAI blog). That is Airboard's own risk in miniature. A personal list is small, which is the favorable regime, but none of these papers or vendors reports false-fire rates on term-free speech with 5–50 personal names. That gap in the evidence matters most for a dictation app.

Retrieve, then decide: the retriever is where rules fail

The post-ASR literature converges on a two-stage design that already resembles Airboard's. SpellMapper (NVIDIA) indexes each user phrase under its character n-grams and under the misspelled n-grams a particular ASR tends to produce for them. It learned these 1.9 million mappings by running TTS audio through the ASR and aligning the output with the reference text. It retrieves the top 10 candidates, and a 67M-parameter character-level tagger labels each span with a candidate or "0, no match". It achieved 87.4% precision and 67% recall of corrections, a 21.4% relative WER gain, with 90.4% top-10 retrieval recall (arXiv 2306.02317). Retrieval-augmented LLM correction (RAGEC) fetches entities using orthography-derived acoustic neighbor embeddings, which reached 84.8% top-1 recall against 53.5% for BM25. It then gives the LLM a capped, distance-thresholded shortlist and achieves 33–39% relative WER reduction. Giving it more candidates caused regressions because "additional matches seem to confuse the model" (arXiv 2409.06062). Cisco's MindMeld shows how much each signal adds to top-1 entity recall. Text features alone scored 0.100. Adding Double Metaphone and grapheme-to-phoneme (G2P) features raised it to 0.255, n-best hypotheses to 0.326, and personalization priors to 0.508. Edge n-grams helped because phonemes at word edges dominate perceived similarity (ACL D19-3011).

On the question of when to replace, the clearest evidence comes from a Parakeet study. Plain LLM correction nearly doubled WER on one set, from 10.83% to 20.01%. Inlining per-word ASR confidences in the prompt, as in RAFELLES[0.61], brought it down to 10.58%, and harmful corrections stayed below 5% on high-confidence speech (arXiv 2509.25048). Phonetic context and n-best lists likewise reduce over-correction of correctly recognized words (arXiv 2505.17410). Purely generative correction struggles to introduce rare names that are absent from its input, which is why retrieval is needed (arXiv 2403.17645). A caveat on confidence: Parakeet TDT v3 word confidences in NeMo have known alignment bugs (NeMo #15143), and whether FluidAudio exposes token probabilities is unverified.

Applied to Airboard's failures, this evidence is specific about where the current system breaks (the mapping is inference, but closely grounded). Equality of Metaphone keys over 1–3 word windows hashes a span into a bucket, and ASR variants scatter across buckets. "PiPai" (one token), "Pi Pai" (two tokens) and "Pype AI" (a word plus a vowel-initial letter-name) split differently. "Bright Eye" maps "AI" to an English word. Every published retriever instead scores graded similarity in a space that ignores word boundaries: character n-grams with spaces as _ in SpellMapper, phoneme edit distance in MindMeld, learned embeddings in RAGEC. Every one also benefits from observed misspellings, whether as SpellMapper's mapping table, CTC-WS's alternative spellings (+3.4 points F), or FluidAudio's aliases. Rule zoos break because each new variant needs a new rule. A similarity threshold plus an alias table generalizes.

What this means for Airboard's design

The table separates what the sources establish from what this report infers for Airboard.

Design question	Evidence	Inference for Airboard
Where to apply	Parakeet takes no prompt; post-decode correction is "the only mechanism" for it (waveform #70); VoiceInk applies vocabulary only after ASR; GPT-4 post-processing beat Whisper prompting (OpenAI Cookbook); FluidAudio rescoring over-fires (#967) and has a corrupting buffer bug (#991)	Keep correction post-ASR. Do not re-add FluidAudio vocabulary boosting until #991 is fixed upstream or patched, and until it can be measured with rescue disabled, cbw ≈ 0–1, minSimilarity ≈ 0.7, against an offline harness (CLAUDE.md already requires the harness). Even then, treat it as a candidate proposer, which the #967 reporter found worked best, not as the final writer.
How to recognize variants	SpellMapper, MindMeld and RAGEC score graded character, phonetic or embedding similarity across word boundaries; observed variants raise recall (CTC-WS +3.4 F; SpellMapper mapping table); VoiceInk and Wispr store wrong forms per term	Store every confirmed (heard span → term) pair as a counted alias. Apply exact alias hits deterministically, since they have high precision. For unseen spans, compare space-stripped character n-grams and phoneme strings (Double Metaphone plus edit distance, or G2P) against the term and its aliases, using a threshold instead of key equality. Generate letter-name variants for acronyms ("AI" → "eye", "ai", "A I").
How to decide when to replace	Confidence-guided correction prevents the LLM from doubling WER (arXiv 2509.25048); fewer, thresholded candidates beat many; deciders need a "no match" option; the AssemblyAI Donahue case shows biasing overwriting correct words	Keep the yes/no decider, but show it only a few retrieved candidates plus the sentence. Scale the evidence required by how word-like the span is. A non-dictionary span ("PiPai", "Pipei") with a close alias can auto-apply. A span made of common English words ("Bright Eye", "pipe") needs the decider and context, because "Pype" and "pipe" sound identical, so acoustics cannot separate them. Add ASR word confidence if FluidAudio exposes it. Log every decision for the eval set.
How to learn where AX text is unreadable	No vendor claims terminal auto-learn, and Wispr admits terminal detection misfires; Dragon learns from explicit acts ("Spell that"); Superwhisper captures clipboard and selection; Sim et al. got 48.6% name recall from TTS-synthesized audio alone, and SpellMapper learns ASR misspellings via TTS→ASR	Do not rely on edit-diff in cmux. Add an explicit correction path that works anywhere, such as a hotkey or voice "fix that" that shows the last dictation and takes the right spelling, and record the result as an alias. Pre-learn aliases offline: synthesize each known name with several macOS TTS voices, transcribe it with the same Parakeet model, and store the outputs ("PiPai", "Pipei") as aliases before the user ever sees them. Keep AX diffing for apps where it works, and copy VoiceInk's AX fallbacks and AXManualAccessibility.
What to accept as a correction	VoiceInk requires a phonetic-evidence gate and a no-semantic-rewrite gate, rejects case-only edits and famous brands, and keeps multiword names atomic; Microsoft's patent uses similarity, timing, edit size and N occurrences	Implement those gates in code before any LLM call. Weight aliases by count, as Apple's PhraseCount weights by contact frequency. Require two or more observations, or one explicit user confirmation, before a learned alias applies silently.

Three things are uncertain. No source measures false-replacement rates for small personal lists on ordinary dictation, so Airboard needs its own measurements. Nobody has published head-to-head precision of Metaphone keys against phoneme edit distance. And the TTS-to-Parakeet alias trick is an extrapolation from Google's and NVIDIA's pipelines, not something tested on Parakeet TDT.

Conclusion

Airboard's approach is not wrong. Its pieces sit in the wrong order of trust. The industry and the literature both treat the user's own observed misspellings as the strongest evidence, stronger than canonical spellings, acoustic scores, or hand rules. Edit learning, explicit fixes and TTS pre-learning should therefore all feed one artifact: a per-term table of counted aliases. A similarity retriever generalizes from that table, and a decider rules only on the ambiguous remainder. That reframing also solves the terminal problem without solving AX. If aliases can come from synthesis and from explicit fixes, edit capture becomes one optional source of evidence instead of the bottleneck. The decider should be stingy in proportion to how much the span looks like real English: being out of vocabulary is the cheapest strong signal available, and the cases it cannot settle ("Bright Eye", "pipe") are exactly the ones no acoustic method can settle either.

The next step is an eval harness, not more rules. Use the logged dictations and decisions, the known variants ("PiPai", "Pipei", "PiPI", "Brightai", "Bright Eye") and a set of term-free sentences, and report both correct fixes and false replacements for every change to the retriever or gate. Every vendor and paper that tuned biasing found that precision and recall move together. FluidAudio's own docs admit that its over-fire and its recall come from the same mechanisms. A design that cannot measure that trade-off will keep rediscovering it one broken rule at a time.