STT error correction, normalization and intent extraction

Speech-to-text is a powerful interface, but a transcript is not the conversation itself. Background noise, accents, interruptions, names, numbers, product codes and incomplete sentences can produce errors. If a voice agent passes the raw transcript directly into a tool or CRM, a small recognition mistake can become a wrong action, a confusing record or an unnecessary human handoff.

Production voice AI needs a layered understanding path: recognise speech, detect uncertainty, normalise entities, interpret the request in context, apply policy and choose the next action. Dring's technology approach, agent layer and quality process connect these steps instead of treating them as one prompt.

Know what kind of error occurred

Not every transcript difference matters. A filler word or punctuation issue may have no effect on the outcome. A product name, amount, date, account identifier or negation can change everything. Classify errors by their operational consequence: harmless wording, wrong entity, missing qualifier, wrong intent, unsafe confidence or lost context.

Use confidence as a signal, not a verdict. A high-confidence transcript can still be wrong for a rare name. A low-confidence fragment may become clear when combined with the previous turn and the expected workflow. The agent should be able to ask for confirmation when the potential consequence is high.

Normalise entities without erasing the caller

Normalisation maps different surface forms to a structured concept. “Two thirty tomorrow,” “tomorrow at 2:30” and a local date expression may refer to the same appointment slot. A product name may have a nickname, a misspelling and a spoken abbreviation. Store the structured value for systems while retaining the original wording for traceability.

Use entity types and context. A number could be an order, a phone extension, a quantity or a date. A name could be a customer, a dealer or a location. The agent should not normalise by pattern alone when the workflow has competing interpretations. Ask a narrow clarifying question rather than guessing.

Use domain adaptation and terminology

Rare and domain-specific phrases deserve a managed vocabulary. Google's Speech-to-Text model adaptation documentation describes phrase sets and custom classes for terms that occur frequently, are uncommon in general speech or are difficult in noisy audio. The same principle applies whether the underlying provider is Google or another speech stack: give recognition a contextual vocabulary and measure whether it improves the intended workflow.

Do not solve recognition by adding every term to every language. Prioritise words tied to identity, routing, money, safety, product choice and action. Review false positives, and connect the vocabulary to the pronunciation dictionary so the agent can recognise and say the concept consistently.

Extract intent from the whole exchange

Intent is not always present in one sentence. A customer may begin with a symptom, answer a question with a date and then reveal the real reason for contact. The agent should maintain the conversation state: what is known, what is uncertain, what has been asked, what has changed and what outcome remains open.

Use a hierarchy of intent. The top level may be support, sales, booking, operations or recruiting. The next level may be return, order status, qualification, appointment change or driver exception. The final level can be the action or handoff reason. This structure gives a manager a useful report and gives the agent a safe route without pretending the world fits one label.

Guard negation, timing and certainty

Voice errors often remove the words that change meaning: “not,” “still,” “already,” “only,” “tomorrow” or “unless.” Include these modifiers in the extraction schema. A customer saying “I have not received the refund” should not be classified as a completed refund. A caller saying “I can do Thursday, not Tuesday” should not be booked into Tuesday.

Represent certainty explicitly. “Customer believes,” “system confirms” and “agent proposes” are different states. The CRM and the next teammate need to know which is which. Dring's write-back model and reporting layer can preserve that difference.

Design correction as a natural turn

When the agent suspects a recognition error, it should not recite the entire transcript. Confirm the smallest risky value: “I heard Thursday at 3 PM. Is that right?” For a name, use spelling or a choice. For a product, repeat the full canonical term. For a payment or identity action, follow the approved verification path before taking the next step.

Correction also has a tone. A calm, short question maintains trust. Repeatedly asking the caller to start again signals that the system has lost the case. Store the correction so the quality process can improve the vocabulary, prompt or routing rule later.

Test the pipeline, not only the model

Build test cases with noise, accents, code-switching, interruptions, incomplete answers, ambiguous numbers, similar names, customer corrections and changing intent. Score the transcript, entity normalisation, intent, policy, tool parameters, spoken confirmation and final record separately.

When a defect appears in a production call, classify its layer. If the recogniser heard the wrong term, update adaptation or vocabulary. If the transcript was adequate but the agent chose the wrong intent, change context or policy. If the action was correct but the record was wrong, fix the integration. The Agent Factory turns that diagnosis into targeted regression coverage.

A practical STT understanding checklist

• Mark high-consequence entities and modifiers such as negation and timing.
• Use contextual vocabulary for names, brands, products and industry terms.
• Preserve raw wording while writing normalised values to systems.
• Ask narrow confirmations when uncertainty could change the outcome.
• Separate caller statement, system fact and agent inference.
• Review recognition, intent, tool, record and handoff defects as different classes.

STT quality is not a race to produce a perfect transcript. It is the discipline of turning imperfect speech into a safe, useful and reviewable next step. That is where a voice AI system starts to feel like a capable operator rather than a dictation layer.

Separate transcript cleanup from intent confidence

Normalisation and intent extraction solve different problems. Normalisation can standardise a phone number, date, product code or common spelling while preserving the raw transcript for review. Intent extraction decides what the caller is trying to accomplish and how confident the system is. Keep both outputs visible. A clean-looking value should not make an uncertain intent appear reliable, and a low-confidence intent should not overwrite the caller's original wording.

Test the boundary with interruptions, accents, background noise, code-switching, local abbreviations and callers who correct themselves. Check the transcript, the normalised field, the selected intent, the spoken confirmation and the downstream record together. When the signals disagree, ask a short clarification or route to a person. The Agent Factory can turn these disagreements into repeatable cases so improvements are measured across languages and workflows.

Further reading

• Google Cloud on speech model adaptation and custom classes
• OpenAI on tool boundaries and agent decision design
• NIST on measuring AI performance and documenting uncertainty