Talking to an AI Assistant Costs More Accuracy Than Typing at It

The microphone button keeps moving closer to the middle of the screen. Assistants now open in voice mode, cars and headphones take dictation, and the implicit promise is that speaking a question and typing it are the same act performed two ways. A group at the University of Southern California decided to check what the two channels actually deliver to the model on the other end.

Zizhao Hu, Nathan Elijah Segura, Mohammad Rostami and Jesse Thomason posted their results to arXiv on Aug. 4. They built a tool they call HIVE, the Human Input-Variation Engine, which applies both families of corruption under one controlled procedure: keyboard noise such as dropped and swapped characters, and voice noise such as filler words, disfluency and the sentence-level rewriting that transcription and AI dictation introduce. Five instruction-tuned open-weight models were run five times each across six benchmarks, among them GSM8K, MMLU-Pro, TruthfulQA and HumanEval.

The channels are not equivalent. Averaged across the runs, voice-transcription distortion cost about 9.7 accuracy points, against about 3.0 for keyboard noise, and the worst single operator on each side was further apart still: compressing a question into terse spoken shorthand cost 24.1 points, while replacing characters at random, the harshest typing corruption tested, cost 5.9.

What drives the damage is not the presence of extra words. The authors report that harm tracks token survival: destroying a word from the original question hurts, while padding new words in beside it costs very little. Homophone substitutions, which leave the sentence structurally intact, cost about 0.4 points. That is why transcription is the more expensive channel. It does not merely add ums and pauses, it rewrites the sentence.

The gap also has a shape. It appears on tasks where the answer has to be constructed or deduced, such as multi-step arithmetic and code, and largely vanishes on multiple choice: MMLU-Pro, the one benchmark in the set that offers options to pick from, showed penalties of only 1.2 to 1.6 points. A model that has to build an answer from the question needs the question intact. A model choosing between five options can recover from a mangled one.

The obvious remedy does not work evenly. Giving a model a larger thinking budget, the standard lever for hard prompts, recovered most of what keyboard noise had taken away and did very little for speech. For the most compressed spoken input it went the wrong way: more reasoning produced worse answers, which fits the token-survival finding, since compression is the operator that removes most of the original question.

Three limits belong with the numbers. The models tested are open-weight systems in the 7-to-14-billion-parameter range, not the commercial assistants most people talk to, so the size of the penalty on those is unmeasured here. And the authors are careful about contamination: they report that the harm does not come solely from benchmark questions the models have seen before, which concedes that some of it might.

For anyone choosing between the mic and the keyboard on a question that matters, the practical reading is narrow and usable: type the ones where the answer has to be worked out, and if you do speak, speak in full sentences rather than clipped ones.