A Chatbot's 'I Don't Know' Has a Number Behind It

A student who skips the questions they are least sure of walks out with a better score on the ones they attempted, and a thinner paper overall. Machine learning has a name for that trade, coverage against accuracy, and the study measures it in a chatbot and then goes looking for what drives the decision to skip.

The paper tests something narrower than it sounds: whether a language model actually uses its own sense of confidence when it decides whether to answer. Dharshan Kumaran and his colleagues took 1,000 factual questions from a public test set, rewrote each as a four-option multiple-choice item, and put four large language models through them twice. The first pass offered no way out. The second added a fifth option that meant, in effect, I don't know. All five authors work at Google DeepMind, which funded the work. One of them, Nathaniel Daw, is also at Princeton. He studies decision confidence in humans and animals, the research tradition this experiment borrows from.

With no way out, GPT-4o answered 63.7% of them correctly. Given the option, it declined on 56.6% of them, and on the 43.4% it chose to answer it was right 69.1% of the time. The two accuracy figures have different denominators: the first covers every question, the second only the ones the model kept. It did not get better at the questions. It answered fewer of them, and the ones it kept were the ones it was surest of. The authors claim no more than that the lift is consistent with a trade of coverage for accuracy, and the main text reports no significance test for it.

The skipping was not random. Fitting GPT-4o's choices against its own confidence, the team recovered an indifference point at about 77%: the confidence level at which the model was equally likely to answer and to decline. Below it, it mostly declined. That number is not something GPT-4o utters about itself. It is a probability read off the model's output and calibrated in advance on a separate set of questions, so that it lines up with how often the model turns out to be right. And the boundary is soft: the paper prints a confidence interval of roughly 10 points either side, and a transition gradual enough that the model still answers some questions below its own bar and declines some above it.

The paper's title uses the word causal, and that word does not reach the numbers above. The phase that produced the threshold and the abstention and accuracy rates is a regression fitted to the model's behavior, and the paper calls it prediction throughout. The causal claim rests on two interventions. In the first, the researchers worked with a model they could open up, the freely available Gemma 3 27B, and pushed its internal confidence signal up and down by editing its activations mid-inference. Abstention moved from 66.5% under maximum suppression to 7.0% under maximum boosting, on 500 held-out questions, against about 28% with no intervention at all. That swing is Gemma's alone. Steering of this kind needs access to a model's internal activations, which a closed model like GPT-4o does not give you.

The second intervention was blunter. GPT-4o was told to answer only when its confidence cleared a stated threshold, and that threshold was raised step by step. Abstention rose with it, and the answers that survived were more accurate, which is the version of the coverage trade the paper backs with a statistical test. So the numbers at the top of this article come from watching the model. The causal weight comes from reaching inside it and from telling it what to do.

The 56.6% abstention rate needs context: the instruction in that abstention round primed caution: it warned that some questions might have no correct answer among the options and told the model it was "better to be safe than sorry." Every question was in fact answerable. Across the four models, abstention ranged from 27% to 82%. So 56.6% belongs to one model, on one task, under one prompt, answering with a single token and no room to think out loud.

Two further details, both disclosed in the paper, show how sensitive this behavior is to framing. A fifth model, Llama 3.1 70B, was tested and then left out of the analyses because it abstained on only 4% of questions. And when the threshold instruction written for GPT-4o barely moved Gemma, the team generated 20 paraphrases of it and ran the experiment with the one that produced the most abstentions.

Underneath the behavior, there seem to be two ways of reading one internal state. Asked separately how likely its own answer was to be correct, each model produced a spoken confidence that was worse at telling right answers from wrong ones than the arithmetic read off its output. That spoken confidence still predicted abstention on its own, over and above the arithmetic. Probing Gemma's activations at the last moment before the answer, the team found both to be partial read-outs of something richer that neither captures.

The authors are careful about what this adds up to. Their results, they write, suggest that abstention is "consistent with structured metacognitive control in large language models," a phrase that means using your own sense of certainty to govern what you do. The caption on the paper's summary figure adds that the pattern does not by itself uniquely establish that reading. What the work puts in a builder's hands is a dial and a price for turning it: a model can be told how sure to be before it speaks, and a more cautious assistant is paid for in questions left unanswered. The paper is open access, and the data and code are posted publicly.