Researchers Can Now Measure a Model's Bias in How It Thinks, Not Just What It Says

Sit a person at a keyboard and ask them to sort words as fast as they can: flowers here, insects there, then pleasant words here and unpleasant ones there. And something small and stubborn shows up in the timing. When the categories are paired the way an association in that person's head already runs, the sorting goes quickly. When the pairing cuts across it, the same person slows down. The gap lives in the pauses rather than in anything the person says. That is the Implicit Association Test, the standard laboratory handle on unconscious bias in people.

A paper published today in Nature Machine Intelligence asks what that test measures when it is pointed at something with no fingers and no reaction time, but with a pause you can count anyway. Reasoning models, the language models that generate explicit step-by-step reasoning before producing an answer, spend a variable amount of that reasoning on any given problem, and the tokens are countable. Messi H. J. Lee, of the Division of Computational and Data Sciences at Washington University in St. Louis, treats that count as the machine's version of hesitation. So does his co-author Calvin K. Lai, a psychologist at Rutgers whose earlier work is on measuring implicit bias in people.

Their method, the reasoning-model implicit association test, borrows the human test's logic wholesale. A model is given a sorting task in which the categories are paired the way a stereotype runs (the association-compatible condition), and then the same task with the pairing reversed. Nobody asks the model whether it holds a bias, and the answer it produces is not the measurement. The measurement is how much reasoning it spent getting there.

Across four models (o3-mini, DeepSeek-R1, gpt-oss-20b and Qwen3-8B), the authors report consistent evidence that the incompatible pairings demanded greater computational effort than the compatible ones. Thinking against the grain of an association cost the machine more, in the one currency a reasoning model has to spend.

The fifth model went the other way. Claude 3.7 Sonnet showed reversed patterns, and to ask why, the authors coded what the models had written while reasoning and looked for themes. That analysis links the reversal to something particular about this model: an internal focus on reasoning about bias and stereotypes. Linked is the operative word.

A theme recovered from reasoning text is an association between two things the researchers observed, not a mechanism they demonstrated. The reversal sits alongside the main finding from the outset, making the split part of the result rather than a footnote. What was measured is the behavior of five named models at particular versions on a particular set of tasks, in a field where a model can be retrained between one study and the next. It is not a property of reasoning models as a class.

The authors also looked for evidence that the measure tracks something beyond itself. RM-IAT effects, they report, predicted biases in two tasks already known to capture bias in language models: one in word association, one in decision-making. That is what gives the method its stakes. Something read out of a model's private reasoning, before an answer exists, carried information about the answers it went on to produce.

That result gives the measure its significance, but its meaning comes from what the measure actually captures. The paper reports bias-like patterns, not bias. A reasoning-token count is an index of computational effort, offered as the analog of response latency in the human test: a proxy for how hard a system is working, not a view into anything mental. The human version of the test has spent decades under precisely that argument about what its milliseconds mean, and the machine version inherits the argument along with the design.

What the work hands other researchers is a handle on the part of a model that auditing has mostly treated as exhaust. Bias evaluations read what a system says; this one reads what it does before it says anything. The data sit on Figshare and the collection and analysis code is on GitHub under an MIT license, with an archived release and a reproducible capsule alongside it, so another group can point the same comparison at models these five do not cover. Given that the fifth went its own way, that is the obvious next thing to do.