AI Health Agents Rarely Beat a Baseline That Ignores the Sensor Data

One program in this benchmark never looks at the data. It ignores the step counts, the sleep hours and the resting heart rates, and returns the same number every time: the average score in the data it was trained on. The researchers included it as a reference point, the floor that anything cleverer ought to clear. In the plainest test they ran, it beat almost all of the artificial intelligence built to do better.

The benchmark is called BALMS, and it was posted to arXiv on Aug. 27 by Yu Yvonne Wu, Andrew Campbell and colleagues at Dartmouth College. It runs prompt-based, tool-based and memory-based agents on top of open and closed language models, over three real datasets of phone and wearable records. Every run asks for one number: the score a participant would write on a wellbeing questionnaire that day. Daily mood on the first dataset. Daily stress, rated one to five, on the second. On the third, the anxiety half of a short self-report questionnaire. Nothing here predicts a diagnosis, and nothing here was used in anyone's care.

Scored the plainest way, with no chain-of-thought prompting and no task-specific training, five of the 60 combinations of agent, model and dataset beat that mean predictor. All five landed on the same dataset. On the other two, the count was zero of 20 each.

The measure is mean absolute error: how far the agent's guess falls from what the person actually wrote, in points on the questionnaire's own scale. Lower is better. It is not an accuracy rate and it does not convert into a percentage. On the mood dataset the mean predictor sat at 0.58, and the best agent in the whole table, Claude Haiku 4.5 driving the simplest prompt-based design, reached 0.42.

A constant answer is close to right on most days

That is the deflating part of this result, and also the ordinary one. People rate themselves near their own usual number most of the time, so a fixed guess is already nearly right, and an agent has to catch the days that depart from the pattern to earn its place. The authors write that the agents do extract useful behavioral signal, and that "much of the predictive power in daily self-reports is still captured by simple dataset-level priors."

The mood dataset follows university students for 28 days, and the evaluation uses the single country cohort that answered its daily prompts most reliably. The stress dataset is five months of Fitbit records from 16 people. The third comes from a multi-year college study, but the authors evaluate one continuous 10-week term and only the anxiety subscale, because participants and behavioral baselines shift between terms. Weeks to months, in other words, from small or narrow groups.

The reasoning was fluent, and often about numbers nobody computed

The tool-based agent, the one that writes and runs Python over a person's records, turned out to be answering with the same label again and again, giving the identical anxiety score on 86% of days whatever the sensors showed. Its error on that dataset was nearly twice the baseline's.

The execution traces show how it got there. The model would emit code with no print statement, receive "Code executed (no output)" back, and carry on as though the calculation had succeeded. Variables defined in one step were out of scope by the next. When the step budget ran out, the harness asked for a final answer, and the model wrote a confident rationale citing trends and differences it had never computed. That is how most of the traces the authors inspected by hand ended. It also invented thresholds it had not derived from anybody's history, so nearly every user fell on the same side of them.

Asking the models to reason step by step helps some of them. On the reasoning-tuned DeepSeek model and on Claude it lowered error across most configurations, by as much as 41.4% in one case. On Mistral, the weakest backbone in the set, it did the reverse, roughly doubling the error in several cells.

The abstract and the conclusion are not saying quite the same thing

BALMS's abstract reports that zero-shot agents "rarely outperform a simple mean baseline." Its conclusion reports that they "can achieve competitive wellbeing-score prediction without task-specific training, but reliable performance depends on how agents access and ground longitudinal evidence." Taken together, the results point to a mixed picture: zero-shot agents can make competitive wellbeing-score predictions without task-specific training, but their performance is inconsistent and often fails to beat a simple mean baseline.

The reported scores include no error bars, multiple seeds or significance tests, so differences of a few hundredths of a point cannot be treated as meaningful on their own. The overall pattern is more informative than any single score.

The benchmark's second task asks each agent to justify its score, and those justifications are graded by another language model rather than by clinicians, against a rubric for temporal reasoning and general quality. The authors flag it themselves: the setup "does not replace expert clinical or human-subject evaluation." Those scores are worth reading beside the appendix, because one system was marked near the top for clarity and near the bottom for whether its prediction followed from the reasoning offered for it.

The team's other cautions are unusually direct. All three cohorts are students or small convenience samples, and the authors "strongly caution against generalizing these specific zero-shot weights to broader, heterogeneous clinical settings without proactive calibration and local alignment." They describe such systems as clinical decision-support tools, "not as autonomous diagnostic replacements for professional medical care." The whole study is retrospective, run over de-identified research datasets; nobody was monitored and nothing was deployed.

What the work puts in someone's hands is the floor. A team building an agent to read wellbeing off a wearable now has an open test, three public datasets and one line to clear before anything else, which is the score you get by ignoring the sensors altogether.