Two AI Agents, One Going Broad and One Going Deep, Mine Biomedicine's Tangled Data

Anyone who has tried to answer a serious biomedical question knows the problem is rarely a shortage of information. It is that the information sits in too many places at once: one database for genes, another for drugs, a third for diseases, a fourth for the clinical-trial record, each built to its own logic, none quite speaking the others' language. Stitching a coherent line of evidence across all of them is slow, expert work, and it is exactly the kind of connect-the-dots labor that a well-directed AI system might, in principle, accelerate.

DeepEvidence, described in a paper published on July 2 in Nature Machine Intelligence, is one attempt to build that system. Its organizing idea is a division of labor between two agents run by an orchestrator. One, which the authors call breadth-first research, ranges wide, casting across many knowledge graphs to gather the entities that might be relevant. The other, depth-first research, goes narrow and deep, following multi-hop chains of reasoning through the evidence to trace how one fact connects to the next. A shared evidence graph keeps track of what has been found and how the pieces link. The pairing is meant to mirror how a good human researcher actually works: survey the landscape, then drill where it matters.

The authors position the tool across the arc of biomedical discovery (from drug discovery and pre-clinical experimentation through clinical-trial development to evidence-based medicine), and they report accuracy gains over strong baselines, including recent general-purpose models and other biomedical agent systems. On the figures the team publishes, the improvements are large: on one medical-reasoning benchmark the system is reported at roughly double the accuracy of its comparators, and on a literature question-answering task the reported margin is wider still.

Here is where the reading has to slow down, because the framing matters more than any single number. Those figures are self-reported. The team ran its own evaluation, and the gains rest partly on six benchmark tasks the authors curated themselves, a setup that leaves the choice of tasks and the choice of baselines in the same hands as the system being tested. That is not an accusation of anything; it is the structural reason self-run benchmarks carry less weight than independent ones. When the people who build the exam also write the questions, an outside proctor is exactly what is missing.

A second gap compounds the first. The paper does not discuss data contamination: the live worry, for any evaluation of a large-language-model agent, that the benchmark's answers leaked into the models' training data, so the system may be recalling rather than reasoning. On public biomedical benchmarks that risk is real and routinely material, and a result that does not address it cannot be read as clean. There is, so far, no independent evaluation or replication to settle the matter either way.

None of this makes DeepEvidence uninteresting. The two-agent, broad-then-deep architecture is a sensible answer to a genuine problem, and peer review at a respected venue is a real check on the method's description. But a self-reported number on a home-built benchmark is a claim, not a scorecard. It should be held as one until an outside group runs the test on data the authors did not choose. That independent check, and a contamination audit, are the two things that would move this from a promising demonstration to a measured capability. Until then, treat the accuracy figures as the team's own report of its own system.