Skip to content
See the World Through ScienceA project of ALLATRA
Source: PreprintarXiv4 sources

Where Pathologists Disagree Most, a Model Was Handed the Same Slides

By Gabriela SzalayováWriterScience5 min read

Republish this story

Our work is licensed under Creative Commons BY-NC 4.0. You may republish this piece for free — with credit to ALLATRA Media and a link to the original, unedited beyond length trims, and not for commercial use.

Read the full license

Microscope view of an esophageal biopsy stained pink and purple, with crowded, elongated gland cells crowding the tissue surface.
This is the grade where expert pathologists agree least, and the one the preprint asked a computer to call from a scanned slide (illustrative)."Low-grade columnar dysplasia of the esophagus -- intermed mag" by Nephron, via Wikimedia Commons, CC BY-SA 3.0, cropped

Two expert gastrointestinal pathologists can look at the same esophageal biopsy, agree that it shows Barrett's esophagus, and still part company on the question that decides what happens next: whether the tissue has begun to show dysplasia, disordered precancerous growth. At the severe end of that scale, the specialists mostly concur. At the mild end, they do not. Among expert pathologists grading Barrett's slides, agreement on low-grade dysplasia falls to 0.42, which the paper's own scale calls weak.

That is the problem a group led from Mayo Clinic set out to hand to a computer. In a preprint posted Nov. 19, 2026, Amirali Khosravi, a radiology researcher at Mayo Clinic in Rochester, Minnesota, and colleagues describe a system that reads a scanned biopsy slide whole and returns one of three grades: no dysplasia, low-grade or high-grade. The corresponding author is Prasad G. Iyer, who chairs the division of gastroenterology and hepatology at Mayo Clinic in Scottsdale, Arizona. The manuscript has not been peer reviewed.

The team pooled 969 stained slides from 961 patients at four American hospitals: Mayo Clinic, Baylor University Medical Center in Dallas, Oregon Health & Science University and the University of Alabama at Birmingham. Two expert pathologists read every slide independently and settled their disagreements by discussion, and that consensus became the answer key. Most of the slides went into training; 130 were held back.

On those held-back slides, the system matched the expert grade on 110 of 130, and its overall ability to tell the three grades apart scored 0.96, where 1 would be perfect. Asked the simpler question of whether a slide showed dysplasia at all, it flagged 80 of the 82 that did, and called nine of the 48 that did not.

Published figure panel labeled NDBE, with a yellow arrow pointing to glandular lining in an esophageal biopsy.
Panel a of a published teaching figure. The arrow marks lining with no dysplasia, the category the preprint's system wrongly flagged nine times out of 48 (illustrative). "Histopathology of non-dysplastic Barret's esophagus" by M. Riegler, I. Kristo, M. Nikolic, E. Rieder, S. F. Schoppmann, via Wikimedia Commons, CC BY 4.0

The step the earlier systems could not skip

The dye that makes a biopsy readable is also the problem. Hematoxylin and eosin, the pink-and-purple stain used in every pathology laboratory, comes out differently depending on the reagents, the protocol and the scanner, and a network trained on one hospital's colors can stumble on another's. The group's own earlier system, tested across three academic centers and published in 2026 in the American Journal of Gastroenterology, needed a separate color-translation model for each institution before it would run there. The authors call that impractical for broad deployment.

This system skips that step. Rather than training an image network from scratch, it borrows one: Midnight-12k, a publicly released pathology foundation model pretrained without labels on more than 12,000 whole-slide images, with color variation deliberately mixed into the pretraining so that what it learns about tissue is less tied to any one laboratory's staining. Khosravi and colleagues froze it and used it only to describe each small square of tissue. Small classifiers then picked out the squares worth looking at, and a second stage weighed those squares into a grade for the whole slide.

Whether the stain robustness is what made the difference was not tested. Stain-robust representations "may remove the site-specific preprocessing that has constrained prior systems to institution-specific deployment," the authors write, and the "may" is carrying weight: the pipeline was never run a second time with color normalization put back, and the backbone was not compared head to head with the other pathology foundation models in circulation. The authors say both of those things plainly, and add that the kind of color augmentation they credit is common across current pathology foundation models, not special to this one.

Four hospitals, none of them new to the model

The sentence that will travel from this paper is that performance was consistent across institutions. It is true as written, and narrower than it sounds. The slides from all four hospitals were pooled and then split at random into training, validation and test sets, stratified by grade and by institution and grouped by patient so that no patient landed on both sides. Every one of the four centers contributed slides to training. The held-out slides are unseen slides from seen hospitals — nothing in the study asks what happens when the system meets a laboratory it has never encountered.

The per-site numbers are small as well. Outside Mayo, which supplied more than half the dataset, each hospital contributed fewer than two dozen test slides, and at the smallest of them accuracy was 71%, the lowest of the four. The statistical test that found no difference between sites was run on those numbers, which makes it a failure to detect a difference, not evidence that there is none. "Geographic restriction to four United States centers limits conclusions about institutions using different tissue processing and staining protocols," the authors write.

The mistakes lean in one direction

Of the held-out slides, the system got 20 wrong, and the pattern of those errors is what the authors dwell on. Fourteen were overcalls and six were undercalls. Nine slides with no dysplasia were graded as having it. Two low-grade slides were called non-dysplastic, and four high-grade slides were graded low-grade. No high-grade slide was called non-dysplastic at all, which the authors single out as the study's most important safety finding, because that is the error that would have sent someone with advanced disease away reassured. They also point at the arithmetic that flatters them: the test set holds the three grades in roughly equal numbers, nothing like a surveillance population, where slides with no dysplasia predominate, and "the absolute frequency of overcalls would likely be higher in clinical practice."

The finding is also older than the posting date. The same work was presented at the American College of Gastroenterology's annual meeting in October 2025; what appeared on arXiv in November 2026 is the full manuscript behind it. The study was paid for by the U.S. Department of Defense and the National Cancer Institute, which the authors say had no role in it, and four of the senior authors declare grant support or consulting arrangements with companies selling Barrett's diagnostics. The code is available from the corresponding author on request and the slides are not shared, so no outside group can rebuild the system from the paper. Two companion studies are underway: a prospective evaluation on 450 low-grade cases from community practice, where overcalling is most common, and a study of how pathologists' own readings change when the model's answer sits in front of them.

Sources

Medical Disclaimer: This content is provided for general informational and educational purposes only. It is not medical advice and should not be used as a substitute for professional medical advice, diagnosis, or treatment. Always consult a qualified healthcare professional regarding any medical condition, concern, symptom, or treatment decision.

Spot an error?

Spot an error?

Report an error

Spotted a mistake on this page? Tell us what's wrong and our editors will take a look.

What kind of problem?

Only if you'd like us to be able to follow up. We won't use it for anything else.

We correct mistakes openly. Select any text to flag it. Fixes are logged under our Corrections Policy.

Report an error

Reporting on

Where Pathologists Disagree Most, a Model Was Handed the Same Slides

What kind of problem?

Only if you'd like us to be able to follow up. We won't use it for anything else.

We read every report. Corrections are logged publicly.