AI Tool Accurately Identifies Precancerous Stomach and Esophagus Conditions From Health Records

A natural language processing (NLP) algorithm was successfully developed and validated to accurately identify esophageal and gastric precancerous conditions and cancers from unstructured electronic health records (EHR). According to a study published in Gastro Hep Advances last month, researchers relied on the VA Million Veteran Program (VA MVP), where their rule-based algorithm achieved excellent accuracy (ranging from 97.5% to 100%) in identifying conditions such as Barrett’s esophagus (BE) and gastric intestinal metaplasia (GIM) from pathology reports.

Gastric and esophageal cancers remain among the leading causes of cancer mortality worldwide. Both gastric adenocarcinoma and esophageal adenocarcinoma are typically preceded by identifiable precancerous conditions, GIM and BE, that can be detected histologically and monitored through endoscopic surveillance. Early detection of these lesions offers the best chance to intervene before malignant transformation, yet current clinical practice faces two major barriers.

A groundbreaking NLP algorithm accurately identifies precancerous gastric and esophageal conditions from EHRs, enhancing early detection and personalized patient care. © Kevin - stock.adobe.com

First, the ability to predict which patients with GIM or BE will progress to cancer is limited—hampering targeted surveillance. Secondly, large-scale epidemiologic studies that could refine risk models are constrained by the difficulty of accurately identifying histologically confirmed precancerous conditions from routine health records. Structured data, such as ICD codes, are often incomplete or inaccurate; GIM, for example, lacked a specific ICD code until late 2021 and cannot capture key histologic or anatomic details essential for risk stratification.

To address this gap, Shailja Shah, M.D., M.P.H., from the division of Gastroenterology and Hepatology, University of California, and colleagues developed and validated a rule-based NLP algorithm capable of extracting detailed, histologically confirmed gastroesophageal precancer and cancer diagnoses from unstructured pathological reports. The work leveraged the VA MVP, a nationwide genomic biobank linked to comprehensive longitudinal EHRs for over 12 million U.S. veterans.

From the MVP cohort, the team identified 121,808 individuals who had undergone esophagogastroduodenoscopy with gastric and/or esophageal biopsies. Pathology reports mentioning relevant anatomic sites were compiled, and 426 reports were manually annotated by an expert gastroenterologist to identify target conditions, intestinal metaplasia, dysplasia and cancer, along with anatomic subsites (e.g., gastric antrum, body and cardia) and clinically relevant “qualifiers” such as dysplasia grade or GIM subtype.

The authors iteratively developed a rule-based NLP pipeline on 100 annotated reports, refining it through six development rounds. The remaining 326 reports served as the validation set. Performance metrics included accuracy, precision, recall, F1 score, specificity and negative predictive value (NPV), with a pre-specified goal of >90% accuracy lower bound after Bonferroni correction.

The algorithm achieved excellent accuracy, 97.5% to 100%, across all primary condition-location pairs. For BE, precision and recall were both 98.9%, with 99.6% specificity and NPV. For GIM, precision was 91.7% and recall 86.8% (F1 score 89.2%), with 99.0% specificity and 98.3% NPV. Gastric dysplasia and gastric cancer were identified with perfect (100%) performance across all metrics. Esophageal dysplasia and cancer also showed high sensitivity and specificity, though precision was modestly lower (71–83%) due to a small number of false positives.

When applied to the full MVP EGD cohort, the algorithm identified GIM in 13.2% (n=16,038; mean age 65.1 years) and BE in 14.5% (n=17,700; mean age 63.9 years). Nearly half of GIM patients and over half of BE patients had at least one follow-up EGD with biopsies, enabling longitudinal outcome tracking. The tool also accurately captured anatomic subsites and histologic qualifiers, which are critical for risk stratification but rarely available in structured data.

This research demonstrates that high-fidelity phenotyping of gastroesophageal precancerous conditions is feasible at scale using NLP applied to unstructured pathology data. For managed care organizations, the implications are significant. For risk stratification, by capturing granular histologic and anatomic details, the algorithm enables more precise identification of high-risk patients who may benefit from intensified surveillance, while avoiding unnecessary procedures in low-risk individuals.

From a population health management standpoint, linking these phenotypes to longitudinal EHR and genomic data supports the development of robust prediction models incorporating genetic, environmental and clinical factors. An efficient NLP model supports resource optimization for health systems, as it may allow for accurate case identification to allocate endoscopic resources more efficiently, focusing on those most likely to progress to cancer.

The validated NLP pipeline offers a scalable, accurate method to identify and characterize gastric and esophageal precancer across large health systems. Its integration into managed care analytics could transform surveillance strategies, support personalized prevention and ultimately reduce the burden of gastroesophageal cancer.