Purja Pun S, Obst O, Basilakis J
… +1 more, Ginige JA
J Am Med Inform Assoc
· 2026 Apr · PMID 41615922
·
Full text
OBJECTIVES: Mapping clinical classification systems, such as the International Classification of Diseases (ICD), is essential yet challenging. While the manual mapping method remains labor-intensive and lacks scalability...OBJECTIVES: Mapping clinical classification systems, such as the International Classification of Diseases (ICD), is essential yet challenging. While the manual mapping method remains labor-intensive and lacks scalability, existing embedding-based automatic mapping methods, particularly those leveraging transformer-based pretrained encoders, encounter 2 persistent challenges: (1) linguistic variation and (2) varying granular details in clinical conditions. MATERIALS AND METHODS: We introduce an automatic mapping method that combines the representational power of pretrained encoders with the reasoning capability of large language models (LLMs). For each ICD code, we generate: (1) hierarchy-augmented (HA) and (2) LLM-generated (LG) descriptions to capture rich semantic nuances, addressing linguistic variation. Furthermore, we introduced a prompting framework (PR) that leverages LLM reasoning to handle granularity mismatches, including source-to-parent mappings. RESULTS: Chapterwise mappings were performed between ICD versions (ICD-9-CM↔ICD-10-CM and ICD-10-AM↔ICD-11) using multiple LLMs. The proposed approach consistently outperformed the baseline across all ICD pairs and chapters. For example, combining HA descriptions with Qwen3-8B-generated descriptions yielded an average top-1 accuracy improvement of 6.5% (0.065) across the mapping cases. A small-scale pilot study further indicated that HA+LG remains effective in more challenging one-to-many mappings. CONCLUSIONS: Our findings demonstrate that integrating the representational power of pretrained encoders with LLM reasoning offers a robust, scalable strategy for automatic ICD mapping.
Mazumder R, Keeney J, Johnson L
… +18 more, Krammer L, McNeely P, Sepulveda J, Hangen D, Martin M, Jyothi D, De Almeida J, McGarvey P, Alaoui A, Cha S, Sedrakyan A, Shoelle E, Matheny M, LeNoue-Newton M, Winter R, Deppen S, Simonyan V, Horvath A
J Am Med Inform Assoc
· 2026 Apr · PMID 41556955
·
Full text
OBJECTIVES: Federated Ecosystems for Analytics and Standardized Technologies (FEAST) is a modular, cloud-based platform developed through the ARPA-H Biomedical Data Fabric initiative to enable secure, federated analysis...OBJECTIVES: Federated Ecosystems for Analytics and Standardized Technologies (FEAST) is a modular, cloud-based platform developed through the ARPA-H Biomedical Data Fabric initiative to enable secure, federated analysis of real-world biomedical data. To guide and iteratively refine its modular design, the FEAST team conducted a cross-institutional survey to systematically identify and prioritize research needs related to authorized-access data across diverse biomedical domains. This study presents a structured synthesis of submitted use cases to uncover infrastructure gaps, data integration challenges, and translational opportunities. The results from the survey inform both front-end user-facing functionality and backend data requirements, shaping how the interface supports user interactions, data types, and compliance with security and interoperability standards. MATERIALS AND METHODS: A structured survey form was distributed to researchers affiliated with participating institutions, including DNA-HIVE, The George Washington University (GW-FEAST), Weill Cornell Medicine, Vanderbilt University Medical Center, Georgetown University, European Bioinformatics Institute, and Kaiser Permanente. Respondents completed standardized fields describing the data types of interest, project goals, analytic methods, and perceived technical barriers. The collected responses were curated and analyzed to identify common needs related to privacy, interoperability, scalability, and workflow reproducibility. RESULTS: The survey compiled 61 use cases spanning genomics, imaging, clinical phenotyping, EHR-driven analytics, and precision medicine. Common themes included the need for multi-modal data integration, HL7 FHIR-based secure access, federated model training without PII retention, and containerized microservices for scalable deployment. Convergent needs across institutions emphasized consistent demand for FAIR-compliant infrastructure and readiness for real-world data analytics. CONCLUSION: The FEAST Use Cases survey provides a cross-sectional view of biomedical informatics priorities grounded in real-world data needs. The findings offer a strategic blueprint for developing federated, privacy-preserving infrastructure to support secure, collaborative, and scalable biomedical research.
Kashyap A, Allsman CJ, Campbell EA
… +7 more, Desai PM, Volpe SG, Massey BP, Bright TJ, Bakken S, Bear Don't Walk Iv OJ, Pichon A
J Am Med Inform Assoc
· 2026 Apr · PMID 41553008
·
Full text
OBJECTIVES: Advancing health through informatics requires attending to justice. Recent policy changes in the United States have introduced significant barriers to promoting justice within informatics due to targeted fund...OBJECTIVES: Advancing health through informatics requires attending to justice. Recent policy changes in the United States have introduced significant barriers to promoting justice within informatics due to targeted funding cuts and hostility to science, especially science that prioritizes justice. MATERIALS AND METHODS: We present five key principles for advancing a justice-oriented informatics agenda, synthesized from our workshop held at the American Medical Informatics Association 2022 Annual Symposium. RESULTS: These principles are: (1) Recognize knowledge and methodologies across communities; (2) Acknowledge historical and cultural contexts of interactions; (3) Facilitate transparency and accountability through clear measures and metrics; (4) Foster trust and sustainability; and (5) Equitably allocate compensation and resources. DISCUSSION AND CONCLUSION: We discuss barriers to implementing these principles that have arisen since the 2022 workshop and provide recommendations for moving towards justice-oriented informatics. We offer examples of how these principles may be used to frame challenges and adapt to new barriers within BMI.
J Am Med Inform Assoc
· 2026 Apr · PMID 41552993
·
Full text
PURPOSE: To highlight the importance of reporting negative results in large language model (LLM) research, particularly as these systems are increasingly integrated into healthcare. POTENTIAL: LLMs offer transformative c...PURPOSE: To highlight the importance of reporting negative results in large language model (LLM) research, particularly as these systems are increasingly integrated into healthcare. POTENTIAL: LLMs offer transformative capabilities in text generation, summarization, and clinical decision support. Transparent documentation of both successes and failures can accelerate innovation, improve reproducibility, and guide safe deployment. CAUTION: Publication bias toward positive findings conceals model limitations, biases, and reproducibility challenges. In healthcare, underreporting failures risks patient safety, ethical lapses, and wasted resources. Structural barriers, including a lack of standards and limited funding for failure analysis, perpetuate this cycle. CONCLUSIONS: Negative results should be recognized as valuable contributions that delineate the boundaries of LLM applicability. Structured reporting, educational initiatives, and stronger incentives for transparency are essential to ensure responsible, equitable, and trustworthy use of LLMs in healthcare.
Hu Y, Zuo X, Zhou Y
… +10 more, Peng X, Huang J, Keloth VK, Zhang VJ, Weng RL, Shyr C, Chen Q, Jiang X, Roberts KE, Xu H
J Am Med Inform Assoc
· 2026 Mar · PMID 41533750
·
Full text
OBJECTIVES: To assess the performance, generalizability, and computational efficiency of instruction-tuned Large Language Model Meta AI (LLaMA)-2 and LLaMA-3 models compared to bidirectional encoder representations from...OBJECTIVES: To assess the performance, generalizability, and computational efficiency of instruction-tuned Large Language Model Meta AI (LLaMA)-2 and LLaMA-3 models compared to bidirectional encoder representations from transformers (BERT) for clinical information extraction (IE) tasks, specifically named entity recognition (NER) and relation extraction (RE). MATERIALS AND METHODS: We developed a comprehensive annotated corpus of 1588 clinical notes from 4 data sources-UT Physicians (UTP) (1342 notes), Transcribed Medical Transcription Sample Reports and Examples (MTSamples) (146), Medical Information Mart for Intensive Care (MIMIC)-III (50), and Informatics for Integrating Biology and the Bedside (i2b2) (50), capturing 4 clinical entities (problems, tests, medications, other treatments) and 16 modifiers (eg, negation, certainty). Large Language Model Meta AI-2 and LLaMA-3 were instruction-tuned for clinical NER and RE, and their performance was benchmarked against BERT. RESULTS: Large Language Model Meta AI models consistently outperformed BERT across datasets. In data-rich settings (eg, UTP), LLaMA achieved marginal gains (approximately 1% improvement for NER and 1.5%-3.7% for RE). Under limited data conditions (eg, MTSamples, MIMIC-III) and on the unseen i2b2 dataset, LLaMA-3-70B improved F1 scores by over 7% for NER and 4% for RE. However, performance gains came with increased computational costs, with LLaMA models requiring more memory and Graphics Processing Unit (GPU) hours and running up to 28 times slower than BERT. DISCUSSION: While LLaMA models offer enhanced performance, their higher computational demands and slower throughput highlight the need to balance performance with practical resource constraints. Application-specific considerations are essential when choosing between LLMs and BERT for clinical IE. CONCLUSION: Instruction-tuned LLaMA models show promise for clinical NER and RE tasks. However, the tradeoff between improved performance and increased computational cost must be carefully evaluated. We release our Kiwi package (https://kiwi.clinicalnlp.org/) to facilitate the application of both LLaMA and BERT models in clinical IE applications.
J Am Med Inform Assoc
· 2026 Mar · PMID 41528460
·
Full text
OBJECTIVE: Accurate phenotyping is an essential task for researchers utilizing electronic health record (EHR)-linked biobank programs like the All of Us Research Program to study human genetics. However, little guidance...OBJECTIVE: Accurate phenotyping is an essential task for researchers utilizing electronic health record (EHR)-linked biobank programs like the All of Us Research Program to study human genetics. However, little guidance is available on how to select an EHR-based phenotyping procedure that maximizes downstream statistical power. This study aims to estimate accuracy of three phenotype definitions of ovarian, female breast, and colorectal cancers in All of Us (v7 release) and determine which is most likely to optimize downstream statistical power for genetic association testing. MATERIALS AND METHODS: We used empirical carrier frequencies of deleterious variants in known risk genes to estimate the accuracy of each phenotype definition and compute statistical power after accounting for the probability of outcome misclassification. RESULTS: We found that the choice of phenotype definition can have a substantial impact on statistical power for association testing and that no approach was optimal across all tested diseases. The impact on power was particularly acute for rarer diseases and target risk alleles of moderate penetrance or low frequency. Additionally, our results suggest that the accuracy of higher-complexity phenotyping algorithms is inconsistent across Black and non-Hispanic White participants in All of Us, highlighting the potential for case ascertainment biases to impact downstream association testing. DISCUSSION: EHR-based phenotyping presents a bottleneck for maximizing power to detect novel risk alleles in All of Us, as well as a potential source of differential outcome misclassification that researchers should be aware of. We discuss the implications of this as well as potential mitigation strategies.
Cross DA, Weiner J, Neprash HT
… +2 more, Melton GB, Olson A
J Am Med Inform Assoc
· 2026 Mar · PMID 41528439
·
Full text
OBJECTIVE: To characterize the nature and consequence(s) of interdependent physician electronic health record (EHR) work across inpatient shifts. MATERIALS AND METHODS: Pooled cross-sectional analysis of EHR metadata ass...OBJECTIVE: To characterize the nature and consequence(s) of interdependent physician electronic health record (EHR) work across inpatient shifts. MATERIALS AND METHODS: Pooled cross-sectional analysis of EHR metadata associated with hospital medicine patients at an academic medical center, January-June 2022. Using patient-day observation data, we use a mixed effects regression model with daytime physician random effects to examine nightshift behavior (handoff time, total EHR time) as a function of behaviors by the preceding daytime team. We also assess whether nighttime patient deterioration is predicted by team coordination behaviors across shifts. RESULTS: We observed 19 671 patient days (N = 2708 encounters). Physicians used the handoff tool consistently, generally spending 8-12 minutes per shift editing patient information. When the day service team was more activated (highest tercile of handoff time, overall EHR time), nightshift experienced increased levels of EHR work and patient risk of overnight decline was elevated. (ie, Busy predicts busy). However, lower levels of dayshift activation were also associated with nightshift spillovers, including higher overnight EHR work and increased likelihood of patient clinical decline. Patient-days in the lowest and highest terciles of dayshift EHR time had a 1 percentage point increased relative risk of overnight decline (baseline prevalence of 4.4%) compared to the middle tercile (P = .04). DISCUSSION: We find evidence of spillovers in EHR work from dayshift to nightshift. Additionally, the lowest and highest levels of dayshift EHR activity are associated with increased risk of overnight patient decline. Results are associational and motivate further examination of additional confounding factors. CONCLUSION: Analyses reveal opportunities to address task interdependence across shifts, using technology to flexibly shape and support collaborative teaming practices in complex clinical environments.
Brown KE, Wrenn JO, Jackson NJ
… +5 more, Cauley MR, Collins BX, Novak LL, Malin BA, Ancker JS
J Am Med Inform Assoc
· 2026 Mar · PMID 41528321
·
Full text
OBJECTIVE: Healthcare decisions are increasingly made with the assistance of machine learning (ML). ML has been known to have unfairness-inconsistent outcomes across subpopulations. Clinicians interacting with these syst...OBJECTIVE: Healthcare decisions are increasingly made with the assistance of machine learning (ML). ML has been known to have unfairness-inconsistent outcomes across subpopulations. Clinicians interacting with these systems can perpetuate such unfairness by overreliance. Recent work exploring ML suppression-silencing predictions based on auditing the ML-shows promise in mitigating performance issues originating from overreliance. This study aims to evaluate the impact of suppression on collaboration fairness and evaluate ML uncertainty as desiderata to audit the ML. MATERIALS AND METHODS: We used data from the Vanderbilt University Medical Center electronic health record (n = 58 817) and the MIMIC-IV-ED dataset (n = 363 145) to predict likelihood of death or intensive care unit transfer and likelihood of 30-day readmission using gradient-boosted trees and an artificially high-performing oracle model. We derived clinician decisions directly from the dataset and simulated clinician acceptance of ML predictions based on previous empirical work on acceptance of clinical decision support alerts. We measured performance as area under the receiver operating characteristic curve and algorithmic fairness using absolute averaged odds difference. RESULTS: When the ML outperforms humans, suppression outperforms the human alone (P < 8.2 × 10-6) and at least does not degrade fairness. When the human outperforms the ML, the human is either fairer than suppression (P < 8.2 × 10-4) or there is no statistically significant difference in fairness. Incorporating uncertainty quantification into suppression approaches can improve performance. CONCLUSION: Suppression of poor-quality ML predictions through an auditor model shows promise in improving collaborative human-AI performance and fairness.
Du X, Zhou Z, Wang Y
… +11 more, Chuang YW, Li Y, Yang R, Zhang W, Wang X, Chen X, Guan H, Lian J, Hong P, Bates DW, Zhou L
J Am Med Inform Assoc
· 2026 Mar · PMID 41528313
·
Full text
BACKGROUND: The use of generative large language models (LLMs) with electronic health record (EHR) data is rapidly expanding to support clinical and research tasks. This systematic review characterizes the clinical field...BACKGROUND: The use of generative large language models (LLMs) with electronic health record (EHR) data is rapidly expanding to support clinical and research tasks. This systematic review characterizes the clinical fields and use cases that have been studied and evaluated to date. METHODS: We followed the Preferred Reporting Items for Systematic Review and Meta-Analyses guidelines to conduct a systematic review of articles from PubMed and Web of Science published between January 1, 2023, and November 9, 2024. Studies were included if they used generative LLMs to analyze real-world EHR data and reported quantitative performance evaluations. Through data extraction, we identified clinical specialties and tasks for each included article, and summarized evaluation methods. RESULTS: Of the 18 735 articles retrieved, 196 met our criteria. Most studies focused on radiology (26.0%), oncology (10.7%), and emergency medicine (6.6%). Regarding clinical tasks, clinical decision support made up the largest proportion of studies (62.2%), while summarizations and patient communications made up the smallest, at 5.6% and 5.1%, respectively. In addition, GPT-4 and GPT-3.5 were the most commonly used generative LLMs, appearing in 60.2% and 57.7% of studies, respectively. Across these studies, we identified 22 unique non-NLP metrics and 35 unique NLP metrics. While NLP metrics offer greater scalability, none demonstrated a strong correlation with gold-standard human evaluations. CONCLUSION: Our findings highlight the need to evaluate generative LLMs on EHR data across a broader range of clinical specialties and tasks, as well as the urgent need for standardized, scalable, and clinically meaningful evaluation frameworks.
Linares M, Rodriguez JA, Wisk LE
… +3 more, Bell DS, Brown A, Casillas A
J Am Med Inform Assoc
· 2026 Mar · PMID 41528308
·
Full text
Using 2023-2024 U.S. National Health Interview Survey data, we found that digital health literacy (dHL) mediated nearly half of the difference in telehealth use between Latino adults with non-English and English language...Using 2023-2024 U.S. National Health Interview Survey data, we found that digital health literacy (dHL) mediated nearly half of the difference in telehealth use between Latino adults with non-English and English language preference. These findings identify dHL as a modifiable mechanism linking linguistic and digital access barriers, underscoring the need for multilingual, inclusive, and equitable telehealth design.
Del Fiol G, Borsato E, Bradshaw RL
… +20 more, Bian J, Woodbury A, Gauchel C, Eilbeck KL, Maxwell W, Ellis K, Madeo AC, Schlechter C, Kukhareva PV, Allen CG, Kean M, Elkin EB, Sharaf R, Ahsan MD, Frey M, Davis-Rivera L, Kohlmann WK, Wetter DW, Kaphingst KA, Kawamoto K
J Am Med Inform Assoc
· 2026 Mar · PMID 41524720
·
Full text
BACKGROUND: Chatbots are increasingly used to deliver health education, patient engagement, and access to healthcare services. GARDE-Chat is an open-source platform designed to facilitate the development, deployment, and...BACKGROUND: Chatbots are increasingly used to deliver health education, patient engagement, and access to healthcare services. GARDE-Chat is an open-source platform designed to facilitate the development, deployment, and dissemination of chatbot-based digital health interventions across different domains and settings. MATERIALS AND METHODS: GARDE-Chat was developed through an iterative process informed by real-world use cases to guide prioritization of key features. The tool was developed as an open-source platform to promote collaboration, broad dissemination, and impact across research and clinical domains. RESULTS: GARDE-Chat's main features include (1) a visual authoring interface that allows non-programmers to design chatbots; (2) support for scripted, large language model (LLM)-based and hybrid chatbots; (3) capacity to share chatbots with researchers and institutions; (4) integration with external applications and data sources such as electronic health records and REDCap; (5) delivery via web browsers or text messaging; and (6) detailed audit log supporting analyses of chatbot user interactions. Since its first release in July 2022, GARDE-Chat has supported the development of chatbot-based interventions tested in multiple studies, including large pragmatic clinical trials addressing topics such as genetic testing, COVID-19 testing, tobacco cessation, and cancer screening. DISCUSSION: Ongoing challenges include the effort required for developing chatbot scripts, ensuring safe use of LLMs, and integrating with clinical systems. CONCLUSION: GARDE-Chat is a generalizable platform for creating, implementing, and disseminating scalable chatbot-based population health interventions. It has been validated in several studies, and it is available to researchers and healthcare systems through an open-source mechanism.
J Am Med Inform Assoc
· 2026 Mar · PMID 41520192
·
Full text
BACKGROUND: Despite rapid integration into clinical decision-making, clinical large language models (LLMs) face substantial translational barriers due to insufficient structural characterization and limited external vali...BACKGROUND: Despite rapid integration into clinical decision-making, clinical large language models (LLMs) face substantial translational barriers due to insufficient structural characterization and limited external validation. OBJECTIVE: We systematically map the clinical LLM research landscape to identify key structural patterns influencing their readiness for real-world clinical deployment. METHODS: We identified 73 clinical LLM studies published between January 2020 and March 2025 using a structured evidence-mapping approach. To ensure transparency and reproducibility in study selection, we followed key principles from the PRISMA 2020 framework. Each study was categorized by clinical task, base architecture, alignment strategy, data type, language, study design, validation methods, and evaluation metrics. RESULTS: Studies often addressed multiple early stage clinical tasks-question answering (56.2%), knowledge structuring (31.5%), and disease prediction (43.8%)-primarily using text data (52.1%) and English-language resources (80.8%). GPT models favored retrieval-augmented generation (43.8%), and LLaMA models consistently adopted multistage pretraining and fine-tuning strategies. Only 6.9% of studies included external validation, and prospective designs were observed in just 4.1% of cases, reflecting significant gaps in translational reliability. Evaluations were predominantly quantitative only (79.5%), though qualitative and mixed-method approaches are increasingly recognized for assessing clinical usability and trustworthiness. CONCLUSION: Clinical LLM research remains exploratory, marked by limited generalizability across languages, data types, and clinical environments. To bridge this gap, future studies must prioritize multilingual and multimodal training, prospective study designs with rigorous external validation, and hybrid evaluation frameworks combining quantitative performance with qualitative clinical usability metrics.
Basu S, Patel SY, Sheth P
… +4 more, Muralidharan B, Elamaran N, Kinra A, Batniji R
J Am Med Inform Assoc
· 2026 Mar · PMID 41460187
·
Full text
OBJECTIVE: Population health management programs coordinate care for over 80 million Medicaid beneficiaries but lack systematic clinical decision support for determining when to intervene and which interventions to selec...OBJECTIVE: Population health management programs coordinate care for over 80 million Medicaid beneficiaries but lack systematic clinical decision support for determining when to intervene and which interventions to select for patients with complex conditions. Our objective was to develop and validate a clinical decision support system integrating acuity prediction and intervention selection models for population health management programs. MATERIALS AND METHODS: We conducted a retrospective cohort study of 155 631 Medicaid patients enrolled in population health programs across Washington, Virginia, and Ohio (January 2023-July 2025). We developed integrated informatics workflows combining time-to-event prediction models for acute care events with heterogeneous treatment effect estimators for intervention selection. Models used structured electronic health record data, claims, and care management records. Performance was evaluated through clinical validation with 3 blinded physicians reviewing 200 cases. RESULTS: The integrated decision support system achieved 81.3% sensitivity (95% CI, 79.8%-82.8%) and 82.1% specificity (95% CI, 80.6%-83.6%) for 30-day acute care prediction. The intervention selection component demonstrated 1.59 percentage points absolute risk reduction compared with standard care (95% CI, 0.21-3.04), translating to preventing one acute event for every 63 patients receiving model-guided rather than standard care. Clinical validation revealed systematic differences: physicians relied on recent utilization patterns (explaining 75.8% of decision variance) while models integrated broader clinical signals, identifying intervention opportunities earlier in disease trajectories. Both approaches recommended similar intervention types, suggesting complementary rather than replacement roles. DISCUSSION: An integrated clinical decision support system can enhance population health management by providing actionable guidance on intervention timing and selection. CONCLUSION: An integrated decision support system's ability to identify opportunities before high utilization manifests offers potential for shifting from reactive to preventive care delivery for vulnerable populations.
J Am Med Inform Assoc
· 2026 Mar · PMID 41453133
·
Full text
BACKGROUND AND SIGNIFICANCE: Ambient digital scribe (ADS) platforms, which combine ambient speech recognition and large language models to generate clinical documentation, are currently undergoing rapid clinical adoption...BACKGROUND AND SIGNIFICANCE: Ambient digital scribe (ADS) platforms, which combine ambient speech recognition and large language models to generate clinical documentation, are currently undergoing rapid clinical adoption. Early data suggest that ADS utilization may reduce documentation burden and improve provider efficiency; however, the ethical implications of this largely unregulated technology remain relatively unexamined. FINDINGS: In this article, we identify and explore 4 key ethical issues surrounding ADS technology-safety, bias, data ownership, and justice-from a range of stakeholder perspectives. We provide an overview of current international regulatory policies, highlighting the need for standardized evaluation and reporting guidelines. RECOMMENDATIONS: Drawing on established ethical frameworks, we propose actionable recommendations for safe and equitable ADS implementation, including standardized evaluation metrics, regulatory oversight, and safeguards at institutional and end-user levels. CONCLUSION: Ensuring the ethical implementation of ADS technology is essential for actualizing its potential benefits while upholding foundational principles of safety, equity, and transparency in clinical practice.
Wissel BD, Percy Z, Zachem TJ
… +5 more, Beaulieu-Jones B, Kohane IS, Goldstein SL, Gecili E, Dexheimer JW
J Am Med Inform Assoc
· 2026 Mar · PMID 41445428
·
Full text
OBJECTIVE: To understand the heterogeneous treatment effects of electronic alerts for acute kidney injury (AKI). MATERIALS AND METHODS: Secondary analysis of individual patient data from 3 randomized controlled trials. O...OBJECTIVE: To understand the heterogeneous treatment effects of electronic alerts for acute kidney injury (AKI). MATERIALS AND METHODS: Secondary analysis of individual patient data from 3 randomized controlled trials. Our outcome measure was 14-day all-cause mortality. Data from the ELAIA-1 trial were used to predict the individualized effect of alerts on mortality based on patients' phenotype. Results were internally validated on a holdout dataset and externally validated using data from 2 additional trials: UPenn and ELAIA-2. We used machine learning-based methods and performed a meta-analysis on individual patient data to identify patient subgroups whose risk of mortality was associated with alerts. In addition, provider actions following alerts were examined to explain how alerts impacted patient mortality. RESULTS: Compared to patients who were predicted to be harmed by an alert, patients predicted to benefit had a lower risk of death in both the internal validation cohort (n = 1809 patients; Pinteraction = .045) and both external validation cohorts (n = 7453 patients; Pinteraction < .0001). In external cohorts, 43 deaths may have been preventable if alerts were restricted to likely beneficiaries. Machine-learning based meta-analysis identified reduced mortality with alerts among patients with higher blood pressures (BP) and lower predicted risk, but increased mortality in non-urban and non-teaching hospitals. Provider responses to alerts differed across subgroups. DISCUSSION: Our findings indicate substantial heterogeneity in the effects of AKI alerts on patient mortality. Tailoring alert delivery based on predicted benefit may mitigate harm and enhance clinical outcomes. CONCLUSION: Individualizing automated alerts may reduce all-cause mortality. A prospective trial of individualized alerts is needed to confirm these results. TRIAL REGISTRATION: https://clinicaltrials.gov/ct2/show/NCT02753751 and https://clinicaltrials.gov/ct2/show/NCT02771977.
Conderino S, Kirchner HL, Thorpe LE
… +35 more, Divers J, Hirsch AG, Nordberg CM, Schwartz BS, Zhang L, Cai B, Rudisill C, Obeid JS, Liese A, Allen KS, Dixon BE, Crume T, Dabelea D, Burgett S, Bellatorre A, Shao H, Bian J, Guo Y, Bost S, Lyu T, Reynolds K, Mefford MT, Zhou H, Zhou M, Lustigova E, Utidjian LH, Maltenfort M, Kamboj M, Mendonca EA, Hanley P, Zaganjor I, Pavkov ME, Rosenman M, Titus AR, DiCAYA Study Group
J Am Med Inform Assoc
· 2026 Mar · PMID 41442443
·
Full text
OBJECTIVE: We discuss implications of potential ascertainment biases for studies examining diabetes risk following SARS-CoV-2 infection using electronic health records (EHRs). We quantitatively explore sensitivity of res...OBJECTIVE: We discuss implications of potential ascertainment biases for studies examining diabetes risk following SARS-CoV-2 infection using electronic health records (EHRs). We quantitatively explore sensitivity of results to misclassification of COVID-19 status using data from the U.S.-based Diabetes in Children, Adolescents and Young Adults (DiCAYA) Network on children (≤17 years) and young adults (18-44 years). MATERIALS AND METHODS: In our retrospective case study from the DiCAYA Network, SARS-CoV-2 was identified using labs and diagnoses from June 1, 2020 to December 31, 2021. Patients were followed through December 31, 2022 for new diabetes diagnoses. Sites examined incident diabetes by COVID-19 status using Cox proportional hazards models. Results were pooled in meta-analyses. A bias analysis examined potential impact of COVID-19 misclassification scenarios on results, guided by hypotheses that sensitivity would be <50% and would be higher among those who developed diabetes. RESULTS: Prevalence of documented COVID-19 was low overall and variable across sites (children: 4.4%-7.7%, young adults: 6.2%-22.7%). Individuals with documented COVID-19 were at higher risk of incident diabetes compared to those with no documented infection, but results were heterogeneous across sites. Findings were highly sensitive to COVID-19 misclassification assumptions. Observed results could be biased away from the null under several differential misclassification scenarios. DISCUSSION: Although EHR-based documentation of COVID-19 was associated with incident diabetes, COVID-19 phenotypes likely had low sensitivity, with considerable variation across sites. Misclassification assumptions strongly impacted interpretation of results. CONCLUSION: Given the potential for low phenotype sensitivity and misclassification, caution is warranted when interpreting analyses of COVID-19 and incident diabetes using clinical or administrative databases.