Journal Of The American Medical Informatics Association[JOURNAL]
Sun
200 papers
Biobanking for intelligent medicine: assessment and evaluation with the SHARE principle.
Yang Y, Ullah A, Zhang Y … +6 more , Zong H, Liu X, Zhang C, Hu S, Li J, Shen B
J Am Med Inform Assoc · 2026 Jul · PMID 42043949 · Full text
BACKGROUND: Biobanks are essential for intelligent medicine but face fragmentation and heterogeneity. No standardized framework exists for assessing biobank data value using public information; this study addresses this...
BACKGROUND: Biobanks are essential for intelligent medicine but face fragmentation and heterogeneity. No standardized framework exists for assessing biobank data value using public information; this study addresses this gap. MATERIALS AND METHODS: We systematically evaluated 94 global biobanks (2010-2024) through a literature review and structured data extraction. Based on 12 international standards, we developed the 5-dimensional SHARE principle (Standardization, Hierarchical structuring, Analytical compatibility, Regulatory compliance, Evolutionary adaptability), operationalized into 10 indicators with a 4-tier scoring system. GPT-4o provided AI-supported prescoring, which was validated by 10 experts and through case studies, including RARPKB. RESULTS: The SHARE principle and a classification map of 94 biobanks were generated. AI and expert scoring showed substantial consistency (κ = 0.62; 95% CI, 0.54-0.70). Biobanks were categorized into 4 tiers: Traditional (60-69), Data-Driven (70-79), Knowledge-Guided (80-89), and Generative and Reasoning-oriented Biobank (90-100). Case validation confirmed utility for disease-specific biobanks. DISCUSSION: We highlight the principal findings, critically examine the reliance on public documentation, propose mitigation strategies, and discuss indicator weighting and implications for translational informatics. CONCLUSIONS: The SHARE principle provides a scalable, standardized method for assessing biobank data value, supporting biobank development, resource discovery, and the development of AI-driven biomedical ecosystems for intelligent medicine.
The digital divide in adoption and advanced use of electronic health records among US hospitals: rural versus urban disparities from 2008 to 2023.
Anzalone AJ, Liu Y, Reisher E … +3 more , Frankel E, Hansen JR, Geary CR
J Am Med Inform Assoc · 2026 Jul · PMID 42043914 · Full text
OBJECTIVE: Using 15 years of hospital survey data (2008-2023), we examined rural-urban trends in electronic health records (EHR) adoption and advanced capabilities for data sharing, public health reporting, and care impr...
OBJECTIVE: Using 15 years of hospital survey data (2008-2023), we examined rural-urban trends in electronic health records (EHR) adoption and advanced capabilities for data sharing, public health reporting, and care improvements. MATERIALS AND METHODS: We conducted a repeated cross-sectional study using the American Hospital Association Annual Survey Information Technology Supplement (2008-2018, 2020-2023). We constructed hospital-year measures of basic and comprehensive EHR capability, certified EHR technology (CEHRT) adoption, and advanced functions in patient engagement, interoperability, public health reporting, and social determinants of health (SDOH). We compared rural and urban adoption using descriptive analyses, Cochran-Armitage trend tests, and logistic regression with clustered standard errors, adjusted for bed size, ownership, and census division. RESULTS: The cohort included 41 838 hospital-years (45% rural). Comprehensive capability rose from 2% in 2008 to 75% in 2020, and CEHRT was nearly universal by 2023. Rural hospitals lagged urban hospitals in adopting basic and comprehensive EHRs, with the gap peaking at 25% in 2016. By 2021-2023, most hospitals reported patient portals, interoperability, public health reporting, and SDOH capabilities. Rural hospitals remained 4%-15% behind urban peers in advanced functions, particularly interoperability, public health reporting, and SDOH data collection and use. DISCUSSION: Persistent rural-urban gaps in advanced EHR use may limit interoperability, data-driven care, and public health reporting in rural settings. CONCLUSION: EHR adoption is widespread, but substantial rural-urban gaps persist in advanced EHR use. Support for advanced capabilities that enable data sharing, reporting, and data-driven improvement is needed to ensure that rural hospitals and communities benefit from digital transformation beyond adoption.
Identifying key user experience and technical features for sustained use of unguided chatbots for health-related behavior change: a systematic review.
Sayed F, Park A, Sullivan PS … +3 more , Jordan A, Kwon S, Ge Y
J Am Med Inform Assoc · 2026 Jul · PMID 42035705 · Full text
OBJECTIVE: This review aims to identify the contribution of user experience features and underlying technical features to sustained engagement in unguided chatbots for improving health-related behaviors. MATERIALS AND ME...
OBJECTIVE: This review aims to identify the contribution of user experience features and underlying technical features to sustained engagement in unguided chatbots for improving health-related behaviors. MATERIALS AND METHODS: Following PRISMA-2020 guidelines, we conducted a systematic review, searching PubMed, ACM, APA PsycINFO, Cochrane, Web of Science, and IEEE Xplore from June to September 2022 and updated in April 2025. Data was analyzed via Synthesis without Meta-Analysis (SWiM), to understand the relationship between user engagement overall and individual experience metrics. RESULTS: Customizable avatars and flexible input interactions may enhance overall user engagement. Conversely, pre-scripted content that lacks personalization and emotional support negatively impacts user satisfaction and adherence to health interventions. Other features contributing to sustained engagement are in-app technical assistance, user learning features, and crisis support systems. A strong positive correlation (r = 0.808, n = 16) was observed between user satisfaction and engagement, specifically for satisfaction dimensions including need fulfillment (r = 0.872, n = 6), willingness to recommend chatbot (r = 0.817, n = 4) and user enjoyment (r = 0.971, n = 3) in SWiM analysis. The limited application of large language models and retrieval augmented generation techniques may constrain the quality of support available to users and overall sustained engagement. CONCLUSION: Effective unguided chatbot design requires an emphasis on interactive educational elements, in-app technical assistance and crisis support, and personalized content. This can be achieved with high context awareness, input understanding, and quality content generation. Our findings suggest that user satisfaction is a primary driver of sustained engagement, though further research is needed to validate individual user satisfaction features for sustained engagement.
Extending the Fundamental Theorem of Biomedical Informatics for the AI era.
Payne PRO, Chen JH, Longhurst CA
J Am Med Inform Assoc · 2026 Jul · PMID 42035470 · Full text
BACKGROUND: Charles Friedman's Fundamental Theorem of Biomedical Informatics holds that a person working in partnership with an information resource outperforms that same person unassisted. Since its publication, advance...
BACKGROUND: Charles Friedman's Fundamental Theorem of Biomedical Informatics holds that a person working in partnership with an information resource outperforms that same person unassisted. Since its publication, advances in artificial intelligence (AI), adaptive learning systems, and large-scale data infrastructures have transformed the biomedical ecosystem, extending informatics beyond clinical care into domains such as public health, consumer health, translational science, and the broader life sciences. Such expansion has further underscored the importance of the Fundamental Theorem while also elucidating ways it can be expanded to meet current needs. OBJECTIVE: To reassess and extend the Fundamental Theorem for the AI era in a manner that preserves its conceptual strength while broadening its applicability across an evolved and more complex biomedical ecosystem. METHODS: This Viewpoint synthesizes empirical evidence and sociotechnical theory related to human-AI collaboration, learning health systems (LHS), learning public health systems (LPHS), AI governance, and systems science to contextualize the Fundamental Theorem within such contemporary frameworks. RESULTS: We argue that the unit of analysis of the Fundamental Theorem should shift from individuals and tools to adaptive sociotechnical systems spanning clinical care, public health, translational research, consumer engagement, and life sciences innovation. We propose an expanded theorem: A learning biomedical ecosystem that continuously optimizes human-AI collaboration will outperform humans or AI alone. CONCLUSIONS: This evolution builds directly upon Friedman's original theorem, reaffirming its human-centered foundation, while incorporating AI-enabled computation, adaptive learning, and systems-level integration across the modern biomedical enterprise.
Optimizing participation in digital health studies: understanding appointment attendance.
Schnall R, Lin H, Brin M … +4 more , Jimenez J, Johnson AK, Kempf MC, Liu N
J Am Med Inform Assoc · 2026 Jul · PMID 42035294 · Full text
OBJECTIVE: This study examined whether attendance at online digital health research appointments in the American Women Assessing Risk Epidemiologically (AWARE) study was associated with (1) participant age, (2) schedulin...
OBJECTIVE: This study examined whether attendance at online digital health research appointments in the American Women Assessing Risk Epidemiologically (AWARE) study was associated with (1) participant age, (2) scheduling factors (time of day, day of week, month), (3) appointment confirmation, and (4) HIV behavioral risk factors. MATERIALS AND METHODS: We analyzed scheduling and eligibility screening data from AWARE, a 24-month U.S.-based longitudinal digital cohort of cisgender women at elevated likelihood of HIV seroconversion. Participant demographic and behavioral data were merged with the study team's Outlook calendar. Chi-square tests and logistic regression models assessed associations between appointment attendance and participant characteristics and scheduling factors. RESULTS: Women aged ≥50 years had higher odds of missing baseline visits compared to those aged 20-29 years (44.7% vs 32.3%). Appointments scheduled at 2:00 pm (45.7%), 4:00 pm (45.2%), and 8:00 am (40.2%) had higher no-show rates than other times. No-show rates were lowest on Fridays (30.2%) and during March (27.7%) and June (25.2%). Confirming appointments 24 hours in advance significantly reduced no-shows compared to no confirmation (19.0% vs 51.6%). Histories of having been physically hurt (44.2% vs 32.1%), forced to have sexual activities (41.8% vs 34.1%) and incarcerated (39.3% vs 33.4%) were also associated with higher no-show rates. Similar patterns were observed for rescheduled visits. CONCLUSION: Attendance in digital research was influenced by age, scheduling, and structural vulnerabilities. Incorporating digital access support into study design and grant budgets may reduce disparities, improve retention, and enhance efficiency.
Optimizing temporal windows for wearable-augmented post-discharge risk prediction: a methods study.
Bressman E, Park SH, Greysen SR … +1 more , Chen J
J Am Med Inform Assoc · 2026 Jul · PMID 42035291 · Full text
OBJECTIVE: Traditional readmission risk models relying on static discharge data have limited predictive performance and fail to capture patients' recovery trajectories after hospitalization. We sought to identify optimal...
OBJECTIVE: Traditional readmission risk models relying on static discharge data have limited predictive performance and fail to capture patients' recovery trajectories after hospitalization. We sought to identify optimal modeling parameters for dynamically predicting readmission risk using post-discharge step-count data from remote monitoring devices. METHODS: We combined data for adults aged 55+ from 2 studies that collected longitudinal activity data after discharge. We constructed a patient-day dataset incorporating static demographic and clinical variables and dynamic activity features aggregated over retrospective windows of 3, 5, 7, or 10 days. Models predicted readmission or death over prospective horizons of 3, 5, 7, or 10 days, within follow-up periods of 30-180 days. Logistic regression and LightGBM models were trained using 5-fold cross-validation on an 80:20 patient-level split. RESULTS: Among 215 participants, LightGBM outperformed logistic regression across all configurations (mean AUC 0.82 vs 0.76). Performance improved with longer prospective horizons but was insensitive to retrospective window length. The LightGBM model was well-calibrated (Hosmer-Lemeshow χ2 = 2.46, P = .96), whereas logistic regression showed miscalibration (χ2 = 51.8, P < .001). In feature-importance analyses, LightGBM ranked static (length of stay, vitals, BMI) and activity (recent steps, distance) features highly, whereas logistic regression emphasized activity variables. DISCUSSION: Prediction performance was impacted by horizon length and training window, with minimal effect of retrospective window. LightGBM achieved better discrimination and calibration, supporting flexible, non-parametric methods for post-discharge risk prediction. CONCLUSION: Post-discharge step count data enhance dynamic readmission risk prediction. Optimizing temporal windows and model type improves discrimination and calibration.
Increasing value in the Veterans Affairs Healthcare System (VA) with precision health: a continuing landmark collaboration with the Department of Energy.
Justice AC, McMahon B, Jacobson DA … +22 more , Cho K, Kapadia AJ, Aguayo SM, Gümüş ZH, Danciu I, Beckham JC, Kimbrel NA, Crivelli S, Boudreau EA, Finley P, Bryant AK, Green M, Yoo S, Joseph J, Reaven P, Zhou J, Luoh SW, Madduri R, Fanous A, Agarwal K, Mukundan H, Muralidhar S
J Am Med Inform Assoc · 2026 Jul · PMID 42035288 · Full text
OBJECTIVE: Phase II of MVP-CHAMPION, a federal collaboration between the Veterans Affairs Healthcare System (VA) and the Department of Energy (DoE), leveraged large-scale clinical, geo-spatial, and genetic data with stat...
OBJECTIVE: Phase II of MVP-CHAMPION, a federal collaboration between the Veterans Affairs Healthcare System (VA) and the Department of Energy (DoE), leveraged large-scale clinical, geo-spatial, and genetic data with state-of-the-art artificial intelligence (AI), and high-performance computing (HPC) to improve value in healthcare. MATERIALS AND METHODS: Eight clinical priority projects for which AI was a critical missing capability were initiated to address: lung cancer screening (MVP 061), suicide risk screening (MVP 062), cardiovascular risk in obstructive sleep apnea (MVP 063), checkpoint inhibitor toxicity (MVP 064), heart failure (MVP 065), renal complications in diabetes (MVP 066), post COVID-19 sequelae (MVP 067), and antipsychotic medication toxicity (MVP 068). RESULTS: Building on a strong regulatory and administrative foundation, we developed multimorbidity-aware analytic frameworks, reusable computational tools, and analytic pipelines. These greatly facilitated identification of novel risk factors including genetic variants and specification of more discriminating prediction models. Novel genetic risk factors are informing development and repurposing of medications and discriminating prediction models promise to improve healthcare value. DISCUSSION: The research foundation developed in Phase I and extended in Phase II of MVP CHAMPION has supported an unprecedented federal collaboration and yielded significant scientific advances. Our clinical findings are poised for near-term application, while advances in machine learning and high-performance computing may accelerate the broader adoption of artificial intelligence in healthcare. CONCLUSION: This maturing VA-DoE federal collaboration is poised to transform the future of Veterans' healthcare and the broader national landscape of precision health.
Informatics matters beyond biological and medical influences on health, well-being, and health equity.
Bakken S
J Am Med Inform Assoc · 2026 Apr · PMID 41990731 · Full text
Abstract loading — click title to view on PubMed.
Targeted use of large language models for EHR-based computable phenotyping.
Owens D, Cao J, Gupta M … +3 more , Nguyen D, Peterson E, Navar AM
J Am Med Inform Assoc · 2026 Jun · PMID 41990328 · Full text
OBJECTIVE: Computable phenotypes derived from electronic health records (EHRs) are central to clinical research and quality reporting. Although large language models (LLMs) can extract clinically rich information from un...
OBJECTIVE: Computable phenotypes derived from electronic health records (EHRs) are central to clinical research and quality reporting. Although large language models (LLMs) can extract clinically rich information from unstructured notes, routine application to all patients is computationally expensive. We evaluated whether uncertainty-guided selective use of LLMs can improve phenotyping accuracy while preserving scalability. MATERIALS AND METHODS: We developed a selective augmentation framework integrating structured and unstructured EHR data using uncertainty-guided triage. An ensemble of heterogeneous classifiers trained on structured data generated probabilistic phenotype predictions and uncertainty measures to identify patients at elevated risk of misclassification. Only flagged patients underwent LLM-based analysis of unstructured clinical notes using retrieval-augmented generation. LLM-derived outputs were incorporated as additional predictors in a final probabilistic model. Performance was evaluated for two registry-based phenotypes: diabetes mellitus and peripheral arterial disease (PAD), using internal cross-registry and external validation cohorts. RESULTS: For diabetes mellitus, selective augmentation improved sensitivity in the internal validation cohort from 0.81 to 0.90 without loss of specificity (0.92). More than 70% of triage-flagged patients represented misclassifications by structured data alone. For PAD, selective augmentation markedly increased sensitivity from 0.18 to 0.97 while maintaining high specificity (0.99), requiring LLM analysis for only 10% of patients. DISCUSSION: Uncertainty-guided triage efficiently concentrated LLM use on patients most likely to benefit, improving case identification-particularly for phenotypes poorly captured by structured data-while minimizing computational burden. CONCLUSION: Selective, uncertainty-guided integration of LLMs enables scalable, interpretable, and accurate EHR-based phenotyping, offering a practical alternative to universal LLM deployment in real-world informatics workflows.
Applying natural language processing and large language models to clinical notes for phenotyping and diagnosing rare diseases: a systematic review.
Kim S, Zhou Y, Guo Y … +2 more , Xiao C, Zheng K
J Am Med Inform Assoc · 2026 Jun · PMID 41990239 · Full text
OBJECTIVES: Patients with rare diseases often face long delays before receiving a diagnosis. Using electronic health records for automated phenotyping and diagnosis of rare diseases is a promising approach but can be cha...
OBJECTIVES: Patients with rare diseases often face long delays before receiving a diagnosis. Using electronic health records for automated phenotyping and diagnosis of rare diseases is a promising approach but can be challenging because critical information is often recorded in unstructured notes rather than structured fields. This systematic review synthesizes the current literature applying natural language processing (NLP) and large language models (LLMs) for rare disease phenotyping and diagnosis from clinical text. MATERIALS AND METHODS: A systematic search was conducted in PubMed, ACM Digital Library, and IEEE Xplore. Two reviewers independently screened papers and extracted data. Methodological rigor and quality of the studies were evaluated using the MI-CLAIM framework. RESULTS: The search resulted in 135 studies; 27 of them met the inclusion criteria. Methods used spanned rule-based systems, classical ML/DL models, transformer architectures, and LLMs. Transformer- and LLM-based approaches outperformed earlier methods in entity recognition, phenotype extraction, and diagnostic ranking. Several studies demonstrated clinical impact, such as increased genetic testing and identification of undiagnosed cases. However, most studies relied on retrospective and single-center datasets. Reporting of preprocessing, evaluation, and reproducibility was largely inconsistent, and interpretability, fairness, and privacy were rarely addressed. DISCUSSION: Natural language processing and LLMs show strong potential to accelerate rare disease diagnosis. However, heterogeneity in methods and metrics hinders cross-study comparability. Data scarcity, lack of generalization, and limited transparency remain significant challenges. CONCLUSIONS: Natural language processing/LLM methods can support timely diagnosis of rare diseases using unstructured clinical text. Future research should prioritize multicenter studies, standardized evaluation frameworks, transparency, and fairness safeguards to enable reliable, equitable deployment.
Federated learning's uncomfortable truth: why human networks matter more than neural networks.
Peltonen LM, Chomutare T
J Am Med Inform Assoc · 2026 Jun · PMID 41984621 · Full text
OBJECTIVES: To examine real-world barriers to implementing federated learning in healthcare and highlight the organizational, regulatory, and socio-technical factors often overlooked in technical research. MATERIALS AND...
OBJECTIVES: To examine real-world barriers to implementing federated learning in healthcare and highlight the organizational, regulatory, and socio-technical factors often overlooked in technical research. MATERIALS AND METHODS: Insights were derived from a 3-year implementation of a Nordic-Baltic federated health data network involving 5 countries and 9 institutions, incorporating legal, organizational, and cross-disciplinary perspectives. RESULTS: Structural challenges included coordination burdens, divergent interpretations of privacy and risk, epistemological gaps between disciplines, and the absence of legal frameworks for multi-country distributed learning in Europe. These constraints limited progress despite the availability of robust technical solutions. DISCUSSION: Technical privacy measures alone cannot replace trust-building, governance development, and cross-disciplinary translation work. Federated learning is more accurately understood as a socio-technical collaboration model rather than a purely technical architecture. CONCLUSION: Pre-implementation planning, tiered participation models, and strengthened governance are essential to support equitable, sustainable, and clinically impactful adoption of federated learning in healthcare.
Opportunities for informatics to improve patient experiences: observations and reflections of ACMI fellows.
Strasberg HR, Hoffer EP, Koppel R … +5 more , Johnson KB, Tierney WM, Rutledge GW, Bernstam EV, ACMI Virtual Patient Experience Participants Collaborative
J Am Med Inform Assoc · 2026 Jun · PMID 41979031 · Full text
OBJECTIVES: We report on findings from a meeting convened by the American College of Medical Informatics (ACMI) to characterize aspects of the patient experience that could be improved using informatics. MATERIALS AND ME...
OBJECTIVES: We report on findings from a meeting convened by the American College of Medical Informatics (ACMI) to characterize aspects of the patient experience that could be improved using informatics. MATERIALS AND METHODS: The American College of Medical Informatics fellows were invited to share their experiences as patients and suggest informatics approaches that may improve the patient experience. RESULTS: We identified 4 themes: (1) getting the right care, (2) data sharing and data interoperability, (3) guiding low-cost evaluations, and (4) predictive analytics. DISCUSSION: Despite widespread adoption of health IT, patient experiences remain far from optimal. CONCLUSION: The American College of Medical Informatics fellows identified informatics approaches, applications, and research areas that have the potential to improve patient experiences with health care systems.
Automating infection indicator extraction in home healthcare through instruction-tuned large language models.
Xu Z, Song J, Zhou S … +5 more , Scharp D, Hobensack M, Hu Y, Shang J, Topaz M
J Am Med Inform Assoc · 2026 Jun · PMID 41974103 · Full text
OBJECTIVE: Home healthcare (HHC) clinical notes contain critical infection indicators that clinicians need in structured "indicator + context" pairs. Data sparsity and limited computing resources hinder automated extract...
OBJECTIVE: Home healthcare (HHC) clinical notes contain critical infection indicators that clinicians need in structured "indicator + context" pairs. Data sparsity and limited computing resources hinder automated extraction in decentralized HHC settings. This study developed and evaluated a resource-efficient pipeline using instruction-tuned, moderate-sized large language models (LLMs) to address these barriers. To address the data sparsity challenge, we also assessed the impact of a targeted LLM-based data augmentation strategy. MATERIALS AND METHODS: An expert-defined schema of 26 infection indicator categories was developed. We expanded the training set using a 3-stage workflow: targeted annotation, context mutation, and synthetic generation. We adapted 2 moderate-sized models (Gemma-12B and Qwen-14B) via Quantized Low-Rank Adaptation (QLoRA). We compared them to a larger-sized, prompted model and a smaller-sized, fully fine-tuned LLM. We evaluated all models on a held-out test set using partial micro-averaged F1 score, output reliability metrics, and qualitative error analysis. RESULTS: Instruction-tuned moderate-sized LLMs outperformed both baselines. The top-performing model, augmented Gemma-12B, achieved a partial micro-averaged F1 score of 0.879. LLM-based data augmentation enhanced overall performance, improving the identification of rare indicators and the interpretation of negations. The best model maintained a partial F1 score above 0.750 across all indicator categories. It also showed high format adherence, confirming its ability to generate reliable structured outputs. DISCUSSION: Instruction-tuning moderate-sized LLMs with QLoRA and targeted data augmentation enables high-accuracy extraction of infection indicators from HHC notes. CONCLUSION: This resource-efficient pipeline provides a scalable foundation for automated infection surveillance in healthcare settings with limited resources.
Electronic health record-based prediction models for dementia detection: a systematic review of model performance and quality.
Lu A, Srikanth V, Westworth S … +6 more , Baey YG, Moran C, Beare R, Siostrom K, Andrew N, Collyer T
J Am Med Inform Assoc · 2026 Jun · PMID 41967052 · Full text
OBJECTIVES: Leveraging routine electronic health records (EHR) for dementia detection is a growing field, but quality and clinical utility of existing models are unclear. This systematic review aimed to evaluate performa...
OBJECTIVES: Leveraging routine electronic health records (EHR) for dementia detection is a growing field, but quality and clinical utility of existing models are unclear. This systematic review aimed to evaluate performance, methodological quality, and risk of bias of EHR-based dementia prediction models. MATERIALS AND METHODS: We systematically searched Medline, EMBASE, Scopus, IEEE Xplore, and ACM from conception until July 2024. All studies and grey literature describing development or validation of probabilistic prediction models using EHR data for dementia detection were included. Risk of bias was assessed using PROBAST. RESULTS: Fifty-six studies (434 prediction models, 155 external validations) were included. Most models were prognostic (66%), used US data (71%), relied solely on structured data, and 47 (11%) were externally validated. Modeled outcomes were extremely heterogeneous: gold-standard clinical criteria were used in 17 models (4%), with others reliant on diagnostic codes for case ascertainment. Discriminative metrics were frequently reported (82% of models), but calibration was rarely assessed (16%). All models were judged high risk of bias, driven by poor outcome definition, inadequate handling of missing data, and potential overfitting. DISCUSSION: Our review highlights significant issues with methodological rigor and reporting transparency in existing EHR dementia prediction models. Ambiguous outcomes, flawed case ascertainment, and incomplete performance reporting, all limit clinical usefulness. Overall, model performance was difficult to assess and compare across studies due to incomplete reporting. CONCLUSION: Electronic health record-based dementia prediction is still in its infancy. Methodological rigor and interdisciplinary collaboration are essential to meet clinical needs and achieve real-world impact.
Leveraging clinical epidemiology concepts to strengthen machine learning fairness evaluations.
Guo LL, Arciniegas SE, Yan AP … +4 more , Tomlinson GA, Beauchemin M, Pfohl SR, Sung L
J Am Med Inform Assoc · 2026 Jun · PMID 41947590 · Full text
OBJECTIVES: The increasing use of machine learning (ML) in clinical care makes fairness a central issue. Fairness, defined as the absence of disparities across individuals or subgroups, shares several parallels with conc...
OBJECTIVES: The increasing use of machine learning (ML) in clinical care makes fairness a central issue. Fairness, defined as the absence of disparities across individuals or subgroups, shares several parallels with concepts in clinical epidemiology. The objective was to apply clinical epidemiology frameworks to the evaluation of ML fairness, both to enhance understanding of these concepts and to strengthen fairness assessments. METHODS: This manuscript addresses: (1) the connection between clinical epidemiology bias terms and ML fairness concepts; (2) the relationship between diagnostic testing metrics and fairness criteria; (3) issues arising from multiple testing; and (4) strategies for fairness considerations leveraging clinical epidemiology principles. RESULTS: Unfairness can arise at different stages: before model development, during model development and post-deployment. The root of unfairness can be conceptualized as a result of clinical epidemiology bias concepts, such as selection, measurement, model misspecification, cognitive and implementation bias. Common approaches to evaluating fairness involve comparing model performance metrics across 1 or more subgroups. Four widely used fairness criteria are independence, separation, sufficiency and predictive parity. They can be assessed using diagnostic testing metrics. Fairness evaluations are vulnerable to multiple testing issues, with subgroup analyses posing particular risks for spurious findings. Solutions can leverage established clinical epidemiology principles such as pre-specifying the analytic strategy. CONCLUSIONS: Many parallels exist between ML fairness and clinical epidemiology, including the conceptualization of the root causes of unfairness, the articulation of fairness criteria, and considerations related to multiple testing. Methodologically sound fairness approaches can leverage well-established principles from clinical epidemiology.
A critical evaluation of generative query expansion on biomedical literature retrieval.
Fang Y, Zhang G, Chen F … +2 more , Peng Y, Weng C
J Am Med Inform Assoc · 2026 Jun · PMID 41921511 · Full text
OBJECTIVE: To evaluate the effectiveness of generative query expansion for biomedical literature retrieval. MATERIALS AND METHODS: We thoroughly examined eight generative query expansion methods using three large languag...
OBJECTIVE: To evaluate the effectiveness of generative query expansion for biomedical literature retrieval. MATERIALS AND METHODS: We thoroughly examined eight generative query expansion methods using three large language models across five datasets for biomedical literature retrieval. We further performed a quantitative analysis, including performance comparisons, rank transition analysis, and article-type effect analysis. We also conducted a qualitative examination of representative cases, from which we derived an error taxonomy. RESULTS: On BioASQ-Y/N, GPT-4o-based query expansion shifts Recall@10 to 0.417-0.512 and nDCG@10 to 0.358-0.479, relative to a baseline of 0.491 and 0.456. For PubMedQA, Precision@1 ranges from 0.764 to 0.876 and nDCG@10 from 0.847 to 0.931, compared with baseline values of 0.893 and 0.935. For 2019-Trec-PM, query expansion yields Recall@100 of 0.217-0.256 and nDCG@100 of 0.272-0.312, versus a baseline of 0.227 and 0.274. Similarly, for 2018-TREC-PM, Recall@100 spans 0.169-0.227 and nDCG@100 spans 0.195-0.250, relative to baseline scores of 0.164 and 0.191. For 2017-TREC-PM, Recall@100 and nDCG@100 fall within 0.111-0.139 and 0.154-0.191 under query expansion, compared with baseline metrics of 0.102 and 0.147. Both general-purpose and domain-specific Llama-based models demonstrate similar performance to GPT-4o. DISCUSSION AND CONCLUSION: The impact of query expansion varies significantly by the expansion methods and type of evidence, but is relatively agnostic to backbone model choice. Notably, query expansion primarily affects article ranking but has a limited impact on the screening stage. Our findings underscore the unique challenges of biomedical literature retrieval and highlight the need to develop domain-specific information retrieval techniques.
A systematic methodological review of best practices, pitfalls, and opportunities in mixed methods research in applied clinical informatics.
Nguyen OT, Ahmad A, Doering M … +1 more , Abraham J
J Am Med Inform Assoc · 2026 Jun · PMID 41921502 · Full text
INTRODUCTION: Mixed methods are used to holistically understand the "what", "how", and "why" questions within a single study by integrating quantitative and qualitative methods. Although this approach has demonstrated va...
INTRODUCTION: Mixed methods are used to holistically understand the "what", "how", and "why" questions within a single study by integrating quantitative and qualitative methods. Although this approach has demonstrated value in other disciplines, their use in applied clinical informatics research remains largely unexplored. This methodological review characterized the types of informatics intervention studies that used mixed methods designs and the specific methods of evaluation and reporting. MATERIALS AND METHODS: On December 13, 2024, we searched Embase, MEDLINE, PubMed, CINAHL and PsycINFO for studies that reportedly used a mixed methods design to study the development or evaluation of clinical informatics interventions. We developed themes in topics studied, designs used, and reporting issues. Quality assessment was conducted using the Mixed Methods Appraisal Tool. RESULTS: We included 54 studies. Despite being described as mixed methods, 14 studies (25.9%) lacked the integration of quantitative and qualitative data. Of the remaining studies, convergent study designs were commonly used. Mixed methods were predominantly used during pre-implementation and post-implementation evaluations. Quality issues included non-representative samples and non-reporting of qualitative approaches and paradigms. DISCUSSION AND CONCLUSIONS: The use of mixed methods to study clinical informatics development and implementation is uncommon. This review confirmed that reporting and design problems found in other disciplines extend to informatics. Calls to action include a need to disseminate mixed methods guidance within the informatics community and offer mixed methods training in graduate programs. We also synthesized an initial reporting checklist from the literature. Our findings offer baseline levels to assess educational efforts.
Optimizing patient portal user interface boosts activation rates for Spanish-speaking pediatric patients and caregivers.
Chilukuri N, Ballard E, Xu X … +7 more , McPherson T, Chug A, Tse G, Lee T, Bassett HK, Carlson JL, Pageler N
J Am Med Inform Assoc · 2026 Apr · PMID 41921501 · Full text
OBJECTIVE: Patient portals (PP) can enhance healthcare access for pediatric patients and families but disparities persist among patients with diverse preferred languages. Our objective was to optimize the PP user interfa...
OBJECTIVE: Patient portals (PP) can enhance healthcare access for pediatric patients and families but disparities persist among patients with diverse preferred languages. Our objective was to optimize the PP user interface and associated activation workflows to improve activation rates for pediatric families with Spanish as preferred language (Spanish-speaking patients and/or parents/guardians). MATERIALS AND METHODS: A quality improvement intervention at a quaternary children's hospital optimized the PP activation user interface by redesigning the email/text activation messages, activation screen and error messages. The interventions were launched on August 3, 2023. Statistical process control charts were used to analyze patterns in activation rates, PP helpdesk calls, PP before and after the intervention implementation. RESULTS: Of 38 575 patients with ambulatory visits in our study period, 8.4% were Spanish-speaking, 89.1% were English-speaking. There was a sustained increase in the monthly rate of newly activated PP accounts (43%-58%), decreased disparity between English- and Spanish-speaking patients by 7%, and a sustained increase in monthly rate of overall currently active PP accounts of Spanish-speaking patients (66%-69%) after interventions. There was a decrease in monthly number of helpdesk tickets due to activation issues and an increase in monthly number of incoming patient messages in Spanish per number of Spanish-speaking patients active on PP. DISCUSSION: Optimizing the PP activation user interface and workflows by applying an equity-lens can help mitigate disparities in PP access. CONCLUSION: Future research is needed to implement and evaluate health system-community partnerships to mitigate the digital health divide.
Predictive performance precision analysis in medicine: identification of low-confidence predictions at patient and profile levels (MED3pa I).
Lefebvre O, Camirand Lemyre F, Ethier JF … +4 more , Chikouche LH, Amriou L, Poenaru D, Vallières M
J Am Med Inform Assoc · 2026 Jun · PMID 41914796 · Full text
OBJECTIVES: Artificial Intelligence models are increasingly used in health care, yet global performance metrics can mask variations in reliability across individual patients or subgroups with shared attributes, called pa...
OBJECTIVES: Artificial Intelligence models are increasingly used in health care, yet global performance metrics can mask variations in reliability across individual patients or subgroups with shared attributes, called patient profiles. This study introduces predictive performance precision analysis in medicine (MED3pa), a method that identifies when models are less reliable, allowing clinicians to better assess model limitations. MATERIALS AND METHODS: We propose a framework that estimates predictive confidence using 3 combined approaches: individualized (IPC), aggregated (APC), and mixed predictive confidence (MPC). Individualized predictive confidence estimates confidence for each patient, APC assesses it across profiles, and MPC combines both. We evaluate our method on 4 datasets: 1 simulated, 2 public, and 1 private clinical dataset. Metrics by declaration rate curves show how performance changes when retaining only the most confident predictions, while interpretable decision trees reveal profiles with higher or lower model confidence. RESULTS: We demonstrate our method in internal, temporal, and external validation settings, as well as through a clinical example. In internal validation, limiting predictions to the 93% most confident cases improved sensitivity by 14.3% and the area under the receiver operating characteristic curve by 5.1%. In the clinical example, MED3pa identified a patient profile with high misclassification risk, demonstrating its potential for safer deployment. DISCUSSION: By identifying low-confidence predictions, our framework improves model reliability in clinical settings. It can be integrated into decision support systems to help clinicians make more informed decisions. Confidence thresholds help balance model performance with the proportion of patients for whom predictions are considered reliable. CONCLUSION: Better leveraging confidence in model predictions could improve reliability and trustworthiness, supporting safer and more effective use in health care.
Impact of announced wait time information on emergency department overcrowding mitigation: a simulation study.
Zou C, Zhang Y, Ouyang H … +1 more , Sun Z
J Am Med Inform Assoc · 2026 Jun · PMID 41911388 · Full text
OBJECTIVE: Despite widespread implementation of predicted patient wait time information systems in hospital emergency departments (EDs), the relationship between quality of announced wait time information and ED overcrow...
OBJECTIVE: Despite widespread implementation of predicted patient wait time information systems in hospital emergency departments (EDs), the relationship between quality of announced wait time information and ED overcrowding mitigation remains unclear. This study investigates how prediction accuracy, update frequency, and patient adoption rates affect ED overcrowding level. MATERIALS AND METHODS: A data-calibrated simulation model was developed using patient visit records from three metropolitan EDs in Hong Kong. We systematically varied patient adoption rates and evaluated seven wait time prediction methods across four update frequencies. Key performance metrics included the mean and standard deviation of patient wait times and percentage of patients left without being seen (LWBS rate). RESULTS: Accurate prediction methods combined with frequent updates significantly reduced the mean and standard deviation of patient wait times and LWBS rate as patient adoption rate increased. Conversely, inaccurate prediction methods exhibited a U-shaped performance curve. Specifically, when the patient adoption rate was sufficiently high, these methods significantly increased the mean and standard deviation of wait times and LWBS rate, compared to the case with no predicted wait time. CONCLUSIONS: Implementing information systems to display predicted patient wait times requires carefully balancing prediction accuracy, update frequency, and patient adoption. Accurate and timely updates can help redistribute patient load across hospital networks and improve efficiency, while poor accuracy or infrequent updates risk worsening ED congestion, especially when patient adoption rate is high. Our study calls for immediate attention from ED managers to carefully evaluate the impact of announced wait time system before wide implementation.
About
- Frequency
- Sun
- Papers found
- 200
- RSS feed
- Subscribe