Journal Of The American Medical Informatics Association[JOURNAL]
Sun
200 papers
Systematic reviews on hot methods topics.
Bakken S
J Am Med Inform Assoc · 2026 Jun · PMID 42173162 · Full text
Abstract loading — click title to view on PubMed.
Building safer artificial intelligence mental health chatbots: a framework for transparency, evaluation, and shared accountability.
Lee H, Handler R, Mungle T … +1 more , Hernandez-Boussard T
J Am Med Inform Assoc · 2026 May · PMID 42161859 · Publisher ↗
BACKGROUND: Generative artificial intelligence (AI) chatbots built on large language models are rapidly entering mental-health care, offering human-like support without meeting evidentiary standards for safety or effecti...
BACKGROUND: Generative artificial intelligence (AI) chatbots built on large language models are rapidly entering mental-health care, offering human-like support without meeting evidentiary standards for safety or effectiveness. OBJECTIVE: To examine the risks, and outline a governance framework capable of supporting safe, accountable, and equitable deployment of AI mental-health chatbots. METHODS: We synthesized recent clinical, regulatory, and behavioral health literature on AI mental health chatbots, including reported harms and system failure modes, to identify governance gaps and develop a 3-stage safety framework. CONCLUSIONS: Embedding transparency, standardized evaluation, and ongoing oversight across the chatbot lifecycle, with clear responsibilities shared among developers, regulators, clinicians, researchers, and professional societies, is essential to ensure that AI systems intended to support mental health do not inadvertently cause harm.
Fine-tuning and evaluating large language models for patient safety tasks: classification of contributing factors in incident reports.
Wang Y, Bowditch L, Molloy C … +3 more , Yu Y, Hibbert P, Magrabi F
J Am Med Inform Assoc · 2026 May · PMID 42160079 · Publisher ↗
OBJECTIVE: To evaluate and compare the performance of large language models (LLMs) in identifying contributing factors (CFs) underlying patient safety incident investigations. MATERIALS AND METHODS: Four open-source, lig...
OBJECTIVE: To evaluate and compare the performance of large language models (LLMs) in identifying contributing factors (CFs) underlying patient safety incident investigations. MATERIALS AND METHODS: Four open-source, lightweight LLMs, including BERT, LLaMA2, GPT2, and Phi-2 were applied to classify CFs across 6 sociotechnical system-levels encompassing 12 categories (eg, person, task, and organizational factors). Reports of real-world patient safety investigations from public health systems were extracted and labelled by domain experts (n_report/CFs = 300/1338). Data were split into training (n = 852), validation (n = 98), and test sets (n = 388). Performance was evaluated using specificity, precision, recall, and F1 scores. RESULTS: The fine-tuned encoder-based BERT model achieved the highest performance, with a micro-averaged F1 score of 63.6%, outperforming all decoder-based models. Among the decoder models, Phi-2 demonstrated the strongest performance (F1 = 54.9%), exceeding both LLaMA2 and GPT2. BERT performed consistently across 6 system-levels but often misclassified "organization" as "person". DISCUSSION: LLMs hold promise for automating the extraction of CFs from complex safety narratives, particularly for frequently reported system-levels such as "person" and "tasks". Such automation may substantially reduce the manual effort required to analyse reports of patient safety investigations while supporting more consistent analysis across large incident datasets. CONCLUSION: Applying LLMs to analyse the underlying causes of patient safety incidents depends on developing high-quality, domain-specific datasets that enhance the representation of patient safety knowledge and improve model understanding of incident causation. Improving data coverage for rare system-levels is essential to address the current limitations of LLMs in capturing nuanced patient safety concepts and domain-specific reasoning.
Alert fatigue measurement in clinical decision support: a systematic review.
Ray CE, Wilson GM, Hughes AM … +7 more , Cunningham Goedken C, Liu EP, Fitzpatrick MA, Suda KJ, Kota SM, Nwankpa C, Evans CT
J Am Med Inform Assoc · 2026 May · PMID 42148822 · Publisher ↗
BACKGROUND: Alert fatigue is defined as alert dismissals due to excessive or irrelevant alerts and is frequently cited as a barrier to clinical decision support system use and impact. However, the criteria for determinin...
BACKGROUND: Alert fatigue is defined as alert dismissals due to excessive or irrelevant alerts and is frequently cited as a barrier to clinical decision support system use and impact. However, the criteria for determining the presence or absence of alert fatigue are poorly defined. The objective of this systematic review of systematic reviews was to identify operationalized definitions and measures of alert fatigue or alert-related metrics. METHODS: Systematic reviews reporting at least one alert-related metric or measure/operationalization of alert fatigue for physician-directed electronic alerts were included. The Cochrane Library, Embase, and PubMed were searched from database start to 2024. The Revised Assessment of Multiple Systematic Reviews was used to assess study quality and risk of bias. Data were synthesized narratively and with descriptive statistics. RESULTS: A total of 22 studies were included in the review. Studies reported between 1 and 11 alert metrics. Studies were most often of medium quality. Reporting of primary study characteristics was frequently judged to be insufficient. Only one article reported an operational definition of alert fatigue. The most common alert metrics were quantity, override rate, and acceptance rate. DISCUSSION: Alert fatigue measurement methods are not clearly or consistently defined in systematic reviews related to alert fatigue in clinical decision support. Reporting of other primary study characteristics is often limited. We recommend that future efforts use a significant, sustained decrease in appropriate alert response rates from an established baseline as a measure of alert fatigue.
Harnessing institutional knowledge: mixed methods evaluation of peer coaching in a multi-site EHR transition.
Brunner J, Sterling R, Cole E … +7 more , Weiss J, Faas B, Nosbush C, Ahlness EA, Wong ES, Rinne ST, Cutrona SL
J Am Med Inform Assoc · 2026 May · PMID 42143681 · Publisher ↗
OBJECTIVE: Health systems undertaking electronic health record (EHR) transitions often struggle to prepare and support clinicians in learning and using the new system. We evaluated a national peer coaching program-the Na...
OBJECTIVE: Health systems undertaking electronic health record (EHR) transitions often struggle to prepare and support clinicians in learning and using the new system. We evaluated a national peer coaching program-the National EHRM Supplemental Staffing Unit (NESSU)-designed to support clinicians during the U.S. Department of Veterans Affairs' (VA's) transition to a new EHR. Our goal was to assess NESSU's reach, perceived usefulness, and association with key EHR user outcomes, and to characterize how NESSU achieved its observed impacts. MATERIALS AND METHODS: Using a convergent mixed-methods design, we surveyed EHR users at the most recent VA facility to implement the new EHR. Descriptive statistics summarized program reach and perceived helpfulness. Regression models assessed associations between NESSU participation and 3 outcomes: burnout, EHR-related stress, and EHR confidence. Qualitative data included 62 interviews with users and open-ended survey responses. We used structured coding and thematic analysis to identify themes. RESULTS: Among 385 respondents, 58.4% reported receiving NESSU support and 83.6% of those rated it as helpful. NESSU participation was associated with lower rates of burnout (29% vs 41%, P = 0.016) but not with differences in EHR confidence or EHR-related stress. Qualitative analysis yielded 4 themes describing how NESSU functioned (filling education gaps, providing responsive support, offering expert guidance, and drawing upon notable interpersonal skills) and one theme describing its overall impact. DISCUSSION: Findings demonstrate that peer coaching can address important support needs during EHR transitions. CONCLUSION: Scalable, clinician-led peer coaching may represent an essential component of large-scale EHR transitions, supporting both implementation and clinician well-being.
Generative artificial intelligence for inpatient documentation summarization: mixed-methods quality assessment and early real-world experience.
Peters SG, Boyum JP, Legler SR … +3 more , Heise KJ, Griffin A, Heaton HA
J Am Med Inform Assoc · 2026 May · PMID 42133622 · Publisher ↗
OBJECTIVE: We evaluated the quality and adoption of a large language model (LLM)-based summarization tool for ongoing hospital care. MATERIALS AND METHODS: An AI summarization tool was created to provide a "Patient Story...
OBJECTIVE: We evaluated the quality and adoption of a large language model (LLM)-based summarization tool for ongoing hospital care. MATERIALS AND METHODS: An AI summarization tool was created to provide a "Patient Story", specialty-specific "Recent Notes", and "Recent Events" over the previous 24 or 72 hours. We conducted a pragmatic mixed-methods quality assessment at three tertiary-care academic hospitals utilizing (1) Provider Documentation Summarization Quality Instrument (PDSQI-9), (2) utilization analytics, and (3) qualitative end user feedback. The PDSQI-9 included whether summaries were accurate, cited, comprehensible, organized, succinct, non-stigmatizing, synthesized, thorough, and useful. 512 users were given access, from whom 52 participants submitted 205 surveys (10.2% response rate). RESULTS: 52 respondents submitted an average of 4.3 surveys (range 1-8). Users rated the tool favorably across all PDSQI-9 domains, with a combined average score of 4.68 (range 4.56-4.78, S.D. 0.67) across the eight domains scored on a 5-point modified Likert scale. Utilization metrics demonstrated strong uptake with frequent views. Positive qualitative feedback revealed cognitive offloading for complex patients and effective summarization of medical problems. Critical feedback showed a need to cross-reference narrative notes to current-state data and lack of detailed specialty-specific summarization. DISCUSSION: Generative artificial intelligence has emerged as a potentially transformative technology for generating succinct, verifiable summaries of ongoing care. In this pragmatic implementation, end-users indicated high perceived quality across PDSQI-9 domains. CONCLUSIONS: An LLM-based summarization tool for ongoing hospitalization care was rated of high quality by diverse clinicians in real-world settings, demonstrated a favorable safety profile, and showed sustained utilization.
Loss function influence on hyperparameter optimization for observational healthcare prediction models.
Vereijken F, Reps JM, Rijnbeek P … +1 more , Williams RD
J Am Med Inform Assoc · 2026 May · PMID 42133612 · Publisher ↗
OBJECTIVES: Prediction models are increasingly used in healthcare for risk stratification and personalized care. Many models are developed using machine learning, which requires tuning hyperparameters to maximize perform...
OBJECTIVES: Prediction models are increasingly used in healthcare for risk stratification and personalized care. Many models are developed using machine learning, which requires tuning hyperparameters to maximize performance based on a chosen loss function metric. In healthcare, the area under the receiver operating characteristic curve (AUROC) is commonly used for this purpose, but it may not always be the most appropriate choice for every clinical application. We empirically characterize whether the choice of loss function metric in hyperparameter optimization leads to systematic differences in model behavior across several clinical prediction tasks using real-world healthcare data. METHODS: We utilized fifteen different loss function metrics to guide hyperparameter selection across three clinical prediction tasks and four machine learning algorithms. We then compared how loss function metric choice affected selected hyperparameters, overall performance, and individual predicted probabilities. RESULTS: We observed that certain hyperparameters tended to have similar optimal values across different loss function metrics, although this pattern differed by algorithm. The best-performing models, evaluated using AUROC, were often not the models with hyperparameters optimized using AUROC. While models performed similarly at a population level, based on discrimination and calibration. The choice of the loss function metric had significant impact on the individual predicted risk for a patient. DISCUSSION: The predictive multiplicity observed can have significant impact on the patient level, while not observed in the population level model evaluation. CONCLUSION: Predictive multiplicity can have a serious impact on patient treatment decisions but is not yet well understood.
Translating innovation into payment: A.I. reimbursement in U.S. health care.
Bains SS, Mohan V, Gold JA
J Am Med Inform Assoc · 2026 Jul · PMID 42128881 · Full text
Artificial intelligence (A.I.) technologies are increasingly deployed across clinical care, yet reimbursement remains a major barrier to sustainable adoption. The Centers for Medicare & Medicaid Services (CMS) currently...
Artificial intelligence (A.I.) technologies are increasingly deployed across clinical care, yet reimbursement remains a major barrier to sustainable adoption. The Centers for Medicare & Medicaid Services (CMS) currently reimburses A.I.-enabled technologies through a fragmented set of procedural codes, add-on payments, and legacy payment models, none of which were designed to support the complexity or workflow integration of clinical A.I. This article examines how existing CMS reimbursement pathways for A.I. function in practice, identifies structural misalignments that limit adoption, and highlights the risks of continuing to rely on these approaches. We then propose policy-level solutions to modernize A.I. reimbursement, including clearer coverage pathways, value-aligned payment models, and mechanisms to promote equitable adoption across diverse healthcare settings. Aligning reimbursement with clinical value is essential to ensure that A.I. improves care delivery and can be sustainably integrated into routine clinical practice.
What do clinicians edit in ambient AI-drafted clinical documentation? A qualitative content analysis.
Guo Y, Hu D, Yang Z … +11 more , Kim S, Tran B, Lee J, Vallabhaneni S, Zehrung R, Sutari S, Tam S, Chow E, Perret D, Pandita D, Zheng K
J Am Med Inform Assoc · 2026 May · PMID 42118974 · Publisher ↗
OBJECTIVE: Ambient artificial intelligence (AI) documentation is increasingly used to draft clinical notes from patient-provider conversations, but how clinicians revise and finalize these drafts is not well understood....
OBJECTIVE: Ambient artificial intelligence (AI) documentation is increasingly used to draft clinical notes from patient-provider conversations, but how clinicians revise and finalize these drafts is not well understood. This qualitative content analysis study characterizes real-world edits to AI-generated drafts and identifies opportunities for improvement of AI design and the implementation process. MATERIALS AND METHODS: Eight coders analyzed clinical documentation generated by ambient AI from 200 clinical encounters. We developed an inductive coding framework with 11 codes across 3 categories: clinical content, terminology, and language style. Interrater reliability was assessed using Cohen's kappa. We then applied thematic analysis to synthesize patterns across the coded edits. RESULTS: The most frequently edited content pertained to clinical facts including orders (eg, procedures, lab tests) (40.0%), symptoms (30.3%), medication prescriptions (27.3%), and diagnosis descriptions (25.9%). In comparison, edits related to terminology use (11.6%) and language style (7.2%) were less frequent. The results of our thematic analysis show that most edits can be categorized into one of the following 5 types: to revise factual discrepancies, to add medical specialty-specific details, to express diagnostic certainties, to convert patient expressions into objective assessments recorded in medical terms, and to reorganize or condense content. CONCLUSION AND DISCUSSION: Clinicians routinely revise ambient AI drafts to modify factual details and clinical specificity. Future work on AI development and clinical implementation should emphasize specialty customization and support personalized documentation practices, alongside clinician education that promotes robust and consistent review routines to ensure documentation quality.
Sociodemographic bias in large language model clinical trial screening.
Soffer S, Omar M, Efros O … +5 more , Apakama DU, Mudrik A, Freeman R, Nadkarni GN, Klang E
J Am Med Inform Assoc · 2026 May · PMID 42118957 · Publisher ↗
OBJECTIVE: To assess whether large language model (LLM)-based clinical trial screening judgments vary by patient sociodemographic characteristics. MATERIALS AND METHODS: We conducted a cross-sectional evaluation of Phase...
OBJECTIVE: To assess whether large language model (LLM)-based clinical trial screening judgments vary by patient sociodemographic characteristics. MATERIALS AND METHODS: We conducted a cross-sectional evaluation of Phase II-III US adult randomized controlled trial (RCT) protocols (2023-2024). Physician-validated clinical vignettes were evaluated in a control version and 33 sociodemographic identity variants differing only by labels. Nine LLMs assessed eligibility and related domains. Mixed-effects models estimated adjusted differences vs control. RESULTS: Across 58 protocols and 5.3 million evaluations, eligibility judgments were largely stable across identities. Race and ethnicity showed minimal effects after accounting for socioeconomic status. Homelessness produced the largest negative eligibility shift and pronounced effects in adherence, resources, and trust. DISCUSSION AND CONCLUSION: LLMs applied explicit eligibility criteria consistently, but disparities emerged in domains requiring inference about behavior or resources, underscoring the need for careful deployment to promote fair trial access.
Exploring patient motivations and preferences for medical data sharing with researchers: a simulation study using the iAgree platform.
Keller MS, Nguyen AT, Leder C … +6 more , Chen Y, Morse B, Schilling LM, Hu D, SooHoo S, Ohno-Machado L
J Am Med Inform Assoc · 2026 Jul · PMID 42108209 · Full text
OBJECTIVE: This study explores patient motivations and preferences for sharing medical data with researchers using the iAgree platform. We examine how study characteristics, including data type requested and data-sharing...
OBJECTIVE: This study explores patient motivations and preferences for sharing medical data with researchers using the iAgree platform. We examine how study characteristics, including data type requested and data-sharing arrangements, influence consent decisions, and assess the role of demographic factors, privacy concerns, and perceived benefits in shaping data-sharing behavior. MATERIALS AND METHODS: We conducted a mixed-methods study with 527 US adults (≥18 years) recruited via advisory boards, social media, clinics, and newsletters. Participants completed 3 of 4 simulated studies on iAgree, each varying by data elements requested and data-sharing scope. Participants provided consent and data-sharing decisions and completed a post-simulation survey capturing demographics, data-sharing motivations, privacy concerns, and patient activation. We used logistic regressions to examine associations between demographics, privacy concerns, and patient activation and: (1) consent status and (2) willingness to share particular data elements. Finally, we applied thematic analysis to open-ended responses. RESULTS: Consent status did not significantly vary by data type or study design. However, participants citing altruism, personal benefit, and patient solidarity were more likely to share data. Higher privacy concerns were linked to lower willingness to share family health and mental health information. Participants with higher patient activation were also less likely to share data. DISCUSSION: Demographic factors were not significantly associated with consent or willingness to share data, countering common assumptions about disparities in sharing preferences. CONCLUSION: Altruism and perceived benefit drive willingness to share health data, while privacy concerns and patient activation may reduce it, emphasizing the need for patient-centered, transparent consent models.
The detectability paradox: bilingual medical report generation with open-weight models and the limits of human oversight.
Rouhizadeh H, Sandralegar A, Yazdani A … +11 more , Feng W, Schreier O, Ahn-Kim Y, Sirbal A, Pirelli V, Yang R, Sveikata L, Tessitore E, Liu N, Bijlenga P, Teodoro D
J Am Med Inform Assoc · 2026 Jul · PMID 42097830 · Full text
OBJECTIVES: The automation of medical report generation using large language models (LLMs) could significantly reduce physicians' documentation burden while enhancing healthcare efficiency. However, the misuse of generat...
OBJECTIVES: The automation of medical report generation using large language models (LLMs) could significantly reduce physicians' documentation burden while enhancing healthcare efficiency. However, the misuse of generative artificial intelligence in medical reporting can lead to important safety risks for patients. We addressed 2 questions: (1) What is the quality of medical reports generated by LLMs in English and French? and (2) Can we distinguish between human-written and LLM-generated medical reports? MATERIALS AND METHODS: We evaluated the quality of reports generated by several multilingual, open-weight LLMs using text similarity metrics on 4212 medical reports in English and French across multiple specialties. A bilingual expert panel of certified physicians (n = 4) and medical residents (n = 5) scored accuracy, fluency, and completeness of generated reports using a 1-5 Likert scale. Experts also completed a Turing-like test, blindly identifying reports as human or machine-generated. RESULTS: Phi-4 achieved the best overall performance (ROUGE-1: 0.70, BERTScore: 0.83). Expert evaluation confirmed high-quality reports in both languages (overall 4.6/5.0). Medical experts performed better than chance but struggled to differentiate human versus machine reports (accuracy: 0.60). Automatic classifiers showed strong performance (accuracy: 0.98). DISCUSSION: The high quality of LLM-generated reports supports their potential to enhance healthcare efficiency in multilingual settings. However, the discrepancy between human detection difficulty and automated detection success reveals inherent limitations in relying solely on human oversight for quality assurance and misuse prevention. CONCLUSIONS: Deployment of LLMs for medical reporting requires combining automated detection tools with human expertise to ensure patient safety. Dataset and code: https://github.com/ds4dh/medical_report_generation.
Enhancing validation of case-control omics signatures through "minimalist" single-subject analysis (N-of-1 trials): proof of concept in sepsis.
Wilson LS, Pouladi N, Nelson RF … +7 more , Middleton EA, Tolley ND, Shabanian M, Kenost C, Campbell RA, Rondina MT, Lussier YA
J Am Med Inform Assoc · 2026 Jul · PMID 42093161 · Full text
OBJECTIVE: To evaluate if a single-subject study (S3) design, utilizing paired transcriptome samples from the same patient (eg, "sepsis" vs "recovered"), can replicate transcriptomic signatures from small case-control st...
OBJECTIVE: To evaluate if a single-subject study (S3) design, utilizing paired transcriptome samples from the same patient (eg, "sepsis" vs "recovered"), can replicate transcriptomic signatures from small case-control studies, addressing challenges in patient accrual for rare or sub-stratified diseases. METHODS: We generated a sepsis gene signature (SGS) comprising 300 differentially expressed genes (DEGs; FDR < 5%) from a human sepsis case-control cohort using general linear models (GLMs). Reproducibility of SGS was assessed through three approaches applied to sub-sampled independent datasets: single-subject analyses (N-of-1-MixEnrich), anticipated to perform better; conventional paired-sample GLM analyses; and a traditional case-control GLM analysis. RESULTS: SGS reproducibility in GLM analyses was inconsistent at smaller cohort sizes (∼80% reproducibility; n = 5) but stabilized at cohort sizes >6. Remarkably, the single-subject-study approach consistently reproduced SGS in each of the 18 subjects individually (100% reproducibility; n = 1). DISCUSSION: Conventional GLMs are not designed for single-subject or small cohort analyses due to their dependence on larger samples to mitigate variable dispersion and human heterogeneity. In contrast, S3 methods enhance statistical power by: reducing multiple testing through gene set aggregation, emphasizing concordant changes in pathway activity rather than exact molecular consistency, and exploiting paired samples from the same individual. CONCLUSION: This proof-of-concept demonstrates that S3 designs effectively validate gene expression signatures derived from case-control studies, highlighting their potential in research or clinical trials constrained by small sample sizes. However, further validation and computational simulation are needed to demonstrate scalability to other conditions and sensitivity to validation subject variations from the "average subject" of discovery cohorts.
Disparate language and model effects on AI-based translation and recognition of genetic conditions.
Duong D, Manoli I, Phadke SR … +3 more , Phornphutkul C, Raymond JD, Solomon BD
J Am Med Inform Assoc · 2026 Jul · PMID 42090313 · Full text
INTRODUCTION: Artificial intelligence (AI) is increasingly prevalent. Patients and clinicians may use AI-based tools in many different languages. OBJECTIVE: To investigate AI translation tools for descriptions of genetic...
INTRODUCTION: Artificial intelligence (AI) is increasingly prevalent. Patients and clinicians may use AI-based tools in many different languages. OBJECTIVE: To investigate AI translation tools for descriptions of genetic conditions and how AI identification of genetic conditions is affected by translations. MATERIALS AND METHODS: We used Neural machine translation (NMT) and large language-model (LLM) translation to translate descriptions of 40 genetic conditions into 191 and 93 languages, respectively. Excluding translations retaining English medical terms verbatim, we respectively focused on 139 and 70 languages. After assessing translations, we assessed the ability of 3 proprietary and 3 open-weight general LLMs to identify conditions in the translations. We analyzed how accuracy was affected by the conditions' prevalence in the literature, and attributes of the languages (the script, language family, and prevalence of the language in training sources). We also investigated adaptive translation for select languages. RESULTS: We found significant differences in condition identification based on the translation method, condition, language, and prediction model. The accuracy of some models was more affected than others by factors like the conditions' literature prevalence, language script, family, and language prevalence. Adaptive translation for select languages did not improve translations or diagnostic accuracy with the 3 tested LLMs. However, further analysis with 1 language showed that this approach was more effective with smaller LLMs. CONCLUSIONS: AI-based translation has variable performance, which can affect the ability of AI models to recognize genetic conditions. These findings should inform safe medical AI use to support consistent performance in different languages.
TRUST: an large language model-based dialogue system for trauma understanding and structured assessments.
Tu S, Powers A, Doogan S … +1 more , Choi JD
J Am Med Inform Assoc · 2026 Jul · PMID 42089587 · Full text
OBJECTIVES: While large language models (LLMs) have been widely used to assist clinicians and support patients, no existing work has explored dialogue systems for standard diagnostic interviews and assessments. This stud...
OBJECTIVES: While large language models (LLMs) have been widely used to assist clinicians and support patients, no existing work has explored dialogue systems for standard diagnostic interviews and assessments. This study aims to bridge the gap in mental healthcare accessibility by developing an LLM-powered dialogue system that replicates clinician behavior. MATERIALS AND METHODS: We introduce TRUST, a framework of cooperative LLM modules capable of conducting formal diagnostic interviews and assessments for post-traumatic stress disorder (PTSD) following the Clinician-Administered PTSD Scale for DSM-5 (CAPS-5). To guide the generation of appropriate clinical responses, we propose a Dialogue Acts schema specifically designed for clinical interviews. Additionally, we develop a patient simulation approach based on real-life interview transcripts to replace time-consuming and costly manual testing by clinicians. RESULTS: A comprehensive set of evaluation metrics is designed to assess the dialogue system from both the agent and patient simulation perspectives. Expert evaluations by conversation and clinical specialists show that TRUST performs comparably to real-life clinical interviews. DISCUSSION: Our system performs with clinical quality approaching that of human clinicians, with room for future enhancements in communication styles and response appropriateness. CONCLUSIONS: Our TRUST framework shows its potential to facilitate mental healthcare availability.
Hybrid care engagement phenotypes and glycemic outcomes in diabetes: a cluster analysis across two health systems.
Clifford N, Kemper-McIsaac KE, Yu H … +3 more , Rapson T, Sarkar U, Khoong EC
J Am Med Inform Assoc · 2026 Jul · PMID 42089583 · Full text
OBJECTIVE: Prior studies often examine single telehealth encounter types or aggregate all digital care, overlooking how patients combine multiple digital and in-person modalities in hybrid care. To address this gap, we d...
OBJECTIVE: Prior studies often examine single telehealth encounter types or aggregate all digital care, overlooking how patients combine multiple digital and in-person modalities in hybrid care. To address this gap, we derived hybrid care engagement phenotypes and assessed sociodemographic differences and associations with glycemic control among adults with type 2 diabetes (T2DM). METHODS: We conducted a retrospective cohort study of 10 671 adults with T2DM receiving primary care at an academic (UCSF) or safety-net system (SFHN) from April 2021 to March 2023. K-medoids clustering was applied to five encounter modalities (in-person, video, telephone visits; portal messages; unscheduled telephone calls) to derive four engagement phenotypes. We assessed sociodemographic differences using chi-square and Kruskal-Wallis tests and evaluated associations between phenotype and follow-up HbA1c control using logistic regression. We tested interactions with baseline HbA1c and estimated predicted probabilities using Tukey-adjusted contrasts. RESULTS: Four phenotypes emerged per system: Digitally Engaged Multimodal, Traditional High Utilizers, Digitally Leaning (UCSF), Telephone Reliant (SFHN), and Low Digital. UCSF patients belonged to digitally forward phenotypes, whereas SFHN patients concentrated in traditional, lower-tech phenotypes. Among patients with uncontrolled diabetes, digitally forward phenotypes had 13-20 percentage points higher predicted probability of achieving control (UCSF: 56% Digitally Leaning vs 36% Traditional; SFHN: 53% Multimodal vs 40% Telephone). DISCUSSION: Phenotypes varied by health system and sociodemographic factors, with modest, system-specific associations between digitally forward phenotypes and glycemic control among patients with uncontrolled diabetes. Findings underscore structural and sociodemographic inequities in hybrid care engagement and the need for proactive, tailored strategies to promote equitable hybrid care.
Engineering biomarker representations of vital signs data enhances deep learning mortality prediction.
Mamandipoor B, Shen I, Hsu CN … +1 more , Gabriel RA
J Am Med Inform Assoc · 2026 Jul · PMID 42068567 · Full text
OBJECTIVES: We evaluated bidirectional long short-term memory models for predicting inpatient mortality using different approaches to processing vital signs data collected during the initial 24 h of intensive care unit (...
OBJECTIVES: We evaluated bidirectional long short-term memory models for predicting inpatient mortality using different approaches to processing vital signs data collected during the initial 24 h of intensive care unit (ICU) admissions. MATERIALS AND METHODS: We compared 3 vital-sign representations: (1) raw data recorded every 5 min, (2) preprocessed data averaged hourly, and (3) preprocessed data using biomarker representations that extends a digital oximetry biomarker toolbox of PhysioZoo software, applied to blood pressure, heart rate, temperature, respiratory rate, and SpO2. RESULTS: Across 2 large ICU datasets, HiRID and eICU, models trained on the frequency-normalized representation achieved higher discrimination and lower Brier scores than those trained on raw 5-min and hourly averaged data. DISCUSSION: The use of biomarker representations of vital signs yielded the largest improvements in discrimination and overall probabilistic performance reflected by lower Brier scores for predicting inpatient mortality by deep learning. CONCLUSION: Thus, we recommend using a similar approach to vital signs preprocessing for time-series predictive models.
Optimising pain identification in resource-limited emergency departments using transfer learning and fine-tuned language models.
Wu Y, Hughes JA, Judge C … +2 more , Appo C, Nguyen A
J Am Med Inform Assoc · 2026 Jul · PMID 42066226 · Full text
OBJECTIVE: To optimise the identification of patients presenting with pain in emergency department (ED) settings with limited resources using multiple transfer learning techniques. METHODS: Two strategies were explored:...
OBJECTIVE: To optimise the identification of patients presenting with pain in emergency department (ED) settings with limited resources using multiple transfer learning techniques. METHODS: Two strategies were explored: (1) fine-tuning a pre-trained language model, previously fine-tuned on data from a well-resourced ED, using labelled data from a target ED, and (2) continual pre-training using task-specific unlabelled data to enhance clinical text classification. RESULTS: With 2000 labelled samples from a target ED, the combined strategies achieved an F1-score of 92%, demonstrating significant benefits of transfer learning in resource-constrained settings. DISCUSSION: Accurately identifying pain in patients upon arrival to the ED is crucial for timely and effective treatment. Findings suggest that combining both transfer learning strategies can significantly enhance pain identification performances in resource-constrained settings. CONCLUSION: Combining fine-tuning on labelled data and continual pre-training on unlabelled data has potential to optimise model performance in both resource-constrained and well-resourced settings, highlighting the broader applicability and potential of these techniques for improving clinical text classification.
Healthcare providers and human-centered health informatics and artificial intelligence.
Bakken S
J Am Med Inform Assoc · 2026 May · PMID 42054484 · Full text
Abstract loading — click title to view on PubMed.
Clinicians' rationale for editing ambient AI-drafted clinical notes: persistent challenges and implications for improvement.
Guo Y, Hu D, Yang Z … +5 more , Chow E, Tam S, Perret D, Pandita D, Zheng K
J Am Med Inform Assoc · 2026 Jul · PMID 42044151 · Full text
OBJECTIVE: The use of ambient AI documentation tools is rapidly growing in US hospitals and clinics. Such tools generate the first draft of clinical notes from scribed patient-provider conversations, which clinicians can...
OBJECTIVE: The use of ambient AI documentation tools is rapidly growing in US hospitals and clinics. Such tools generate the first draft of clinical notes from scribed patient-provider conversations, which clinicians can then review and edit before signing into electronic health records (EHR). Understanding how and why clinicians make modifications to AI-generated drafts is critical to improving AI design and clinical efficiency, yet it has been under-studied. This study aims to address this gap. MATERIALS AND METHODS: We conducted semistructured interviews with 30 clinicians from the University of California, Irvine Health who used a commercial ambient AI tool in routine outpatient care. We invited them to describe how and why they edited AI drafts based on both their personal experience and review of some real-world examples identified from our previous studies. RESULTS: Modifications to AI drafts were primarily made to improve clinical accuracy and specialty-specific precision, reduce medico-legal and liability risk, and meet billing, coding, and documentation standards. Such editing was necessary due to reasons such as transcription errors, speaker attribution mistakes, overconfident statements without evidence, missing key clinical details, and AI's lack of information about the patient context. CONCLUSION AND DISCUSSION: Improving ambient AI documentation will require coordinated effort from vendors, institutions, and clinicians. Key targets include core model reliability (eg, transcription accuracy), specialty- and encounter-level customization, clinician-level personalization, more effective EHR integration, and institutional support (eg, training, governance, and standardized review guidance), complemented by clinicians' adaptive communication strategies that strengthen human-AI collaboration.
About
- Frequency
- Sun
- Papers found
- 200
- RSS feed
- Subscribe