Journal Of The American Medical Informatics Association[JOURNAL]
Sun
200 papers
Improving patient understanding of radiology reports using generative artificial intelligence: a vignette study of 2000 US adults.
Chen AH, Rudin RS, Levine DM … +1 more , Mehrotra A
J Am Med Inform Assoc · 2026 Feb · PMID 41241949 · Full text
OBJECTIVES: Patients value access to their medical reports on patient portals, but the terminology in those reports can cause confusion and anxiety. Can the artificial intelligence (AI) simplification of radiology report...
OBJECTIVES: Patients value access to their medical reports on patient portals, but the terminology in those reports can cause confusion and anxiety. Can the artificial intelligence (AI) simplification of radiology reports into plain language improve patient comprehension? MATERIALS AND METHODS: Twenty original radiology reports (breast imaging, chest X-ray) were simplified into plain language using ChatGPT-4 using a customized prompt for each report type. For each report, clinicians created a gold standard of key findings and appropriate follow-up. In August 2024, a national sample of 2000 US adults reviewed 2 randomly assigned reports, 1 original and 1 AI-generated plain language. Participants answered questions focused on comprehension of key findings, follow-up, confidence, anxiety, and preferences. Comprehension and follow-up were compared to the gold standard. We compared patient accuracy for original vs AI-generated plain language reports. RESULTS: Participants (mean age 48 years) were 62.3% female. Compared to original reports, participants shown AI-generated plain language reports had higher accuracy in comprehension (68.0% vs 58.0%; marginal difference 10.8% [95% CI, 7.8%-13.8%]) and follow-up (64.5% vs 58.4%; marginal difference 6.8% [95% CI, 4.1%-9.4%]). Improvements were larger among participants aged >44 years and with less than college education. With plain language reports, participants reported higher confidence in their answers and lower anxiety. Despite these improvements, 60.0% of participants preferred the original report over the plain language version. DISCUSSION: Integrating AI simplification into patient portals may be helpful, but trust concerns remain. CONCLUSION: AI simplification improved patient comprehension and confidence. Further research is needed to address patient resistance to AI simplification.
Development of a robust corpus for automated evaluation of online health information in Chinese using the DISCERN scale.
E T, Li X, Liang J … +5 more , Ma J, Fang Q, Chen S, Lei J, Chute CG
J Am Med Inform Assoc · 2026 Feb · PMID 41223037 · Full text
OBJECTIVE: To develop the first comprehensive, standardized annotated corpus of Chinese online health information (OHI) using the full 16-item DISCERN instrument and to establish a reliable annotation process that suppor...
OBJECTIVE: To develop the first comprehensive, standardized annotated corpus of Chinese online health information (OHI) using the full 16-item DISCERN instrument and to establish a reliable annotation process that supports automated quality assessment. MATERIALS AND METHODS: We assembled 510 web-sourced articles on breast cancer, arthritis, and depression. All the articles were independently annotated by three trained raters using the DISCERN scale. Annotation followed a four-step workflow: data collection and preprocessing, rater training, iterative annotation, and quality control. Raters calibrated through consensus sessions and calibration articles. The Dawid-Skene model aggregated individual annotations into final consensus scores. Original five-point ratings were retained and binarized (scores 1-3 as low quality, 4-5 as high quality) to enable both fine-grained and coarse evaluation for machine learning. RESULTS: Initial annotation of a 60-article pilot produced low agreement (mean Krippendorff's α ≈ 0.022) due to subjective variability. Successive calibration exercises improved agreement markedly, culminating in a corpus-wide Krippendorff's α of 0.834. Consensus scores correlated strongly with individual rater scores, confirming annotation robustness. The dual-scale design yielded a relatively balanced distribution of labels across topics, with roughly equal representation of low- and high-quality articles, and preserved granularity for detailed DISCERN analysis. DISCUSSION: Our iterative calibration approach and consensus modeling effectively addressed the subjective ambiguity inherent in quality assessment. The binary and five-class labeling strategies facilitate flexible downstream applications, allowing automated systems to perform both broad filtering and nuanced quality differentiation. The high inter-rater reliability demonstrates that rigorous training and consensus methods can overcome domain-specific annotation challenges. CONCLUSION: The resulting Chinese OHI corpus, annotated via a standardized DISCERN framework and refined through iterative calibration, provides a robust benchmark for training and evaluating machine learning models. This resource lays the foundation for scalable, reliable automated quality assessment of OHI in Chinese public health settings.
PhenoFit: a framework for determining computable phenotyping algorithm fitness for purpose and reuse.
Wiley LK, Rasmussen LV, Levinson RT … +9 more , Malinowski J, Manemann SM, Wilson MP, Chapman M, Pacheco JA, Walunas TL, Starren JB, Bielinski SJ, Richesson RL
J Am Med Inform Assoc · 2026 Feb · PMID 41223026 · Full text
BACKGROUND: Computational phenotyping from electronic health records (EHRs) is essential for clinical research, decision support, and quality/population health assessment, but the proliferation of algorithms for the same...
BACKGROUND: Computational phenotyping from electronic health records (EHRs) is essential for clinical research, decision support, and quality/population health assessment, but the proliferation of algorithms for the same conditions makes it difficult to identify which algorithm is most appropriate for reuse. OBJECTIVE: To develop a framework for assessing phenotyping algorithm fitness for purpose and reuse. FITNESS FOR PURPOSE: Phenotyping algorithms are fit for purpose when they identify the intended population with performance characteristics appropriate for the intended application. FITNESS FOR REUSE: Phenotyping algorithms are fit for reuse when the algorithm is implementable and generalizable-that is, it identifies the same intended population with similar performance characteristics when applied to a new setting. CONCLUSIONS: The PhenoFit framework provides a structured approach to evaluate and adapt phenotyping algorithms for new contexts increasing efficiency and consistency of identifying patient populations from EHRs.
A novel, standardised approach to balancing effectiveness, efficiency and utility of surveillance AI prediction models for hospitalised patients using sepsis prediction as an exemplar.
van der Vegt AH, Campbell VK, Webb R … +12 more , Venkatesh B, Lane PJ, Wilks K, McPhail S, Rice M, Isaacs T, Abdel-Hafez A, Whebell S, Irwin A, Schnetler RJ, Shetty A, Scott IA
J Am Med Inform Assoc · 2026 Feb · PMID 41217035 · Full text
OBJECTIVE: To introduce a novel, standardised approach to evaluating AI prediction models in balancing effectiveness, efficiency and utility, using a sepsis prediction model case study. MATERIALS AND METHODS: Retrospecti...
OBJECTIVE: To introduce a novel, standardised approach to evaluating AI prediction models in balancing effectiveness, efficiency and utility, using a sepsis prediction model case study. MATERIALS AND METHODS: Retrospective patient data from electronic medical records of 7 public hospitals was used to retrain and evaluate a machine learning sepsis prediction model. Four conventional metrics-area under the receiver operating curve (AUROC), sensitivity, positive predictive value, and specificity-were compared with a novel graphical display integrating metrics of predictive accuracy (effectiveness), alert burden (efficiency) and lead time of alerts relative to clinical events (utility) for different alert thresholds. RESULTS: The dataset comprised 977,506 inpatient admissions. The novel methodology produced a plot of four vertically aligned graphs that enables decision-makers to identify an alert threshold that optimally balances effectiveness, efficiency and utility (EEU) at the level of an entire admission, and which differs from that derived using conventional metrics. DISCUSSION: Conventional evaluation metrics do not consider alert timing relative to clinical events and are often applied to different evaluation datasets (sample and admission level), introducing bias and confusion. In contrast, the EEU methodology (i) generates admission level evaluations at different alert thresholds; (ii) measures alert timing relative to clinical events; and (iii) provides a visual display that enables identification of the alert threshold that optimally balances EEU factors. CONCLUSION: Evaluations of prediction models for adverse events in hospitalised patients should incorporate the EEU approach in assessing model suitability and selecting alert thresholds.
Supporting electronic health record data usage in research for teams with varying data science and clinical knowledge: a food service analogy approach.
Magoc T, Tang LA, Nguyen KA … +1 more , Harle CA
J Am Med Inform Assoc · 2026 Feb · PMID 41211699 · Full text
OBJECTIVE: To guide research data services (RDS) teams in managing researcher variability (eg, differing deadlines, funding, expertise) when honest-brokering data, we present a framework based on operations management pr...
OBJECTIVE: To guide research data services (RDS) teams in managing researcher variability (eg, differing deadlines, funding, expertise) when honest-brokering data, we present a framework based on operations management principles and a food service analogy. MATERIALS AND METHODS: Our framework describes 4 data service offerings with different levels of efficiency and service customization: vending machine, fast food, custom meal, and personal chef. We describe examples from 2 institutions. RESULTS: Vending machine and fast food are efficient but less customizable, making them better-suited for researchers with limited funding or time. Custom meal and personal chef are less efficient but more customized, making them well suited for better-resourced researchers. DISCUSSION: Efficiency and service tradeoffs should be balanced to align with demand and institutional goals. RDS teams can overcome such tradeoffs through uncompromised reduction or low-cost accommodation approaches. CONCLUSION: Our framework can be applied by RDS teams in their design and implementation of data services.
Assessing genetic counseling efficiency with natural language processing.
Nguyen MH, Applegate CD, Murray B … +6 more , Zirikly A, Tichnell C, Gordon C, Yanek LR, James CA, Taylor CO
J Am Med Inform Assoc · 2026 Feb · PMID 41211696 · Full text
OBJECTIVE: To build natural language processing (NLP) strategies to characterize measures of genetic counseling (GC) efficiency and classify measures according to phase of GC (pre- or post-genetic testing). MATERIALS AND...
OBJECTIVE: To build natural language processing (NLP) strategies to characterize measures of genetic counseling (GC) efficiency and classify measures according to phase of GC (pre- or post-genetic testing). MATERIALS AND METHODS: This study selected and annotated 800 GC notes from 7 clinical specialties in a large academic medical center for NLP model development and validation. The NLP approaches extracted GC efficiency measures, including direct and indirect time and GC phase. The models were then applied to 24 102 GC notes collected from January 2016 through December 2023. RESULTS: NLP approaches performed well (F1 scores of 0.95 and 0.90 for direct time in GC and GC phase classification, respectively). Our findings showed median direct time in GC of 50 minutes, with significant differences in direct time distributions observed across clinical specialties, time periods (2016-2019 or 2020-2023), delivery modes (in person or telehealth), and GC phase. DISCUSSION: As referrals to GC increase, there is increasing pressure to improve efficiency. Our NLP strategy was used to generate and summarize real-world evidence of GC time for 7 clinical specialties. These approaches enable future research on the impact of interventions intended to improve GC efficiency. CONCLUSION: This work demonstrated the practical value of NLP to provide a useful and scalable strategy to generate real world evidence of GC efficiency. Principles presented in this work may also be valuable for health services research in other practice areas.
Automated detection of stigmatizing language in Electronic Health Records (EHRs) using a multi-stage transfer learning approach.
Xue L, Rahman AMM, Senteio CR … +1 more , Singh VK
J Am Med Inform Assoc · 2026 Feb · PMID 41206907 · Full text
OBJECTIVE: Stigmatizing language (SL) in Electronic Health Records (EHRs) can perpetuate biases and negatively impact patient care. This study introduces a novel method for automatically detecting such language to improv...
OBJECTIVE: Stigmatizing language (SL) in Electronic Health Records (EHRs) can perpetuate biases and negatively impact patient care. This study introduces a novel method for automatically detecting such language to improve healthcare documentation practices. MATERIALS AND METHODS: We developed a multi-stage transfer learning framework integrating semantic, syntactic, and task adaptation using three datasets: hate speech, clinical phenotypes, and stigmatizing language. Experiments were conducted on stigmatizing language dataset which consists of 4,129 de-identified EHR notes (72.7% stigmatizing, 27.3% non-stigmatizing), split 80/20 for training and testing. Longformer, BERT, and ClinicalBERT models were evaluated, and model performance was assessed on 35 randomized subsets of the test set (each comprising 70% of test data). The Wilcoxon-Mann-Whitney test was used to evaluate statistical significance, with Bonferroni correction applied to control for multiple hypothesis testing. Baseline models included zero-shot and few-shot GPT-4o, Support Vector Machine, Random Forest, Logistic Regression, and Multinomial Naive Bayes. RESULTS: The proposed framework achieved the highest accuracy, with fully adapted Longformer reaching 89.83%. Performance improvements remained statistically significant after Bonferroni correction compared to all baselines (p < .05). The framework demonstrated robust gains across different stigmatizing language types. DISCUSSION: This study underscores the value of domain-adaptive NLP for detecting stigmatizing language in EHRs. The multi-stage transfer learning framework effectively captures subtle biases often missed by conventional models, enabling more objective and respectful clinical documentation. CONCLUSION: This framework offers a statistically validated, high-performing framework for detecting stigmatizing language in EHRs, supporting responsible AI and promoting equity in clinical care.
A scoping review of models to identify transgender patients in electronic health records.
Becker RA, Kolli JUL, Walsh CG
J Am Med Inform Assoc · 2026 Feb · PMID 41189089 · Full text
OBJECTIVE: Electronic health records (EHRs) lack a widely adopted standard for recording transgender and gender diverse (TGD) status, complicating research on TGD health. Computational models have been developed to ident...
OBJECTIVE: Electronic health records (EHRs) lack a widely adopted standard for recording transgender and gender diverse (TGD) status, complicating research on TGD health. Computational models have been developed to identify TGD individuals in EHRs; however, gaps remain in understanding which components contribute to stronger phenotyping approaches. This scoping review evaluates EHR-based models for identifying TGD individuals, focusing on identifier types, performance, external validation, and ethical reporting to guide best practices. MATERIALS AND METHODS: We searched PubMed, CINAHL, Web of Science, and Embase for peer-reviewed articles published before January 2024, following PRISMA-ScR guidelines. Included studies used EHR data to identify TGD individuals, verified TGD status, reported or allowed calculation of positive predictive value (PPV), and listed identifiers. Two authors screened and extracted data. We categorized models by data type and logic (structured, unstructured, and multimodal), summarized PPV distributions, and synthesized author-reported ethical considerations. RESULTS: Fourteen studies describing 50 models met inclusion criteria. Models using TGD-related diagnostic codes alone (n = 11) or requiring both structured and unstructured data (n = 6) showed the highest mean PPVs (85.3% and 97.1%). Models validated on larger confirmed TGD cohorts reported more stable performance, but external validation was rare. Most studies minimally addressed ethics; only 3 described protective measures or stakeholder engagement. DISCUSSION: Phenotyping of TGD individuals in EHR data remains heterogeneous in design and ethical transparency. Reported PPVs should be interpreted cautiously, as performance is influenced by study design, sample size, and verification methods. CONCLUSIONS: Our recommendations emphasize the components that strengthen phenotyping approaches-identifier choice, multimodal intersection logic, validation practices, and ethical safeguards-rather than endorsing any single model.
Enterprise-wide simultaneous deployment of ambient scribe technology: lessons learned from an academic health system.
Wright AP, Nall CK, Franklin JJH … +4 more , Horst SN, Kumah-Crystal YA, Wright AT, Mize DE
J Am Med Inform Assoc · 2026 Feb · PMID 41175896 · Full text
OBJECTIVES: To report on the feasibility of a simultaneous, enterprise-wide deployment of EHR-integrated ambient scribe technology across a large academic health system. MATERIALS AND METHODS: On January 15, 2025, ambien...
OBJECTIVES: To report on the feasibility of a simultaneous, enterprise-wide deployment of EHR-integrated ambient scribe technology across a large academic health system. MATERIALS AND METHODS: On January 15, 2025, ambient scribing was made available to over 2400 ambulatory and emergency department clinicians. We tracked utilization rates, technical support needs, and user feedback. RESULTS: By March 31, 2025, 20.1% of visit notes incorporated ambient scribing, and 1223 clinicians had used ambient scribing. Among 209 respondents (22.1% of 947 surveyed), 90.9% would be disappointed if they lost access to ambient scribing, and 84.7% reported a positive training experience. DISCUSSION: Enterprise-wide simultaneous deployment combined with a low-barrier training model enabled immediate access for clinicians and reduced administrative burden by concentrating go-live efforts. Support needs were manageable. CONCLUSION: Simultaneous enterprise-wide deployment of ambient scribing was feasible and provided immediate access for clinicians.
Interpretable machine learning for identifying ICU readmission risk in subgroups with probabilistic rules.
Yang L, van der Meijden SL, Arbous SM … +1 more , van Leeuwen M
J Am Med Inform Assoc · 2026 Mar · PMID 41159698 · Full text
OBJECTIVE: Estimating readmission risk for intensive care unit (ICU) patients is critical for clinicians to optimize resource allocation and prevent premature discharges. Machine learning models currently applied to this...
OBJECTIVE: Estimating readmission risk for intensive care unit (ICU) patients is critical for clinicians to optimize resource allocation and prevent premature discharges. Machine learning models currently applied to this task either lack interpretability or cannot identify patient subgroups with distinctive readmission risks and characteristics. We addressed this gap by introducing a cutting-edge rule-based model, namely truly unordered rule sets (TURS), to reveal heterogeneous readmission risks and subgroup-level patient characteristics. MATERIALS AND METHODS: We trained TURS on all ICU admissions from January 2011 to January 2020 at Leiden University Medical Center. For each subgroup, patient characteristics and the influence of feature variables on readmission risk were analyzed. RESULTS: TURS identified subgroups with heterogeneous feature distributions and feature importance, providing actionable insights for ICU discharge planning. Its predictive performance (area under the receiver operating characteristic curve [ROC-AUC] 70.5%) and model complexity (5 rules, average length 2) surpassed other rule-based models. DISCUSSION: Subgroup analysis highlighted the heterogeneity of patients. First, we compared the conditional probability distribution of each feature variable, conditioned on the fact that a patient was in a certain subgroup, with its unconditional distribution. We identified features deviating from the unconditional distribution, illustrating unique subgroup-specific implications. Furthermore, we demonstrated that features with the highest impact on the readmission risk were distinctive for each subgroup. CONCLUSION: The TURS model provided a concise summary of patient subgroups, aiding ICU discharge decisions and advancing knowledge discovery in the ICU.
Health interoperability across phenotypes of family physician practices.
Everson J, Strawley C
J Am Med Inform Assoc · 2026 Feb · PMID 41144321 · Full text
OBJECTIVES: To inform initiatives to improve the interoperability of healthcare data, we described the experience of distinct phenotypes of physicians when obtaining information from outside sources. MATERIALS AND METHOD...
OBJECTIVES: To inform initiatives to improve the interoperability of healthcare data, we described the experience of distinct phenotypes of physicians when obtaining information from outside sources. MATERIALS AND METHODS: A total of 6175 family physicians across the United States responded to information technology questions on the 2022 and 2023 American Board of Family Medicine Continuous Certification Questionnaire (100% response rate). Latent class analysis grouped physicians by individual and practice characteristics and then compared reported experience with interoperability. RESULTS: A 4-class model ("Safety Net," "Health System," "Independent Practice," and "Large Practice") best fit. Health system and large practice physicians (predominately Epic users) were more likely to report information was integrated in their Electronic Health Record (EHR) than independent practice physicians (38% and 40%, respectively, compared to 24%), and to report that information from organizations using the same EHR was usable (52% and 51%, respectively, compared to 25%). Safety net physicians were least likely to report that information from outside organizations was usable (17% compared to 23% of independent physicians). Between 42% and 50% of each phenotype reported commonly encountering external records with a large volume of low-value information. DISCUSSION: Independent practice and safety net physicians reported worse experience in some dimensions of interoperability, likely driven by differences in access to information from organizations using the same EHR. Many other challenges were consistent across physician phenotypes. CONCLUSION: Initiatives to improve interoperability among physicians may be most effective if targeted at independent practices and safety net practices; however, broad improvements will be necessary to address similar challenges across phenotypes.
A novel analysis methodology for assessment of re-identification risks for the National Cancer Institute cancer registry privacy preserving record linkage technique.
Kantarcioglu M, Howe W, Liu B … +5 more , Petkov V, Casas-Silva E, Velasquez-Kolnik D, Malin BA, Penberthy L
J Am Med Inform Assoc · 2026 Mar · PMID 41144319 · Full text
OBJECTIVE: The National Cancer Institute (NCI), part of the National Institutes of Health (NIH) supports efforts to address critical challenges in advancing cancer research. As part of this effort, NCI sponsored the deve...
OBJECTIVE: The National Cancer Institute (NCI), part of the National Institutes of Health (NIH) supports efforts to address critical challenges in advancing cancer research. As part of this effort, NCI sponsored the development of a privacy-preserving record linkage (PPRL) software that transforms identifying patient information into multiple tokens through a set of cryptographically secure keyed hash functions. This project aims to evaluate the PPRL software in the perspective of re-identification risks and propose effective strategies to sufficiently mitigate these risks. MATERIALS AND METHODS: To achieve the goals, we developed a novel re-identification risk assessment framework, based on token frequency analysis, to estimate the privacy impact of hashed tokens shared for record linkage. We assessed privacy risk through empirical analysis on a state-level voter registration database, a public dataset commonly used for re-identification, under various scenarios. These scenarios are defined based on several factors, including the size of the dataset used for linkage and a group size parameter that determines when an adversary can claim that a record has been re-identified. RESULTS: We found that the re-identification risk based on frequency analysis attack is approximately 0.0002 (ie, 2 patients out of 10 000 are potentially identifiable) under reasonable adversarial settings, with a group size parameter of k = 12 and a dataset size of 400 000 patients. Additionally, our analysis reveals a negative correlation between dataset size and re-identification risk. DISCUSSION: Re-identification risk is deemed low for the new NCI PPRL software. Token frequency analysis provides a reliable estimate of the re-identification risk in token-based PPRL tools.
Observer: creation of a novel multimodal dataset for outpatient care research.
Johnson KB, Alasaly B, Jang KJ … +3 more , Eaton E, Mopidevi S, Koppel R
J Am Med Inform Assoc · 2026 Feb · PMID 41144304 · Full text
OBJECTIVE: To support ambulatory care innovation, we created Observer, a multimodal dataset comprising videotaped outpatient visits, electronic health record (EHR) data, and structured surveys. This paper describes the d...
OBJECTIVE: To support ambulatory care innovation, we created Observer, a multimodal dataset comprising videotaped outpatient visits, electronic health record (EHR) data, and structured surveys. This paper describes the data collection procedures and summarizes the clinical and contextual features of the dataset. MATERIALS AND METHODS: A multistakeholder steering group shaped recruitment strategies, survey design, and privacy-preserving design. Consented patients and primary care providers (PCPs) were recorded using room-view and egocentric cameras. EHR data, metadata, and audit logs were also captured. A custom de-identification pipeline, combining transcript redaction, voice masking, and facial blurring, ensured video and EHR HIPAA compliance. RESULTS: We report on the first 100 visits in this continually growing dataset. Thirteen PCPs from 4 clinics participated. Recording the first 100 visits required approaching 210 patients, from which 129 consented (61%), with 29 patients missing their scheduled encounter after consenting. Visit lengths ranged from 5 to 100 minutes, covering preventive care to chronic disease management. Survey responses revealed high satisfaction: 4.24/5 (patients) and 3.94/5 (PCPs). Visit experience was unaffected by the presence of video recording technology. DISCUSSION: We demonstrate the feasibility of capturing rich, real-world primary care interactions using scalable, privacy-sensitive methods. Room layout and camera placement were key influences on recorded communication and are now added to the dataset. The Observer dataset enables future clinical AI research/development, communication studies, and informatics education among public and private user groups. CONCLUSION: Observer is a new, shareable, real-world clinic encounter research and teaching resource with a representative sample of adult primary care data.
Transfer-learning on federated observational healthcare data for prediction models using Bayesian sparse logistic regression with informed priors.
Li KM, Reps JM, Nishimura A … +2 more , Schuemie MJ, Suchard MA
J Am Med Inform Assoc · 2026 Feb · PMID 41144298 · Full text
OBJECTIVE: To develop a transfer-learning Bayesian sparse logistic regression model that transfers information learned from one dataset to another by using an informed prior to facilitate model fitting in small-sample cl...
OBJECTIVE: To develop a transfer-learning Bayesian sparse logistic regression model that transfers information learned from one dataset to another by using an informed prior to facilitate model fitting in small-sample clinical patient-level prediction problems that suffer from a lack of available information. METHODS: We propose a Bayesian framework for prediction using logistic regression that aims to conduct transfer-learning on regression coefficient information from a larger dataset model (order 105-106 patients by 105 features) into a small-sample model (order 103 patients). Our approach imposes an informed, hierarchical prior on each regression coefficient defined as a discrete mixture of the Bayesian Bridge shrinkage prior and an informed normal distribution. Performance of the informed model is compared against traditional methods, primarily measured by area under the curve, calibration, bias, and sparsity using both simulations and a real-world problem. RESULTS: Across all experiments, transfer-learning outperformed the traditional L1-regularized model across discrimination, calibration, bias, and sparsity. In fact, even using only a continuous shrinkage prior without the informed prior increased model performance when compared to L1-regularization. CONCLUSION: Transfer-learning using informed priors can help fine-tune prediction models in small datasets suffering from a lack of information. One large benefit is in that the prior is not dependent on patient-level information, such that we can conduct transfer-learning without violating privacy. In future work, the model can be applied for learning between disparate databases, or similar lack-of-information cases such as rare outcome prediction.
Biases in an artificial intelligence image-generator's depictions of healthy aging and Alzheimer's.
Osinga C, Jintaganon N, Steijger D … +2 more , De Vugt M, Neal D
J Am Med Inform Assoc · 2026 Feb · PMID 41144296 · Full text
OBJECTIVE: This content analysis study investigates potential biases in image generation by 2 artificial intelligence (AI) tools, DALL-E 3 and Midjourney, in portraying older adults and individuals living with dementia....
OBJECTIVE: This content analysis study investigates potential biases in image generation by 2 artificial intelligence (AI) tools, DALL-E 3 and Midjourney, in portraying older adults and individuals living with dementia. Despite widespread use of generative AI in various sectors, there is limited research on how these models might perpetuate stereotypes and stigmatization through their images. MATERIALS AND METHODS: 1056 images were generated using specified prompts categorized into 3 groups: general older adults, dementia-related, and control. Each prompt began with "photorealistic portrait" followed by specific scene descriptions. Four researchers conducted content analysis on each generated image, focusing on factors, such as portrait style, setting, posture, apparent sex of subjects, and emotional affect. The analysis was executed with blinding and randomization protocols to ensure unbiased assessment. Chi-square tests examined the relationship between prompt categories and variables. RESULTS: Results revealed significant disparities in depictions of older adults and those with dementia compared with control images. Both models more often portrayed subjects in response to dementia-related prompts with negative affect, in less favorable emotional states. However, DALL-E 3 also generated more personas displaying positive affect in response to these prompts. Variations in depiction styles between the 2 AI models were noted, with DALL-E 3 showing a broader diversity of outputs. DISCUSSION AND CONCLUSIONS: The findings highlight AI's potential to reinforce stigmatizing stereotypes through biased image generation. Recommendations include selecting prompts carefully to avoid negative depictions and advocating for greater AI explainability and inclusivity by design. Future research should explore other AI models, other forms of bias, and strategies to mitigate biases.
Compliance and factuality of large language models for clinical research document generation.
Wang Z, Gao J, Danek B … +5 more , Theodorou B, Shaik R, Thati S, Won S, Sun J
J Am Med Inform Assoc · 2026 Mar · PMID 41144289 · Full text
OBJECTIVES: Large language models' (LLMs') performance in high-stakes, compliance-driven settings such as drafting clinical research documents remains underexplored. This study aims to build a benchmark and an evaluation...
OBJECTIVES: Large language models' (LLMs') performance in high-stakes, compliance-driven settings such as drafting clinical research documents remains underexplored. This study aims to build a benchmark and an evaluation framework for assessing LLMs' compliance and factuality in generating informed consent forms (ICFs) from clinical trial protocols. MATERIALS AND METHODS: We introduce InformBench, a benchmark comprising 900 clinical trial documents, and propose an evaluation framework grounded in regulatory guidelines and site-specific consent templates. We assess LLM performance on transforming trial protocols, often hundreds of pages, into concise, patient-facing ICFs. Additionally, we design InformGen, a retrieval-augmented, human-in-the-loop pipeline aimed at improving generation quality. RESULTS: Baseline LLMs such as GPT-4o achieved only 70%-80% compliance and exhibited factual errors in 18%-43% of cases. In contrast, InformGen substantially improved outputs, achieving nearly 100% regulatory compliance and over 90% factual accuracy, as validated by 5 domain-expert annotators. DISCUSSION: The study reveals critical limitations in current LLMs for clinical research document drafting, particularly in regulatory sensitivity and factual grounding. Our results highlight the need for domain-specific benchmarks and structured evaluations to support safe deployment in real-world clinical research workflows. CONCLUSION: LLMs offer value in clinical research document generation but must be adapted and rigorously evaluated for high-stakes applications. Our benchmark and framework provide a foundation for improving and assessing LLM-generated outputs in compliance-critical domains.
An exploratory analysis of SNOMED CT national editions.
Lee DH, Lau FY
J Am Med Inform Assoc · 2026 Feb · PMID 41133785 · Full text
OBJECTIVES: The objectives of this study were to: (1) explore and analyze the structure and content of 19 SNOMED Clinical Terms (SNOMED CT) national editions; (2) identify overlaps and types of quality issues; and (3) of...
OBJECTIVES: The objectives of this study were to: (1) explore and analyze the structure and content of 19 SNOMED Clinical Terms (SNOMED CT) national editions; (2) identify overlaps and types of quality issues; and (3) offer recommendations to improve the quality of these national editions. MATERIALS AND METHODS: We used expression constraint language and structured query language to analyze and compare the national editions hosted on the SNOMED International SNOMED CT Browser and the International and Canadian Edition Release Format Two files. RESULTS: The 19 national editions authored over 275 000 concepts and over 2.1 million descriptions. Modules were used to organize drug extensions, language translations, patient-friendly descriptions, maps, and subsets. The national editions included 27 languages and dialects reference sets, maps to international, national, and local terminologies, and over 1100 subsets. Since 2012, over 28 000 extensions have been promoted to the International Edition. Overlaps were also identified between national editions. DISCUSSION: Challenges of extensions included inconsistent modeling of concepts and quality issues, versioning and maintenance, and risks to semantic interoperability and data analysis. We suggest improved functionality in authoring tools to identify overlapping content across national editions and the incorporation of auditing methods to ensure high-quality extensions, increased collaboration between countries, and the accelerated harmonization of extensions into the International Edition. CONCLUSION: Nineteen countries have developed over 2.4 million extension concepts and descriptions, and it is important to harmonize the national editions through ongoing collaboration to maintain the integrity and consistency of SNOMED CT as a global reference terminology.
A multifaceted approach to advancing data quality and fitness standards in multi-institutional networks.
Razzaghi H, Dickinson K, Wieand K … +9 more , Boss S, Weidlich H, Huang Y, Morse K, Mutyala SK, Nandagopal JPA, Viswanathan K, Forrest CB, Bailey LC
J Am Med Inform Assoc · 2026 Feb · PMID 41128352 · Full text
OBJECTIVE: To construct a data quality (DQ) system that incorporates combinations of methods to evaluate data characteristics and analytic fitness across research questions for multiple uses. MATERIALS AND METHODS: Drawi...
OBJECTIVE: To construct a data quality (DQ) system that incorporates combinations of methods to evaluate data characteristics and analytic fitness across research questions for multiple uses. MATERIALS AND METHODS: Drawing from experience of other data quality programs, network data extraction needs, and recurring study requirements, we developed 5 standards to guide development of a modular, multifaceted data quality system. These included annotation and documentation, ability to measure research readiness, reproducibility across networks, flexibility for the user, and interpretability to research and project teams. Implementation of checks based on these principles focused on reusability and interactive visualization of results. RESULTS: We identified 10 check types producing over 444 check applications and deployed them in 2 multi-institutional networks. Check types span structural conformance to a data model, utility for common research needs, and study-specific customization. All check types are customizable without dependencies between them. A dashboard visualizes results, permitting adjustments based on number of data sources, need for source masking, and the user's focus. All components can be applied as written to any data source using OMOP and are readily modified for other data models. DISCUSSION: We have extended previous work through our novel and multifaceted approach to data quality assessment, addressing needs in both network data improvement and research usage. We developed a capable and deployable system rather than tailoring to specific use cases. CONCLUSION: Our novel DQ assessment system provides essential components for future standardization and collaboration to improve fitness of clinical data for intended use.
AcuKG: a comprehensive knowledge graph for medical acupuncture.
Li Y, Peng X, Peng S … +11 more , Li J, Pei D, Zhang Q, Lu Y, Hu Y, Li F, Zhou L, He Y, Tao C, Xu H, Hong N
J Am Med Inform Assoc · 2026 Feb · PMID 41124298 · Full text
BACKGROUND: Acupuncture, a key modality in traditional Chinese medicine, is gaining global recognition as a complementary therapy and a subject of increasing scientific interest. However, fragmented and unstructured acup...
BACKGROUND: Acupuncture, a key modality in traditional Chinese medicine, is gaining global recognition as a complementary therapy and a subject of increasing scientific interest. However, fragmented and unstructured acupuncture knowledge spread across diverse sources poses challenges for semantic retrieval, reasoning, and in-depth analysis. To address this gap, we developed AcuKG, a comprehensive knowledge graph that systematically organizes acupuncture-related knowledge to support sharing, discovery, and artificial intelligence-driven innovation in the field. METHODS: AcuKG integrates data from multiple sources, including online resources, guidelines, PubMed literature, ClinicalTrials.gov, and multiple ontologies (SNOMED CT, UBERON, and MeSH). We employed entity recognition, relation extraction, and ontology mapping to establish AcuKG, with human-in-the-loop to ensure data quality. Two cases evaluated AcuKG's usability: (1) how AcuKG advances acupuncture research for obesity and (2) how AcuKG enhances large language model (LLM) application on acupuncture question-answering. RESULTS: AcuKG comprises 1839 entities and 11 527 relations, mapped to 1836 standard concepts in 3 ontologies. Two use cases demonstrated AcuKG's effectiveness and potential in advancing acupuncture research and supporting LLM applications. In the obesity use case, AcuKG identified highly relevant acupoints (eg, ST25, ST36) and uncovered novel research insights based on evidence from clinical trials and literature. When applied to LLMs in answering acupuncture-related questions, integrating AcuKG with GPT-4o and LLaMA 3 significantly improved accuracy (GPT-4o: 46% → 54%, P = .03; LLaMA 3: 17% → 28%, P = .01). CONCLUSION: AcuKG is an open dataset that provides a structured and computational framework for acupuncture applications, bridging traditional practices with acupuncture research and cutting-edge LLM technologies.
Re-identification risk for common privacy preserving patient matching strategies when shared with de-identified demographics.
Eliazar A, Brown JT, Cinamon S … +2 more , Kantarcioglu M, Malin B
J Am Med Inform Assoc · 2026 Feb · PMID 41105504 · Full text
OBJECTIVE: Privacy preserving record linkage (PPRL) refers to techniques used to identify which records refer to the same person across disparate datasets while safeguarding their identities. PPRL is increasingly relied...
OBJECTIVE: Privacy preserving record linkage (PPRL) refers to techniques used to identify which records refer to the same person across disparate datasets while safeguarding their identities. PPRL is increasingly relied upon to facilitate biomedical research. A common strategy encodes personally identifying information for comparison without disclosing underlying identifiers. As the scale of research datasets expands, it becomes crucial to reassess the privacy risks associated with these encodings. This paper highlights the potential re-identification risks of some of these encodings, demonstrating an attack that exploits encoding repetition across patients. MATERIALS AND METHODS: The attack leverages repeated PPRL encoding values combined with common demographics shared during PPRL in the clear (e.g., 3-digit ZIP code) to distinguish encodings from one another and ultimately link them to identities in a reference dataset. Using US Census statistics and voter registries, we empirically estimate encodings' re-identification risk against such an attack, while varying multiple factors that influence the risk. RESULTS: Re-identification risk for PPRL encodings increases with population size, number of distinct encodings per patient, and amount of demographic information available. Commonly used encodings typically grow from <1% re-identification rate for datasets under one million individuals to 10%-20% for 250 million individuals. DISCUSSION AND CONCLUSION: Re-identification risk often remains low in smaller populations, but increases significantly at the larger scales increasingly encountered today. These risks are common in many PPRL implementations, although, as our work shows, they are avoidable. Choosing better tokens or matching tokens through a third party without the underlying demographics effectively eliminates these risks.
About
- Frequency
- Sun
- Papers found
- 200
- RSS feed
- Subscribe