AMIA ... Annual Symposium Proceedings / AMIA Symposium. AMIA Symposium[JOURNAL]
Sun
200 papers
AKI-Detector: A Multi-Agent Framework by Integrating Machine Learning and Large Language Models for Early Prediction of Acute Kidney Injury in ICU.
Shi T, Xiao M, Xu H … +2 more , Zhao H, Kong G
AMIA Annu Symp Proc · 2024 · PMID 41726403
Acute kidney injury (AKI) is a severe condition in the ICU, where early prediction is crucial for timely intervention and prevention. Traditional machine learning (ML) models lack interpretability, which limits real-worl...
Acute kidney injury (AKI) is a severe condition in the ICU, where early prediction is crucial for timely intervention and prevention. Traditional machine learning (ML) models lack interpretability, which limits real-world applicability. We propose AKI-Detector, a novel multi-agent framework that integrates structured electronic health records (EHR)-based ML models, large language models (LLMs), and retrieval-augmented generation (RAG) to enhance clinical reasoning, accuracy, and interpretability of AKI prediction. The proposed AKI-Detector mitigates LLM hallucinations by integrating ML models and bridges the gap between algorithmic output and clinically interpretable reports. Evaluated on ICU data from MIMIC-IV, AKI-Detector outperformed ML models such as CatBoost and GRU, and achieved an accuracy of 0.827, precision of 0.672, recall of 0.542, and F1-score of 0.600 on the test cohort, demonstrating balanced and reliable predictive performance. This work highlights the promise of real-world big data and LLM-powered multi-agent systems to support trustworthy and explainable AI for clinical prediction.
Technology and Human Support Systems in Decentralized Studies: A Participant-Centered Case Study in Cystic Fibrosis.
Sarrieddine A, Lai C, Walk OBD … +5 more , Reid NFH, Sawicki G, Berlinski A, Rosenfeld M, Hartzler AL
AMIA Annu Symp Proc · 2024 · PMID 41726402
As the integration of informatics into clinical research reshapes the landscape of decentralized studies, optimizing participant experience remains a key challenge. Although prior research has established foundations for...
As the integration of informatics into clinical research reshapes the landscape of decentralized studies, optimizing participant experience remains a key challenge. Although prior research has established foundations for decentralized study design, a more comprehensive understanding of participant perspectives is essential to ensure remote methods for data collection meet participant needs. This study contributes to a growing literature in participant-centered decentralized studies through an analysis of OUTREACH, a 3-month home spirometry study among individuals with cystic fibrosis. Through a qualitative analysis of 46 participant exit interviews, we identified three overarching categories that influenced participant experience: motivators, technological infrastructure, and human coordination. Our findings emphasize the value of reliable technology and comprehensive interpersonal support systems. These findings shed light upon the importance of sociotechnical elements for optimizing participant experience, which may enhance the quality of clinical study data through meaningful participant engagement.
Identifying TRIM11 Upregulators as Therapeutic Targets in Alzheimer's Disease: A Bayesian Network Approach.
Pham A, Garai S, Zhai T … +2 more , Kan M, Shen L
AMIA Annu Symp Proc · 2024 · PMID 41726401
A recent study shows that TRIM11 is downregulated in Alzheimer's Disease (AD) but has been demonstrated to improve cognitive function when overexpressed in mouse AD models. Based on this discovery, our study aims to iden...
A recent study shows that TRIM11 is downregulated in Alzheimer's Disease (AD) but has been demonstrated to improve cognitive function when overexpressed in mouse AD models. Based on this discovery, our study aims to identify potential genetic regulators of TRIM11 using single-cell and single-nucleus RNA sequencing, and graph learning methods. In this study we explore two publicly available datasets: GSE173731 and GSE227222. To identify the potential regulators of TRIM11, we use a probabilistic approach and Bayesian networks. Our approach identified a set of candidate genes in both datasets that may exert regulatory influence on TRIM11, offering potential targets for further research in future therapeutic strategies in AD.
Survivorship Navigator: Personalized Survivorship Care Plan Generation using Large Language Models.
Pradeepkumar J, Kumar SP, Reamer CB … +4 more , Dreyer M, Patel J, Liebovitz D, Sun J
AMIA Annu Symp Proc · 2024 · PMID 41726400
Cancer survivorship care plans (SCPs) are critical tools for guiding long-term follow-up care of cancer survivors. Yet, their widespread adoption remains hindered by the significant clinician burden and the time- and lab...
Cancer survivorship care plans (SCPs) are critical tools for guiding long-term follow-up care of cancer survivors. Yet, their widespread adoption remains hindered by the significant clinician burden and the time- and labor-intensive process of SCP creation. Current practices require clinicians to extract and synthesize treatment summaries from complex patient data, apply relevant survivorship guidelines, and generate a care plan with personalized recommendations, making SCP generation time-consuming. In this study, we systematically explore the potential of large language models (LLMs) for automating SCP generation and introduce Survivorship Navigator, a framework designed to streamline SCP creation and enhance integration with clinical systems. We evaluate our approach through automated assessments and a human expert study, demonstrating that Survivorship Navigator outperforms baseline methods, producing SCPs that are more accurate, guideline-compliant, and actionable.
Cross Biobank Comparison of Phenomic Profiles.
Newbury A, Jiang X, Natarajan K … +1 more , Gürsoy G
AMIA Annu Symp Proc · 2024 · PMID 41726399
The (AoU) Research Program and UK Biobank (UKBB) boast a wealth of EHR data, which can be harnessed to refine cohort selection via rule-based phenotyping algorithms. The Observational Health Data Sciences and Informatic...
The (AoU) Research Program and UK Biobank (UKBB) boast a wealth of EHR data, which can be harnessed to refine cohort selection via rule-based phenotyping algorithms. The Observational Health Data Sciences and Informatics (OHDSI) Phenotype Library (PL) hosts many complex phenotyping rules. Here, we compare prevalence for 423 OHDSI PL cohorts in AoU and UKBB. For three select diseases (T2D, COPD, Acute MI), we analyze differences in demographics, social determinants of health (SDOH), geographic prevalence, and genome-wide association study (GWAS) results. We found that AoU has a significantly higher prevalence for 80% of phenotypes compared to UKBB. We also found that for the select diseases, SDOH variables between the two biobanks differ significantly. Findings for each of these three diseases confirm known regions of high risk. Additionally, GWAS in UKBB discovered more genes associated with each of the three diseases than GWAS in AoU.
Penalized Regression Based Proteome-Wide Association Study Reveals Potential Population-Specific Drug Targets for Alzheimer's Disease in European and African American Cohorts.
Shi S, Liu S, Liu Y … +1 more , Lu Q
AMIA Annu Symp Proc · 2024 · PMID 41726398
Alzheimer's disease (AD) is a complex neurodegenerative disorder with significant genetic underpinnings, yet effective treatments remain elusive. To bridge the gap between genetic discoveries and therapeutic development,...
Alzheimer's disease (AD) is a complex neurodegenerative disorder with significant genetic underpinnings, yet effective treatments remain elusive. To bridge the gap between genetic discoveries and therapeutic development, we conducted a penalized regression based proteome-wide association study (PWAS) in both European and African American populations. Using publicly available GWAS summary statistics and the BLISS model, we identified 37 protein-coding genes significantly associated with AD risk, including APOE and BCAM in both populations. We further applied the GREP model to prioritize repositionable drugs targeting these genes, identifying 30 significant disease-target-drug pairs. Notably, Ramipril and BAY 85-8501 emerged as top candidates for AD treatment in European and African American populations, respectively. These findings highlight ancestry-specific drug targets, demonstrating the importance of diverse genetic studies in AD research and providing novel avenues for therapeutic intervention.
A Multi-Phase Analysis of Blood Culture Stewardship: Machine Learning Prediction, Expert Recommendation Assessment, and LLM Automation.
Amrollahi F, Marshall N, Haredasht FN … +10 more , Black KC, Zahedivash A, Maddali MV, Ma SP, Chang A, Deresinski SC, Goldstein MK, Asch SM, Banaei N, Chen JH
AMIA Annu Symp Proc · 2024 · PMID 41726397
Blood cultures are often overordered without clear justification, straining healthcare resources and contributing to inappropriate antibiotic use-pressures worsened by the global shortage. In the study of 135,483 emergen...
Blood cultures are often overordered without clear justification, straining healthcare resources and contributing to inappropriate antibiotic use-pressures worsened by the global shortage. In the study of 135,483 emergency department (ED) blood culture orders, we developed machine learning (ML) models to predict the risk of bacteremia using structured electronic health record (EHR) data and provider notes via a large language model (LLM). The structured model's AUC improved from 0.76 to 0.79 with note embeddings and reached 0.81 with added diagnosis codes. Compared to an expert recommendation framework applied by human reviewers and an LLM-based pipeline, our ML approach offers higher specificity without compromising sensitivity. The recommendation framework achieved sensitivity 86%, specificity 57%, while the LLM maintained high sensitivity (96%) but overclassified negatives, reducing specificity (16%). These findings demonstrate that ML models integrating structured and unstructured data can outperform consensus recommendations, enhancing diagnostic stewardship beyond existing standards of care.
Machine Learning-Based Prediction of Antimicrobial Susceptibility: A Step Towards Precision Antimicrobial Stewardship.
Amrollahi F, Haredasht FN, Vansomphone A … +12 more , Marshall N, Maddali MV, Ma SP, Chang A, Deresinski SC, Goldstein MK, Kanjilal S, Medford RJ, Cooper LN, Asch SM, Banaei N, Chen JH
AMIA Annu Symp Proc · 2024 · PMID 41726396
Antimicrobial resistance (AMR) represents an urgent global health crisis exacerbated by the frequent empirical use of broad-spectrum antibiotics. AMR is exacerbated by inherent delays in obtaining culture results and ant...
Antimicrobial resistance (AMR) represents an urgent global health crisis exacerbated by the frequent empirical use of broad-spectrum antibiotics. AMR is exacerbated by inherent delays in obtaining culture results and antimicrobial susceptibility data after sample collection. In this study, we developed and validated Machine Learning (ML) models using routinely collected EHR data from inpatient and outpatient encounters to predict antibiotic resistance at the time of blood, urine or respiratory bacterial culture collection. The models demonstrated robust predictive accuracy, particularly in inpatient settings where clinical data was more consistently available. Notably, the model independently identified patterns that predict resistance, similar to how a clinician would attempt to predict resistance using prior culture and susceptibility data combined with their clinical training and knowledge of microbiological resistance patterns. Integrating these predictive tools into clinical workflows could significantly enhance empirical antibiotic selection, reduce unnecessary broad-spectrum antibiotic use, and meaningfully advance antimicrobial stewardship efforts.
Beyond Random Splitting: Evaluating the Impact of Data Partitioning Strategies on Ventilator-Associated Pneumonia Prediction Using EHRs.
Asare-Baiden M, Zhang W, Stover Hertzberg V … +1 more , C Ho J
AMIA Annu Symp Proc · 2024 · PMID 41726395
Ventilator-Associated Pneumonia (VAP) significantly impacts critical care outcomes, yet prediction models often over-look healthcare data's meronomic structure. Using MIMIC-III data, we developed a multi-source extractio...
Ventilator-Associated Pneumonia (VAP) significantly impacts critical care outcomes, yet prediction models often over-look healthcare data's meronomic structure. Using MIMIC-III data, we developed a multi-source extraction approach integrating structured data with clinical notes, identifying 679 VAP and 3,207 non-VAP cases. We compared four data splitting strategies: Ventilator Session-Based Split, Ventilator Session-Based Split on Single ICU Stays, Hospital Admission-Based Split, and Hospital Admission-Based Split on Single ICU Stays. Evaluating four machine learning models revealed that conventional random splitting yielded moderately high performance (AUROC: 76-81%) while restricting to single ICU stays surprisingly improved performance (AUROC: 86-87%). Admission-based approaches showed realistic results (AUROC: 72-76%). Feature analysis identified mechanical ventilation hours, systolic blood pressure, and urine counts as consistently important predictors. These findings demonstrate that robust VAP prediction requires evaluation frameworks respecting healthcare data's meronomic nature.
Failure Modes of Time Series Interpretability Algorithms for Critical Care Applications and Potential Solutions.
Yadav S, Subbian V
AMIA Annu Symp Proc · 2024 · PMID 41726394
Interpretability plays a vital role in aligning and deploying deep learning models in critical care, especially in constantly evolving conditions that influence patient survival. However, common interpretability algorith...
Interpretability plays a vital role in aligning and deploying deep learning models in critical care, especially in constantly evolving conditions that influence patient survival. However, common interpretability algorithms face unique challenges when applied to dynamic prediction tasks, where patient trajectories evolve over time. Gradient, Occlusion, and Permutation-based methods often struggle with time-varying target dependency and temporal smoothness. This work systematically analyzes these failure modes and supports learnable mask-based interpretability frameworks as alternatives, which can incorporate temporal continuity and label consistency constraints to learn feature importance over time. Here, we propose that learnable mask-based approaches for dynamic time-series prediction problems provide more reliable and consistent interpretations for applications in critical care and similar domains.
Using LLMs to Interpret Arterial Blood Gases: Comparison of a Novel Math Scratchpad with Different Prompting Methods in a Three-Arm Trial.
Meka P, Silvers CT, Gunapati B … +1 more , Liu Q
AMIA Annu Symp Proc · 2024 · PMID 41726393
---.
---.
Web-based Interventions for Substance Use Disorders and Mental Health: Preliminary findings from a Scoping Review.
Quintana Y, Joseph AL, Srivastava G
AMIA Annu Symp Proc · 2024 · PMID 41509880
This scoping review evaluated the efficacy and potential of web-based interventions for substance use disorders and mental health conditions. The studies comprise randomized controlled trials, pilot trials, and effective...
This scoping review evaluated the efficacy and potential of web-based interventions for substance use disorders and mental health conditions. The studies comprise randomized controlled trials, pilot trials, and effectiveness trials. Web-based interventions consistently demonstrated significant reductions in substance use, improvements in mental health outcomes (e.g., PTSD, depression, anxiety), and enhancements in emotion regulation, help-seeking, and quality of life. Several studies found web-based interventions to be non-inferior or superior to traditional face-to-face treatments. Despite limitations in the current evidence base, such as methodological issues and lack of long-term follow-up, the findings highlight the promise of web-based interventions in expanding access to evidence-based care, particularly for underserved populations. Future research should focus on refining interventions, exploring novel technologies, and evaluating long-term effectiveness and cost-effectiveness. The integration of web-based interventions into healthcare systems has the potential to significantly impact public health by increasing treatment accessibility and improving outcomes for individuals with substance use disorders and mental health conditions.
Toward Large Language Models as a Therapeutic Tool: Comparing Prompting Techniques to Improve GPT-Delivered Problem-Solving Therapy.
Filienko D, Wang Y, Jazmi CE … +4 more , Xie S, Cohen T, De Cock M, Yuwen W
AMIA Annu Symp Proc · 2024 · PMID 40417590
While Large Language Models (LLMs) are being quickly adapted to many domains, including healthcare, their strengths and pitfalls remain under-explored. In our study, we examine the effects of prompt engineering to guide...
While Large Language Models (LLMs) are being quickly adapted to many domains, including healthcare, their strengths and pitfalls remain under-explored. In our study, we examine the effects of prompt engineering to guide Large Language Models (LLMs) in delivering parts of a Problem-Solving Therapy (PST) session via text, particularly during the symptom identification and assessment phase for personalized goal setting. We present evaluation results of the models' performances by automatic metrics and experienced medical professionals. We demonstrate that the models' capability to deliver protocolized therapy can be improved with the proper use of prompt engineering methods, albeit with limitations. To our knowledge, this study is among the first to assess the effects of various prompting techniques in enhancing a generalist model's ability to deliver psychotherapy, focusing on overall quality, consistency, and empathy. Exploring LLMs' potential in delivering psychotherapy holds promise with the current shortage of mental health professionals amid significant needs, enhancing the potential utility of AI-based and AI-enhanced care services.
Secondary Use of Clinical Problem List Descriptions for Bi-Encoder Based ICD-10 Classification.
Kreuzthaler M, Pfeifer B, Schulz S
AMIA Annu Symp Proc · 2024 · PMID 40417589
Annotated language resources are essential for supervised machine learning methods. In the clinical domain, such data sets can boost use-case specific natural language processing services. In this work, we have analyzed...
Annotated language resources are essential for supervised machine learning methods. In the clinical domain, such data sets can boost use-case specific natural language processing services. In this work, we have analyzed a clinical problem list table consisting of millions of ICD-10 codes assigned to short problem list descriptions in German. We have investigated whether the given data forms a valuable resource within a secondary use case scenario for coding support. Our proposed methodology exploits an embedding-based k-NN classifier, which was evaluated based on its coding performance, leveraging the multilingual BERT based language model SapBERT-UMLS in comparison with medBERT.de, which is specifically tailored to medical and clinical language resources in German. Our approach reached a weighted F1-measure of 0.87 using SapBERT-UMLS and an F1-measure of 0.86 for medBERT.de. The approach revealed promising coding results when reusing annotated language resources out of clinical routine documentation.
Large Language Models Struggle in Token-Level Clinical Named Entity Recognition.
Lu Q, Li R, Wen A … +3 more , Wang J, Wang L, Liu H
AMIA Annu Symp Proc · 2024 · PMID 40417588
Large Language Models (LLMs) have revolutionized various sectors, including healthcare where they are employed in diverse applications. Their utility is particularly significant in the context of rare diseases, where dat...
Large Language Models (LLMs) have revolutionized various sectors, including healthcare where they are employed in diverse applications. Their utility is particularly significant in the context of rare diseases, where data scarcity, complexity, and specificity pose considerable challenges. In the clinical domain, Named Entity Recognition (NER) stands out as an essential task and it plays a crucial role in extracting relevant information from clinical texts. Despite the promise of LLMs, current research mostly concentrates on document-level NER, identifying entities in a more general context across entire documents, without extracting their precise location. Additionally, efforts have been directed towards adapting ChatGPTfor token-level NER. However, there is a significant research gap when it comes to employing token-level NER for clinical texts, especially with the use of local open-source LLMs. This study aims to bridge this gap by investigating the effectiveness of both proprietary and local LLMs in token-level clinical NER. Essentially, we delve into the capabilities of these models through a series of experiments involving zero-shot prompting, few-shot prompting, retrieval-augmented generation (RAG), and instruction-fine-tuning. Our exploration reveals the inherent challenges LLMs face in token-level NER, particularly in the context of rare diseases, and suggests possible improvements for their application in healthcare. This research contributes to narrowing a significant gap in healthcare informatics and offers insights that could lead to a more refined application of LLMs in the healthcare sector.
Pruning the Path to Optimal Care: Identifying Systematically Suboptimal Medical Decision-Making with Inverse Reinforcement Learning.
Bovenzi I, Carmel A, Hu M … +5 more , Hurwitz R, McBride F, Benac L, Ayala JRT, Doshi-Velez F
AMIA Annu Symp Proc · 2024 · PMID 40417587
In aims to uncover insights into medical decision-making embedded within observational data from clinical settings, we present a novel application of Inverse Reinforcement Learning (IRL) that identifies suboptimal clinic...
In aims to uncover insights into medical decision-making embedded within observational data from clinical settings, we present a novel application of Inverse Reinforcement Learning (IRL) that identifies suboptimal clinician actions based on the actions of their peers. This approach centers two stages of IRL with an intermediate step to prune trajectories displaying behavior that deviates significantly from the consensus. This enables us to effectively identify clinical priorities and values from ICU data containing both optimal and suboptimal clinician decisions. We observe that the benefits of removing suboptimal actions vary by disease and differentially impact certain demographic groups.
Modeling Precision Feedback Knowledge for Healthcare Professional Learning and Quality Improvement.
Landis-Lewis Z, Cao Y, Chung H … +9 more , Boisvert P, Renji AD, Galante P, Jagadeesan A, Seifi F, Janda A, Shah N, Krumm A, Flynn A
AMIA Annu Symp Proc · 2024 · PMID 40417586
Healthcare providers learn continuously, but better support for provider learning is needed as new biomedical knowledge is produced at an increasing rate alongside widespread use of EHR data for clinical performance meas...
Healthcare providers learn continuously, but better support for provider learning is needed as new biomedical knowledge is produced at an increasing rate alongside widespread use of EHR data for clinical performance measurement. Precision feedback is an approach to improve support for provider learning by prioritizing coaching and appreciation messages based on each message's motivational potential for a specific recipient. We developed a Precision Feedback Knowledge Base as an open resource to support precision feedback systems, containing knowledge models that hold potential as key infrastructure for learning health systems. We describe the design and development of the Precision Feedback Knowledge Base, as well as its key components, including quality measures, feedback message templates, causal pathway models, signal detectors, and prioritization algorithms. Presently, the knowledge base is implemented in a national-scale quality improvement consortium for anesthesia care, to enhance provider feedback email messages.
Development of a Human Evaluation Framework and Correlation with Automated Metrics for Natural Language Generation of Medical Diagnoses.
Croxford E, Gao Y, Patterson B … +6 more , To D, Tesch S, Dligach D, Mayampurath A, Churpek MM, Afshar M
AMIA Annu Symp Proc · 2024 · PMID 40417585
In the evolving landscape of clinical Natural Language Generation (NLG), assessing abstractive text quality remains challenging, as existing methods often overlook generative task complexities. This work aimed to examine...
In the evolving landscape of clinical Natural Language Generation (NLG), assessing abstractive text quality remains challenging, as existing methods often overlook generative task complexities. This work aimed to examine the current state of automated evaluation metrics in NLG in healthcare. To have a robust and well-validated baseline with which to examine the alignment of these metrics, we created a comprehensive human evaluation framework. Employing ChatGPT-3.5-turbo generative output, we correlated human judgments with each metric. None of the metrics demonstrated high alignment; however, the SapBERT score-a Unified Medical Language System (UMLS)- showed the best results. This underscores the importance of incorporating domain-specific knowledge into evaluation efforts. Our work reveals the deficiency in quality evaluations for generated text and introduces our comprehensive human evaluation framework as a baseline. Future efforts should prioritize integrating medical knowledge databases to enhance the alignment of automated metrics, particularly focusing on refining the SapBERT score for improved assessments.
Enhancing Antibiotic Stewardship: A Machine Learning Approach to Predicting Antibiotic Resistance in Inpatient Care.
Haredasht FN, Maddali MV, Ma SP … +7 more , Chang A, Kim GYE, Banaei N, Deresinski S, Goldstein MK, Asch SM, Chen JH
AMIA Annu Symp Proc · 2024 · PMID 40417584
Antibiotics have been crucial in advancing medical treatments, but the growing threat of antibiotic resistance challenges these achievements and emphasizes the need for innovative stewardship strategies. In this study, w...
Antibiotics have been crucial in advancing medical treatments, but the growing threat of antibiotic resistance challenges these achievements and emphasizes the need for innovative stewardship strategies. In this study, we developed machine learning models, 'personalized antibiograms', to predict antibiotic resistance across five key antibiotics using Stanford's electronic health record data of 49,872 urine, blood, and respiratory infections. We aimed to ascertain the efficacy of these models in predicting antibiotic susceptibility and identify the clinical factors most indicative of resistance. Employing LightGBM, we incorporated demographics, prior resistance, prescriptions, and comorbidities as features. The models demonstrated notable discriminative ability, with AUROCs between 0.74 and 0.78, and highlighted prior resistance and prescriptions as significant predictive factors. The high specificity demonstrates machine learning models' potential to inform antibiotic de-escalation, aiding stewardship without risking safety. By leveraging machine learning with relevant clinical features, we show that it is feasible to improve empirical antibiotic prescribing.
Identifying acute kidney injury subtypes based on serum electrolyte data in ICU via -medoids clustering.
Liu W, Shi T, Xu H … +3 more , Zhao H, Hao J, Kong G
AMIA Annu Symp Proc · 2024 · PMID 40417583
This study proposes to use the K-medoids clustering method to identify subtypes of Intensive Care Unit (ICU)-acquired acute kidney injury (AKI) patients based on serum electrolyte data. Three distinct AKI subtypes with d...
This study proposes to use the K-medoids clustering method to identify subtypes of Intensive Care Unit (ICU)-acquired acute kidney injury (AKI) patients based on serum electrolyte data. Three distinct AKI subtypes with different serum electrolyte characteristics were identified by clustering analysis. Further, descriptive analysis was employed to characterize in-hospital mortality and renal replacement therapy, diuretic and vasopressor usage in the three subtypes, and Chi-square tests were conducted to check the differences of prognosis and treatments among the identified subtypes. This study enables the subclassification of AKI patients in the ICU, facilitating ICU physicians to make timely clinical decisions about AKI, and ultimately may contribute to patient outcome improvement.
About
- Frequency
- Sun
- Papers found
- 200
- RSS feed
- Subscribe