AMIA ... Annual Symposium Proceedings / AMIA Symposium. AMIA Symposium[JOURNAL]
Sun
200 papers
Time-series Machine Learning Models to Support Emergency Department Operational Planning.
Munia TTK, Marshall K, Kim K … +6 more , Misra D, DeLong G, Durrani A, Manikowski J, Vawdrey DK, Bhattacharya BS
AMIA Annu Symp Proc · 2024 · PMID 41726443
Predicting emergency department (ED) utilization can assist in resource planning like staff scheduling. Traditional time series methods and newer machine learning methods have been used to forecast ED metrics; however, t...
Predicting emergency department (ED) utilization can assist in resource planning like staff scheduling. Traditional time series methods and newer machine learning methods have been used to forecast ED metrics; however, they have seldom been implemented in operational settings. We leverage a user-centered design approach that engages nursing operations managers across multiple hospital sites in an integrated health system to identify the key metrics to predict, design and select the best models and time horizons, and design a production dashboard for ED operational planning. We tested various models in terms of mean absolute error and mean absolute percentage error and determined that Prophet (a non-linear open-source method) performed the best across multiple sites. We present the implementation and monitoring design for this model, generating daily, 14-day ahead predictions for ED arrival, admission, sitter needs, and ED holds, to be used by operational leaders to guide staffing decisions.
Building a Consumer Health Informatics Introductory Course Consensus Curriculum: An eDelphi Study.
Krol OF, Clark K, Mishra V … +5 more , Iyengar S, Martin C, Blondon K, Thanawala R, Doberne J
AMIA Annu Symp Proc · 2024 · PMID 41726442
Digital health technology is becoming increasingly sophisticated and prevalent in modern healthcare. Consumer health informatics (CHI) introductory courses provide a baseline proficiency on this important topic, yet ther...
Digital health technology is becoming increasingly sophisticated and prevalent in modern healthcare. Consumer health informatics (CHI) introductory courses provide a baseline proficiency on this important topic, yet there are no standardized competencies for graduates. In the rapidly evolving digital technology landscape, clear competencies are particularly crucial, as content may shift rapidly. We developed core competencies for introductory graduate level CHI courses utilizing the eDelphi method. To conduct this study, we explored literature and current teaching methods to identify themes among CHI, created of a panel of experts to serve on our advisory panel, then systematically solicited expert feedback using an iterative process. Our preliminary results have identified several areas experts deem as integral to any consumer health informatics curriculum - defining CHI and its applications, understanding how health literacy and numeracy influence design, and analyzing how information platforms can influence consumer outcomes and decision making.
Navigating Variability in Prostate RT Planning: Real-Time Insights for Human-Centered CDS Design.
Foster M, Byrd E, Kwong E … +7 more , Karunaker A, Anderson BM, Repka MC, McGurk R, Das SK, Marks LB, Mazur L
AMIA Annu Symp Proc · 2024 · PMID 41726441
Clinical variability in prostate radiation therapy (RT) planning is well documented, but little is known about how radiation oncologists experience and adapt to the factors that drive it. This study explores variability...
Clinical variability in prostate radiation therapy (RT) planning is well documented, but little is known about how radiation oncologists experience and adapt to the factors that drive it. This study explores variability as a human-centered design challenge, with the goalofinformingclinicaldecision support (CDS) design through real-timeinsight into planning decisions. We conducted observation sessions with the think aloud method followed by semi-structured interviews with five radiation oncologists while they contoured prostate cases. Using the Systems Engineering Initiative for Patient Safety (SEIPS) framework, we thematically analyzed the contributors to variability across tasks, technology, and organizational conditions. Results suggest that variability arises not only from anatomical or guidelineambiguity, butalso fromindividual interpretations of inputs, variation in contouring decisions, andadaptive strategies such as reliance on prior experience and estimation under uncertainty. Findings support the design of context-sensitive CDS tools that reflect real-world clinical reasoning while preserving clinical flexibility.
REPEAT BP: Reviewing Effective Practices for Elevating Adherence in Treatment of Hypertension.
Ly K, Harberger S, Chang A … +6 more , Wheeler J, Heimer S, Murali NS, Boell K, Kobylinski M, Vawdrey DK
AMIA Annu Symp Proc · 2024 · PMID 41726440
We report on the implementation of an evidence-based program for improving blood pressure (BP) control-the American Medical Association's MAP framework-across an integrated health system with almost 400 primary care prov...
We report on the implementation of an evidence-based program for improving blood pressure (BP) control-the American Medical Association's MAP framework-across an integrated health system with almost 400 primary care providers. We developed dashboards to track key metrics: the number of patients with uncontrolled BP, the frequency of recording confirmatory BP measurements during office visits, and the follow-up rates within 30 days for patients with repeated elevated readings. Our findings highlight the complexity of incorporating confirmatory BP measurements into clinic encounters, revealing significant variability in adoption by clinic and by provider. The implementation of the evidence-based program demonstrates that clinic-to-clinic and week-to-week inconsistencies in repeating BP measurements necessitate a systemic approach for effective BP control.
Machine Learning for Predicting Drug Release Behavior of PLGA Microspheres.
Catapano AF, Zheng L, Yuan X … +1 more , Yuan K
AMIA Annu Symp Proc · 2024 · PMID 41726439
PLGA microspheres are widely used in long-acting drug formulations due to their ability to provide sustained release, improving patient adherence and reducing dosing frequency. However, drug release behavior is influence...
PLGA microspheres are widely used in long-acting drug formulations due to their ability to provide sustained release, improving patient adherence and reducing dosing frequency. However, drug release behavior is influenced by complex formulation and processing factors, making traditional trial-and-error development inefficient. This study leverages machine learning to predict drug release profiles from PLGA (poly(lactic-co-glycolic acid)) microsphere formulations. A dataset of 113 PLGA formulations containing small-molecule drugs and large-molecule peptides was collected from published literature. Multiple machine learning models were developed and compared. The best-performing model achieved an R value of 0. 9415, a RMSE of 6.99% and a MAE of 4.35%, demonstrating strong predictive accuracy for in vitro drug release. Additionally, feature importance analysis was conducted, offering insights into key factors influencing release behavior and guiding the rational design of PLGA-based microspheres.
Right Patient, Right Specialist, Right Time: Retrieval Augmented Generation for Specialty Referral Routing.
Haredasht FN, Goh E, Ravi V … +17 more , Ashtari P, Jiang Y, Yuldashev N, Grolleau F, Gallo RJ, Shah A, Hur E, Chopra K, Jee O, Lee JJ, Rosengaus L, Giang L, Schulman K, Hom J, Milstein A, Ng AY, Chen JH
AMIA Annu Symp Proc · 2024 · PMID 41726438
We present an embedding-based retrieval system that automatically directs physician clinical questions to the most relevant specialist-curated question template, which is necessary for the specialist to provide a clinica...
We present an embedding-based retrieval system that automatically directs physician clinical questions to the most relevant specialist-curated question template, which is necessary for the specialist to provide a clinically relevant response. The system utilizes MPNet, a transformer-based model, to generate dense vector representations of both clinical queries and 24 predefined clinical templates. Given a clinical question, the system computes cosine similarity between the query and template embeddings to retrieve the most relevant matches. When validated against real-world, retrospective eConsults across five specialties, the system accurately identified the most relevant template in 87% of cases (success@1) and included it in the top three results 99% of the time (success@3). Automating specialty selection and clinical question referrals reduces the administrative burden on physicians, minimizes care delivery delays, and improves specialist responses by providing proper context.
No Black Box Anymore: Demystifying Clinical Predictive Modeling with Temporal-Feature Cross Attention Mechanism.
Li Y, Yao X, Padman R
AMIA Annu Symp Proc · 2024 · PMID 41726437
Despite the outstanding performance of deep learning models in clinical prediction tasks, explainability remains a significant challenge. Inspired by transformer architectures, we introduce the Temporal-Feature Cross Att...
Despite the outstanding performance of deep learning models in clinical prediction tasks, explainability remains a significant challenge. Inspired by transformer architectures, we introduce the Temporal-Feature Cross Attention Mechanism (TFCAM), a novel deep learning framework designed to capture dynamic interactions among clinical features across time, enhancing both predictive accuracy and interpretability. In an experiment with 1,422 patients with Chronic Kidney Disease, predicting progression to End-Stage Renal Disease, TFCAM outperformed LSTM and RE-TAIN baselines, achieving an AUROC of 0.95 and an F1-score of 0.69. Beyond performance gains, TFCAM provides multi-level explainability by identifying critical temporal periods, ranking feature importance, and quantifying how features influence each other across time before affecting predictions. Our approach addresses the "black box" limitations of deep learning in healthcare, offering clinicians transparent insights into disease progression mechanisms while maintaining state-of-the-art predictive performance.
Implementation and Assessment of Machine Learning Models for Forecasting Suspected Opioid Overdoses in Emergency Medical Services Data.
Mullen AD, Harris DR, Rock P … +4 more , Thompson K, Slavova S, Talbert J, Cody Bumgardner VK
AMIA Annu Symp Proc · 2024 · PMID 41726436
We present efforts in the fields of machine learning and time series forecasting to accurately predict counts of future suspected opioid overdoses recorded by Emergency Medical Services (EMS) in the state of Kentucky. Fo...
We present efforts in the fields of machine learning and time series forecasting to accurately predict counts of future suspected opioid overdoses recorded by Emergency Medical Services (EMS) in the state of Kentucky. Forecasts help government agencies properly prepare and distribute resources related to opioid overdoses. Our approach uses county and district level aggregations of suspected opioid overdose encounters and forecasts future counts for different time intervals. Models with different levels of complexity were evaluated to minimize forecasting error. A variety of additional covariates relevant to opioid overdoses and public health were tested to determine their impact on model performance. Our evaluation shows that useful predictions can be generated with limited error for different types of regions, and high performance can be achieved using commonly available covariates and relatively simple forecasting models.
Refining Substance Use Classification: An Ontological Framework for Enhancing Large-Scale Data Collection.
Lu CH, Leonard KE, Ceusters W
AMIA Annu Symp Proc · 2024 · PMID 41726435
Substance use disorders (SUD) remain prevalent in the United States. The Office of Addiction Services and Support plays a critical role in tracking SUD trends in New York State and reports data to the federal system. How...
Substance use disorders (SUD) remain prevalent in the United States. The Office of Addiction Services and Support plays a critical role in tracking SUD trends in New York State and reports data to the federal system. However, ambiguities in substance classification pose challenges to data accuracy and consistency. To address these issues, we developed the foundations for the Addiction Substance Ontology (ASO) using Basic Formal Ontology principles. Definitions in the ASO are expressed in terms of genus and differentiae which form the backbone for a taxonomy of substances in function of their chemical composition and certain other characteristics essential for tracking their acquisition and use. While 143 classes have been developed thus far based on a specific program admission use case, pilot testing and stakeholder collaboration are necessary to refine the ASO and validate its application in real-world settings. These efforts aim to improve data reliability, enhance tracking of SUD patterns, and support effective public health interventions.
Shifting Information Needs in Clinical Practice: The Evolving Role of Generative AI in Addressing Clinician Demands for Context-Specific Knowledge.
Tuteja SK, Boventer EL, Alkattan A … +2 more , Elhadad N, Rossetti SC
AMIA Annu Symp Proc · 2024 · PMID 41726434
This study explores clinicians' evolving information needs and evaluates the potential of Generative Artificial Intelligence (Gen AI) to address these gaps by reassessing and extending the Currie et al. (2003) taxonomy....
This study explores clinicians' evolving information needs and evaluates the potential of Generative Artificial Intelligence (Gen AI) to address these gaps by reassessing and extending the Currie et al. (2003) taxonomy. Despite advancements in electronic health records (EHRs), unresolved information needs persist, impacting clinical efficiency and patient care. A cross-sectional survey conducted at Columbia University Irving Medical Center (CUIMC) analyzed clinician-generated Gen AI prompts, comparing them against the 2003 taxonomy. Findings reveal that while 80% of prompts align with existing categories, 20% represent emerging needs, including AI-driven workflow optimization and fairness-related inquiries. These findings highlight the necessity of adapting clinical decision support frameworks to integrate AI-driven solutions, ensuring that modern tools meet evolving clinician needs. By formally extending the Currie et al. taxonomy, this study provides a foundational framework for leveraging Gen AI to bridge long-standing information gaps and enhance patient outcomes in an increasingly complex healthcare environment.
Humans and Large Language Models in Clinical Decision Support: A Study with Medical Calculators.
Wan NC, Jin Q, Chan J … +8 more , Xiong G, Applebaum S, Gilson A, McMurry R, Andrew Taylor R, Zhang A, Chen Q, Lu Z
AMIA Annu Symp Proc · 2024 · PMID 41726433
Although large language models (LLMs) have been assessed for general medical knowledge using licensing exams, their ability to support clinical decision-making, such as selecting medical calculators, remains uncertain. W...
Although large language models (LLMs) have been assessed for general medical knowledge using licensing exams, their ability to support clinical decision-making, such as selecting medical calculators, remains uncertain. We assessed nine LLMs, including open-source, proprietary, and domain-specific models, with 1,009 multiple-choice question-answer pairs across 35 clinical calculators and compared LLMs to humans on a subset of questions. While the highest-performing LLM, OpenAI's o1, provided an answer accuracy of 66.0% (CI: 56.7-75.3%) on the subset of 100 questions, two human annotators nominally outperformed LLMs with an average answer accuracy of 79.5% (CI: 73.5-85.0%). Ultimately, we evaluated medical trainees and LLMs in recommending medical calculators across clinical scenarios like risk stratification and diagnosis. With error analysis showing that the highest -performing LLMs continue to make mistakes in comprehension (49.3% of errors) and calculator knowledge (7.1% of errors), our findings highlight that LLMs are not superior to humans in calculator recommendation.
Detection of Youth Suicide Interventions in Clinical Record Text using an Open-Source Language Model.
Edgcomb JB, Klomhaus A, Lee J … +3 more , Ponce CG, Tascione E, Saha A
AMIA Annu Symp Proc · 2024 · PMID 41726432
This study presents an automated approach to detect youth suicide prevention interventions documented in emergency department notes. Expert review classified four interventions across 1,794 notes from 200 emergency visit...
This study presents an automated approach to detect youth suicide prevention interventions documented in emergency department notes. Expert review classified four interventions across 1,794 notes from 200 emergency visits for suicidality among children aged 6-17. The open-source language model Llama3.3-70B generated Likert scores (-3 to +3) for intervention presence. Scores approximated human classification: Lethal Means Restriction (AUROC 0.972, 95%CI:0.961-0.982), Hotline (AUROC 0.980, 95%CI:0.966-0.990), Outpatient Referral (AUROC 0.935, 95%CI:0.922-0.947), Safety Planning (AUROC 0.954, 95%CI:0.930-0.975). Application to 6,687 notes from 723 encounters revealed increased odds of omitted interventions among youth with prior emergency visits (OR 0.454-0.625), missing screening questions (OR 0.169-0.653), and ideation (vs. acts or attempts) (OR 0.296, 95%CI 0.133-0.657). Findings demonstrate systematic clinical text analysis with an open-source language model can expose new targets to inform decision support and strengthen evidence-based care for youth suicide.
Automating Lung-RADS Categorization And Follow-Up Recommendations Using In-Context Learning With Large Language Models.
Zhou T, Chen A, Hu Y … +8 more , Lou X, He X, Huang Y, Hochhegger B, Mehta H, Prosperi M, Guo Y, Bian J
AMIA Annu Symp Proc · 2024 · PMID 41726431
Lung cancer remains a significant challenge in public health, ranking among the leading causes of cancer-related mortality. Low-dose computed tomography (LDCT) -based lung cancer screening has emerged as an effective too...
Lung cancer remains a significant challenge in public health, ranking among the leading causes of cancer-related mortality. Low-dose computed tomography (LDCT) -based lung cancer screening has emerged as an effective tool for early detection, particularly in high-risk populations. However, interpreting lung nodule characteristics from radiology reports can often be time-consuming and labor-intensive due to the length and inherent ambiguity of the reports, even with standardized reporting requirements like Lung-RADS. Generating Lung-RADS assessments from original radiology reports is a significant task for radiologists. This study addresses these challenges by developing an in-context learning framework utilizing large language models (LLMs). In this process, we aimed to identify the best approach that accurately categorizes lung nodules and streamlines management decisions, providing robust and interpretable decision support. Overall, this research aims to reduce the time and effort of the radiologist in lung cancer screening, ultimately enhancing efficiency and accuracy and enabling timely and precise interventions.
Multi-Adversarial Debiasing in Clinical Artificial Intelligence.
Zawad MRS, Chen IY, Washington P
AMIA Annu Symp Proc · 2024 · PMID 41726430
While multiple types of biases can occur in clinical machine learning, the status quo in algorithmic debiasing is to optimize a single fairness metric in the training procedure. We propose a multi-adversarial debiasing f...
While multiple types of biases can occur in clinical machine learning, the status quo in algorithmic debiasing is to optimize a single fairness metric in the training procedure. We propose a multi-adversarial debiasing framework that builds on the established technique of adversarial debiasing to jointly optimize two or more fairness definitions. Our experiments use two adversaries corresponding to demographic parity (DP) and disparate mistreatment (DM). Evaluating four datasets, including two clinical datasets (UCI Heart Disease and a Parkinson's Disease digital health dataset) and two algorithmic fairness benchmarks (COMPAS and Adult Income), we find that our multi-adversarial approach reduces DP between 0.03-0.22 and DM between 0.02-0.12 while maintaining the F1 score within 0-16% of the baseline models. Analyzing these performance variations, we find that adversarial debiasing is most effective for datasets with adequate representation of positive and negative labels across protected attribute values, but the effectiveness declines when this is not the case.
Recommending Clinical Trials for Online Patient Cases using Artificial Intelligence.
Chan J, Jin Q, Wan N … +3 more , Floudas CS, Xue E, Lu Z
AMIA Annu Symp Proc · 2024 · PMID 41726429
Clinical trials are crucial for assessing new treatments; however, recruitment challenges-such as limited awareness, complex eligibility criteria, and referral barriers-hinder their success. With the growth of online pla...
Clinical trials are crucial for assessing new treatments; however, recruitment challenges-such as limited awareness, complex eligibility criteria, and referral barriers-hinder their success. With the growth of online platforms, patients, caregivers, and family members increasingly post medical cases on social media and health communities, while physicians publish case reports accessible on platforms like PubMed-collectively expanding recruitment pools beyond traditional clinical trial pathways. Recognizing this potential, we utilized TrialGPT, a framework that leverages a large language model, to match 50 online patient cases (collected from case reports and social media) to clinical trials and evaluate performance against traditional keyword-based searches. Our results show that TrialGPT outperformed traditional methods by 46%, with patients eligible, on average, for 7 of the top 10 recommended trials. Additionally, outreach to case authors and trial organizers yielded positive feedback. These findings highlight TrialGPT's potential to expand patient access to specialized care through non-traditional sources.
Metabolic Monitoring among Patients with Type 2 Diabetes Prescribed Second Generation Antipsychotics.
Guo J, Varghese JS
AMIA Annu Symp Proc · 2024 · PMID 41726428
.
.
Bias Evaluation and Mitigation in Retrieval-Augmented Medical Question-Answering Systems.
Ji Y, Zhang H, Wang Y
AMIA Annu Symp Proc · 2024 · PMID 41726427
Medical Question-Answering (QA) systems based on Retrieval-Augmented Generation (RAG) are promising for clinical decision support due to their capability to integrate external knowledge, thus reducing inaccuracies inhere...
Medical Question-Answering (QA) systems based on Retrieval-Augmented Generation (RAG) are promising for clinical decision support due to their capability to integrate external knowledge, thus reducing inaccuracies inherent in standalone large language models (LLMs). However, these systems may unintentionally propagate biases associated with sensitive demographic attributes like race, gender, and socioeconomic factors. This study systematically evaluates demographic biases within medical RAG pipelines across multiple QA benchmarks, including MedQA, MedMCQA, MMLU, and EquityMedQA. We quantify disparities in retrieval consistency and answer correctness by generating and analyzing queries sensitive to demographic variations. We further implement and compare several bias mitigation strategies-including Chain-of-Thought reasoning, Counterfactual filtering, Adversarial prompt refinement, and Majority Vote aggregation-to address identified biases. Experimental results reveal significant demographic disparities, highlighting that Majority Vote aggregation improves accuracy and fairness metrics. Our findings underscore the critical need for explicitly fairness-aware retrieval methods and prompt engineering strategies to develop truly equitable medical QA systems.
When Patients Go to "Dr. Google" Before They Go to the Emergency Department.
Grasso MA, Rogalski A, Farrukh N … +2 more , Kotal A, Calleros E
AMIA Annu Symp Proc · 2024 · PMID 41726426
Approximately one-third of adults search the internet for health information before visiting an emergency department (ED), with 75% encountering inaccurate content. This study examineshow such searches influence patient...
Approximately one-third of adults search the internet for health information before visiting an emergency department (ED), with 75% encountering inaccurate content. This study examineshow such searches influence patient care. We conducted an observational study of ED visits overa 12-month period, surveying 214 of 576 patients about pre-ED internet use. Data on demographics, comorbidities, acuity, orders, prescriptions, and dispositions were extracted. Patients who searched were typically younger, healthier, and more educated. Most used a general search engine to ask symptom-related questions. Compared to non-searchers, they were less likely to receive lab tests (RR 0.78, p=0.053), imaging(RR 0.75, p=0.094), medications (RR 0.67, p=0.038), oradmission (RR 0.68, p=0.175). They were more likely to leave against medical advice (RR 1.67, p=0.067) and receive opioids (RR 1.56, p=0.151). Findings suggest inaccurate health information may contribute to mismatched expectations and alter ed care delivery.
Unpacking Situational Awareness in Emergency Medical Services: An Eye-Tracking Study of Visual Attention.
Bai E, Zhang Z, Adelgais K … +1 more , Ozkaynak M
AMIA Annu Symp Proc · 2024 · PMID 41726425
Situational awareness (SA) is critical for Emergency Medical Services (EMS) providers as they operate in high-stakes, dynamic environments requiring rapid information processing and decision-making. While prior research...
Situational awareness (SA) is critical for Emergency Medical Services (EMS) providers as they operate in high-stakes, dynamic environments requiring rapid information processing and decision-making. While prior research has explored SA challenges in EMS, little is known about how visual attention patterns influence SA and clinical performance. This study employs eye-tracking technology to objectively assess how EMS providers allocate their visual attention during simulated pediatric emergency scenarios in urban and rural settings. We investigate variations in visual attention across experience levels, team structures, and task roles and examine differences between high- and low-performing teams. Results reveal that high-performing teams demonstrate more frequent and evenly distributed visual scanning, whereas lower-performing teams exhibit a narrowed focus, increasing the risk of missing critical cues. Our findings underscore the need for training interventions and technology solutions to enhance SA and optimize EMS performance.
Interpretable Machine Learning to Identify Risk Factors for Recidivism in Intimate Partner Violence.
Ogğuztüzün Ç, Koyutürk M, Karakurt G
AMIA Annu Symp Proc · 2024 · PMID 41726424
Intimate Partner Violence (IPV) remains a significant global health issue with severe consequences ranging from physical injury to death, with rates rising in recent years. Prediction of recidivism is critical for preven...
Intimate Partner Violence (IPV) remains a significant global health issue with severe consequences ranging from physical injury to death, with rates rising in recent years. Prediction of recidivism is critical for prevention and treatment. Using data from a four-year clinical study, we develop interpretable machine-learning models to identify features for physical assault recidivism among IPV offenders. To standardize clinician-assigned severity scores and address non-linear associations, we apply filtered target encoding, which reduces subjectivity and bias in assessment. We find that combining self-reported and partner-reported variables enhances predictive power. Through feature importance analyses, we identify factors associated with lower recidivism risk, including decreased substance use and avoiding partner contact, while separation processes correlate with higher reoffending likelihood. These findings advance IPV risk assessment by providing a deeper understanding of risk factors critical for improving treatment effectiveness and addressing disparities in IPV management.
About
- Frequency
- Sun
- Papers found
- 200
- RSS feed
- Subscribe