International Journal Of Medical Informatics[JOURNAL]
Sun
200 papers
Medical students' use of large language models: a national survey.
Barr AA, Rozman RC, Liu K … +12 more , Pham M, Klarenbach Z, Chinna-Meyyappan A, Hassan AY, Zarychta M, Ferri OE, Al-Khaz'Aly A, Datt P, Herik AI, Sadek K, Paget M, Holodinsky JK
Int J Med Inform · 2026 Jul · PMID 42400960 · Publisher ↗
BACKGROUND: Large language models (LLMs) are increasingly embedded in medical education and clinical care settings, yet contemporary Canadian data describing medical students' use and perceptions remain limited. OBJECTIV...
BACKGROUND: Large language models (LLMs) are increasingly embedded in medical education and clinical care settings, yet contemporary Canadian data describing medical students' use and perceptions remain limited. OBJECTIVE: To quantify the prevalence, frequency, and patterns of LLM use among medical students in Canada; to characterize perceptions of utility, accuracy, limitations, and impact; and to describe perceived barriers, challenges, and ethical/privacy concerns. METHODS: We conducted a national, cross-sectional survey distributed to English-speaking medical students between November and December 2025. Recruitment occurred through medical school channels, student unions, and national/regional student organizations. RESULTS: Among 286 respondents from 10 medical schools, 96.50% reported using at least one LLM. The most commonly used LLMs were ChatGPT (93.36%) and OpenEvidence (57.69%). Daily/weekly use was most frequent for coursework assistance (60.22%) and clinical questions (57.14%). Most respondents reported positive impacts on efficiency (81.62%), learning (77.01%), and academic performance (59.49%). Students commonly reported encountering inaccurate information (90.18%). Formal instruction on LLM use was uncommon (10.95%), though 67.67% of students agreed medical schools should integrate formal instruction on LLMs. Only 21.43% of respondents felt adequately educated on data privacy regulations applicable to these tools. CONCLUSION: While LLM use among surveyed medical students in Canada was nearly universal and perceived favorably, students reported exposure to inaccurate outputs and substantial gaps in formal training and privacy literacy. These findings support the development of structured curricular guidance on appropriate application of these tools, including information verification practices and ethical, privacy-aware engagement.
BlockFedMed: A blockchain-federated learning framework for privacy-preserving mortality prediction across heterogeneous intensive care units.
Yadav AK, Deshmukh M
Int J Med Inform · 2026 Jun · PMID 42391667 · Publisher ↗
BACKGROUND: Electronic health records are distributed across different hospitals that work on powerful AI models but cannot be shared due to HIPAA and GDPR regulations. Federated learning (FL) avoids raw data sharing, ye...
BACKGROUND: Electronic health records are distributed across different hospitals that work on powerful AI models but cannot be shared due to HIPAA and GDPR regulations. Federated learning (FL) avoids raw data sharing, yet lacks tamper-evident consent governance, adversarial robustness, and verifiable differential privacy (DP) accounting leaving regulatory compliance undemonstrated. OBJECTIVE: To develop and externally validate BlockFedMed, a blockchain-orchestrated FL framework providing cryptographically verifiable consent, model-update integrity, and on-chain DP audits for multi-site ICU mortality prediction, and to quantify its operational clinical impact beyond algorithmic performance. METHODS: BlockFedMed integrates Hyperledger Fabric v2.5 with a federated bidirectional LSTM and Gaussian DP (ε=3.2, δ=10). Three smart contracts govern consent (CMC), integrity (Mic), and incentive (Idc). The Byzantine fault-tolerant aggregator FedMed-Bft accepts only Mic-verified updates. Design-phase training used MIMIC-IV (n=52,167 ICU admissions). External validation used the entirely independent eICU Collaborative Research Database (n=200,859; 208 hospitals), unseen during model development. RESULTS: On external eICU validation, BlockFedMed achieved an AUROC of 0.841 (95% CI: 0.828-0.854) for in-hospital mortality, which was 7.4 points above Local-Only (p<0.001) and within 3.1% of the regulatory-prohibited centralised upper bound. Simulated consent-management latency fell 71% (from 28.3 min to 8.2 min per cohort) under controlled workflow conditions; prospective clinical measurement remains as future work. The Fabric network sustained 1240 TPS at 1.83 s latency. FedMed-Bft maintained AUROC ≥0.836 under six simultaneous Byzantine participants, all correctly flagged on-chain. CONCLUSIONS: BlockFedMed delivers externally validated ICU mortality prediction with cryptographically auditable privacy and consent governance, demonstrating that blockchain-FL provides strong promise for meeting both clinical performance and regulatory compliance requirements simultaneously, pending prospective multi-centre deployment validation.
Integrating clinical decision support systems in pediatric oncology: A scoping review of applications, implementation gaps, and management Implications.
Zhu M, Yao Z, Feng X … +8 more , Li Y, Zhang H, Peng J, Li Y, Li Y, Chu X, Bao W, Tian L
Int J Med Inform · 2026 Jul · PMID 42385458 · Publisher ↗
BACKGROUND: Clinical Decision Support Systems (CDSSs) are acknowledged as tools that leverage health data analytics to support clinical decision-making; however, systematic evidence regarding their deployment in pediatri...
BACKGROUND: Clinical Decision Support Systems (CDSSs) are acknowledged as tools that leverage health data analytics to support clinical decision-making; however, systematic evidence regarding their deployment in pediatric oncology remains limited. This gap in the literature gives rise to unique challenges, particularly in addressing the individualized nature of pediatric cancer diagnosis, treatment, and the "doctor-nurse-patient" tripartite decision-making. AIM: To synthesize evidence on the application of CDSSs in the management of pediatric cancer patients, elucidate prevailing patterns of clinical utilization, and critically examine barriers to effective implementation in real-world practice. METHODS: This review adhered to PRISMA-ScR guidelines. Six databases were systematically searched: PubMed, Web of Science, Embase, China National Knowledge Infrastructure (CNKI), Wanfang Database, and China Science and Technology Journal Database (VIP). Two independent reviewers conducted study selection and content analysis to ensure rigor and minimize bias. RESULTS: Seventeen studies were included, revealing four critical pertinent issues: (1) CDSSs demonstrate potential for application in pediatric cancer management; (2) the majority of applications are concentrated in the treatment phase (41.2%), with comparatively less emphasis on risk assessment, diagnosis, nursing workflow, and survivorship care; (3) user interaction and follow-up functionalities remain suboptimal and warrant further optimization; and (4) there is a critical need for multicenter randomized controlled trials to validate the clinical efficacy of CDSSs. CONCLUSIONS: This review delineates the characteristics of included studies, the specific features of CDSSs, and their clinical applications, including decision-making content, system functionalities, evaluation indicators of application effectiveness, in pediatric cancer. It also identifies critical gaps and future research directions to advance the field.
Understanding digital health capability of allied health professionals - a mixed-methods study with content validity analysis.
Pearson E, D'Souza AN, O'Brien K … +1 more , Feely K
Int J Med Inform · 2026 Jun · PMID 42385457 · Publisher ↗
BACKGROUND AND PURPOSE: Digital technology is an essential enabler in modern health care. To maximise potential benefits, the health workforce requires adequate digital skills. Understanding a workforce's skills provides...
BACKGROUND AND PURPOSE: Digital technology is an essential enabler in modern health care. To maximise potential benefits, the health workforce requires adequate digital skills. Understanding a workforce's skills provides the opportunity to identify and address gaps in knowledge. This study aimed to enhance content validity of the Victorian Allied Health Digital Health Self-Evaluation Tool Domain 3 and to investigate the perceived digital capability of an Australian allied health workforce. METHODS: Content validity of Domain 3: Data and Informatics of the self-evaluation tool was assessed and modified using cognitive interviewing methods. Interviews with allied health professionals involved participants 'thinking out loud' while completing the tool. The research team updated the tool until no further issues were identified. Allied health professionals from three public hospitals in Melbourne's Parkville precinct were then invited to complete the modified self-evaluation tool via electronic survey. Analyses were descriptive. RESULTS: Eleven cognitive interviews were conducted with allied health participants in three rounds, after which no further adjustments to the self-evaluation tool were required. Researchers simplified the tool iteratively, removing ambiguities, adding educational material and infographics to create a modified data and informatics self-evaluation tool. A total of 346 allied health professionals (46 science and 300 therapy) completed the modified tool. Most participants were female (n = 291, 84%) and had practised their profession a median of 9 (interquartile range 5-19) years. Most self-rated their capability across the ten data and informatics statements as Foundation (42.1%) or Consolidation (43.5%) with few self-ratings as Expert (11.8%) or Leader (2.6%). CONCLUSION: Workforce gaps in perceived data and informatics capability were identified. The results enable future development of a targeted digital training strategy to uplift workforce capability. Further work is required to improve content validity for the other three domains of the Victorian Allied Health Digital Health Self-Evaluation Tool.
On-premises open-source large language models for privacy-preserving multimodal depression screening.
Kwon S, Kim Y, Jhon M … +6 more , Park JH, Lim B, Jeon E, Kim JM, Kim JW, Lee H
Int J Med Inform · 2026 Jun · PMID 42379124 · Publisher ↗
OBJECTIVE: Narrative and speech data can provide valuable signals for depression screening, yet privacy and data-governance requirements often limit the use of closed-based models in clinical practice. In addition, exist...
OBJECTIVE: Narrative and speech data can provide valuable signals for depression screening, yet privacy and data-governance requirements often limit the use of closed-based models in clinical practice. In addition, existing large language model (LLM)-based approaches are largely text-centric, and multimodal integration of acoustic features and structured clinical variables remains limited. This study aimed to develop and externally validate a privacy-preserving multimodal depression screening prediction framework using open-source large language models that integrate sociodemographic information, emotion-memory narratives, and acoustic features. METHOD: This study analyzed 3536 participants collected at Chonnam National University Hospital. Inputs combined sociodemographic and lifestyle variables, Korean transcripts of happy- and sad-memory narratives, and speech-derived extended Geneva minimalistic acoustic parameter set (eGeMAPS) features. To maintain prompt conciseness, statistically significant features were selected from the 88 eGeMAPS features extracted for each happy- and sad-memory narrative, with Mann-Whitney U tests conducted exclusively on the internal training split to prevent data leakage. Five open-source LLMs (Gemma-3-27B, Qwen-3-32B, Llama-3.3-70B, Phi4-14B, and gpt-oss-20b) were evaluated under zero-shot prompting, Chain-of-Thought prompting, and supervised fine-tuning. External validation used Extended Distress Analysis Interview Corpus (E-DAIC) (N = 275). RESULTS: Under zero-shot prompting, the best internal F1-score was 0.735 (Gemma-3-27B). Chain-of-Thought prompting improved Llama-3.3-70B (F1-score = 0.708) but reduced performance for other models. Supervised fine-tuning improved all models, yielding internal accuracies of 0.852 to 0.881 and F1-scores of 0.818 to 0.865 across five models, corresponding to F1 gains of 0.12 to 0.30 versus prompting-only approaches. In external validation, accuracy ranged from 0.764 to 0.822 and F1-score ranged from 0.683 to 0.807. CONCLUSION: This study suggests that multimodal open-source LLMs integrating clinical variables, narrative text, and acoustic features can support privacy-preserving depression screening in an on-premises setting. Supervised fine-tuning provided the most consistent performance improvements, and external validation supported robustness beyond the development cohort.
Data mining methods, tasks, and algorithms for adverse drug reaction analysis in pharmacovigilance: A scoping review.
Jacoby AC, Matsuda MA, Blatt CR … +1 more , Cazella SC
Int J Med Inform · 2026 Jun · PMID 42378892 · Publisher ↗
BACKGROUND: Pharmacovigilance is dedicated to the identification, evaluation, and prevention of adverse effects associated with the use of medicines after their commercialization. In this context, data mining techniques...
BACKGROUND: Pharmacovigilance is dedicated to the identification, evaluation, and prevention of adverse effects associated with the use of medicines after their commercialization. In this context, data mining techniques have been widely employed for the detection of safety signals. Although machine learning algorithms show potential to identify complex patterns and improve the prediction of adverse drug reactions, their application in pharmacovigilance databases remains limited. OBJECTIVE: To map the computational approaches used in pharmacovigilance, to identify the prevalence of traditional statistical methods and data mining techniques, and to assess the role of machine learning algorithms. METHODS: This scoping review followed PRISMA‑ScR guidelines (protocol registered on OSF: https://doi.org/10.17605/OSF.IO/KZJDT). We searched PubMed, Scopus, Embase, and Web of Science for English primary studies published from 2015 to July 2025 that applied data mining techniques to pharmacovigilance databases. Data extraction used a standardized form. RESULTS: The search identified 1,468 records, of which 162 studies were included after screening and eligibility assessment. Traditional disproportionality methods, such as Reporting Odds Ratio (ROR) and Proportional Reporting Ratio (PRR), were used in 87.7% of the studies, whereas 12.3% applied machine learning or deep learning techniques, mainly in classification tasks, with logistic regression being the most frequently employed algorithm. The most investigated drugs included immune checkpoint inhibitors, such as nivolumab, pembrolizumab, atezolizumab, and durvalumab. The most studied therapeutic classes were antineoplastic agents, immunosuppressants, and psycholeptics. CONCLUSION: Signal detection in pharmacovigilance remains predominantly based on classical statistical methods. Progress in the field is still constrained by the slow incorporation of advanced machine learning techniques and the limited public availability of datasets used in analyses.
Development and validation of an interpretable machine learning model for predicting systemic inflammatory response syndrome after percutaneous nephrolithotomy: A multicenter study.
Wang L, He W, Qiu T … +3 more , Shi G, Zhao C, Li D
Int J Med Inform · 2026 Jun · PMID 42378891 · Publisher ↗
BACKGROUND: Systemic inflammatory response syndrome (SIRS) is a common and potentially reversible early infectious complication after percutaneous nephrolithotomy (PCNL). Accurate perioperative risk stratification may en...
BACKGROUND: Systemic inflammatory response syndrome (SIRS) is a common and potentially reversible early infectious complication after percutaneous nephrolithotomy (PCNL). Accurate perioperative risk stratification may enable timely intervention; however, existing prediction models show limited generalisability and poor clinical interpretability. METHODS: We conducted a multicenter retrospective cohort study including patients who underwent PCNL at three hospitals in China between Jan 1, 2015, and Dec 30, 2024. Patients from one center were randomly divided into training and internal validation cohorts, while two independent cohorts served for external validation. Perioperative demographic, laboratory, imaging, and surgical variables were collected. Feature selection was performed using least absolute shrinkage and selection operator regression and the Boruta algorithm. Seven machine learning models were developed and compared. Given outcome imbalance, the area under the precision-recall curve (AUPRC) was prespecified as the primary performance metric. Model calibration, decision curve analysis, and SHapley Additive exPlanations (SHAP) were used to assess reliability, clinical utility, and interpretability. RESULTS: A total of 2,684 patients were included, with postoperative SIRS occurring in 9.8%-12.7% across cohorts. Six predictors were consistently identified: stone size, urine nitrite, urine culture results, operative time, residual stone status, and the neutrophil-to-albumin ratio. The random forest model showed the most balanced performance, with AUPRC values ranging from 0.581 to 0.641 and AUROC values from 0.873 to 0.920 across validation cohorts. Calibration was satisfactory, and decision curve analysis demonstrated a higher net clinical benefit than alternative models. SHAP analysis revealed clinically coherent, non-linear associations between key predictors and SIRS risk. An online prediction tool was developed to support individualized risk estimation. CONCLUSION: An interpretable machine learning model based on routinely available perioperative variables can reliably predict SIRS after PCNL across multiple centers. This approach may facilitate early postoperative risk stratification and support timely clinical decision-making to mitigate infectious complications. Prospective and multi-regional validation is warranted.
Prompt engineering experiment on ChatGPT's ability to recommend orthopedic surgeons.
Haupt MR, Massillon D, Yang L … +4 more , Chen Y, Natarajan A, Ward SR, Mackey TK
Int J Med Inform · 2026 Jun · PMID 42378890 · Publisher ↗
People often use search engine results and online reviews to find health services and medical practitioners. However, it can be overwhelming to choose a practitioner when given hundreds of physician listings, making it t...
People often use search engine results and online reviews to find health services and medical practitioners. However, it can be overwhelming to choose a practitioner when given hundreds of physician listings, making it tempting for people to use large language models (LLMs) such as ChatGPT to provide doctor recommendations. The present study examines ChatGPT's ability to provide recommendations for real medical practitioners (i.e., orthopedic surgeons) and experimentally tests how the inclusion of patient characteristics (e.g., age, race, income) in prompt queries impacts responses. Out of 40,500 queries, ChatGPT stated that it was unable to provide recommendations for 52.8 % of responses. Results show that ChatGPT varied its recommendation response depending on personal characteristic of tested patient personas (e.g., race). Patient characteristics most relevant to healthcare access - income, health insurance status, and location - were the strongest predictors on whether ChatGPT provided a surgeon recommendation. Specifically, prompts where the patient persona had a high income and stated they had health insurance were significantly more likely to receive recommendations. Further, patient prompts based in New York City and Chicago were more likely to receive a recommendation compared to Phoenix and Houston. A subset of coded responses (n = 1000) show that only 44.6 % of recommended surgeons were valid. Among the valid surgeons, 97.1 % (n = 433) were male and 81.4 % (n = 363) were White. Our findings show that ChatGPT does not reliably provide valid surgeon recommendations and suggests it may be biased when given personal descriptions of patients when making queries.
Forecasting NHS hospital demand under two structural breaks: A multiple-comparison-corrected audit of seven forecasting methods on 15 years of public data.
Cece S, Köse İ, Elmas BÖ
Int J Med Inform · 2026 Jun · PMID 42378889 · Publisher ↗
BACKGROUND: Machine learning (ML) is increasingly used for hospital demand forecasting, but few published studies report multiple-comparison-corrected statistical tests against competitive baselines, and few characterise...
BACKGROUND: Machine learning (ML) is increasingly used for hospital demand forecasting, but few published studies report multiple-comparison-corrected statistical tests against competitive baselines, and few characterise the structural-break context in which forecasts are evaluated. OBJECTIVE: To audit seven AI/ML methods against statistical baselines on 15 years of NHS England activity data, with explicit attention to structural breaks and pandemic confounding. METHODS: Monthly A&E activity (n = 189) and 52-quarter KH03 bed occupancy (2010-2026) were obtained from public NHS England releases. Andrews sup-F (q = 2 parameters, 15% trimming) and Bai-Perron breakpoint analyses characterised structural breaks. Seven forecasters were benchmarked on a 12-month chronological hold-out and 12-fold rolling-origin cross-validation, with Diebold-Mariano tests and Bonferroni-Holm + Benjamini-Hochberg correction. A pre-pandemic sensitivity (n = 115) assessed COVID-19 confounding. Detailed methodology is in Appendix A (separate file). RESULTS: 4-h wait failures grew from ∼ 3,700 to ∼ 123,000 per month (∼33-fold). Andrews sup-F = 25.4 placed the demand-side break at 2014-10 (compatible with NHS Five Year Forward View); an independent bed-occupancy break at 2020-09 aligned with NHS COVID-19 reorganisation. On the COVID-inclusive hold-out, only LSTM was significantly worse than naive under Holm correction (p = 0.029); six of seven methods showed no significant difference, but every modern-ML method had a worse point MAPE than the seasonal-naive baseline. Pre-pandemic, only the statistical methods ETS and Prophet significantly beat naive. Under horizon-matched rolling-origin one-step evaluation with Diebold-Mariano testing, no method significantly outperformed the seasonal-naive y[t-12] baseline; the apparent ML advantage held only against the weaker three-year-mean baseline. CONCLUSION: After horizon-matched, multiple-comparison-corrected evaluation, modern ML showed no reliable advantage over a strict seasonal-naive baseline on monthly system-level NHS data; the statistical methods Prophet and ETS were the only models to beat naive significantly, and only in the pre-pandemic regime. The apparent rolling-origin ML "recovery" reflected an easier one-step task and a weaker baseline rather than a genuine evaluation-scheme effect. The practical message is that complex ML should not be assumed superior for monthly NHS demand planning unless it demonstrably beats a strong seasonal-naive baseline under rolling-origin, multiple-comparison-corrected testing.
Informed consent and ethical considerations in AI for dentistry and medicine: a scoping review.
Allende AB, Ehsani SS, Eber P … +12 more , Ikram Ullah A, Kosan E, Saif N, Alimohammadi M, Maureen O, Kientega FD, Galrão T, Ghazali AB, Meisha DE, Youssef Abdelsalam MA, Moore S, Khare P
Int J Med Inform · 2026 Jun · PMID 42372578 · Publisher ↗
BACKGROUND: Informed consent (IC) is central to patient autonomy, yet its role in artificial intelligence (AI) for clinical deployment, model development, and secondary data use remains unclear in medicine and dentistry....
BACKGROUND: Informed consent (IC) is central to patient autonomy, yet its role in artificial intelligence (AI) for clinical deployment, model development, and secondary data use remains unclear in medicine and dentistry. OBJECTIVES: This review characterised how IC is justified and operationalised for AI; synthesised ethical, legal, governance, and practical requirements; identified gaps in consent models, stakeholders, and AI functionality; and developed author-derived communication thresholds for notification, routine clinical consent with explicit AI disclosure, or AI-specific IC. METHODS: We conducted a PRISMA-ScR-guided scoping review with an OSF-registered protocol. MEDLINE, Scopus, IEEE Xplore, arXiv, Google Scholar, Web of Science, and HeinOnline were searched for English-language sources published 2015 to 25 May 2026. From 6,242 records, 116 reports were assessed; 69 were included, plus one manual source, yielding 70. Data were charted across 24 domains, synthesised, and appraised with JBI tools. RESULTS: Publications peaked in 2024 (22/70, 31.4%). The evidence base was non-empirical: conceptual analyses (37/70, 52.9%) and narrative reviews/book chapters (17/70, 24.3%). Medicine-only sources predominated (60/70, 85.7%); dentistry-only sources accounted for 8/70 (11.4%). Traditional IC appeared alone in 46/70 sources (65.7%) and overall in 52/70 (74.3%); dynamic consent was uncommon (6/70, 8.6%). IC was endorsed in 67/70 (95.7%) and qualified in 40/70 (57.1%). Explainability/transparency was addressed in 65/70 (92.9%), and proposed solutions in 57/70 (81.4%), but formal protocols remained uncommon (6/70, 8.6%). Thresholds consolidated rules by AI application, automation, risk, data use, and patient decision relevance. CONCLUSIONS: AI-related IC is widely endorsed but remains fragmented and largely conceptual. Findings support a risk-adaptive approach to AI-informed consent, calibrated to AI function, automation, risk, data use, explainability, and clinical decision impact. The author-derived thresholds offer a synthesis-informed basis for future governance guidance or framework development, pending empirical and stakeholder validation.
An explainable machine learning framework for computable physiologic risk representation in preanesthetic assessment: Development and external validation.
Huang HI, Huang CC, Fan CH … +2 more , Chuang HI, Wu KP
Int J Med Inform · 2026 Jun · PMID 42364468 · Publisher ↗
BACKGROUND: Preanesthetic evaluation requires clinicians to synthesize heterogeneous clinical information, yet perioperative risk communication often relies on categorical representations that do not fully capture physio...
BACKGROUND: Preanesthetic evaluation requires clinicians to synthesize heterogeneous clinical information, yet perioperative risk communication often relies on categorical representations that do not fully capture physiologic risk. The American Society of Anesthesiologists Physical Status (ASA-PS) classification is widely used but remains subjective, with inter-rater and institutional variability. This study aimed to develop and externally validate an explainable machine learning-based framework for physiologic severity representation in preanesthetic risk assessment. METHODS: A retrospective study used data from two institutions in Taiwan: a regional hospital (n = 1,200) for model development and internal validation, and a tertiary medical center (n = 113) for external validation. The target was a physiologic severity label derived from postoperative Acute Physiology and Chronic Health Evaluation II (APACHE II) scores ≥ 12, used as a standardized reference for supervised learning. Feature selection used least absolute shrinkage and selection operator (LASSO) regression, and class imbalance was addressed using an EasyEnsemble-Light Gradient Boosting Machine (LightGBM) approach. Model performance was evaluated using the area under the receiver operating characteristic curve (AUROC), calibration analysis, and decision curve analysis (DCA). RESULTS: The model incorporated six variables: surgical site, age, white blood cell count, heart rate, mean arterial pressure, and smoking status. In internal validation, AUROC was 0.94 (95 % CI 0.90-0.97) versus ASA-PS 0.71 (P < 0.001). In external validation, AUROC was 0.88 (95 % CI 0.81-0.94) versus ASA-PS 0.76. DCA demonstrated favorable net benefit for the APACHE II-derived physiologic severity reference compared with ASA-PS across clinically relevant thresholds. SHapley Additive exPlanations (SHAP) analysis identified cases with discordant risk patterns compared with ASA-PS, illustrating individualized physiologic severity estimates beyond categorical classification. CONCLUSIONS: The proposed framework provides an externally validated, computable physiologic severity representation that complements ASA-PS-based assessment and may support consistent perioperative risk communication and preanesthetic decision support.
Large language models for automated PRISMA 2020 adherence checking.
Kataoka Y, So R, Banno M … +7 more , Tsujimoto Y, Takayama T, Yamagishi Y, Tsuge T, Yamamoto N, Suda C, Furukawa TA
Int J Med Inform · 2026 Jun · PMID 42364467 · Publisher ↗
BACKGROUND: Evaluating adherence to PRISMA 2020 guideline remains a burden in the peer review process. However, there is a lack of shareable benchmarks for evaluating large language model (LLM) performance in this task....
BACKGROUND: Evaluating adherence to PRISMA 2020 guideline remains a burden in the peer review process. However, there is a lack of shareable benchmarks for evaluating large language model (LLM) performance in this task. METHODS: We constructed a copyright-aware benchmark of 108 Creative Commons-licensed systematic reviews. We first conducted parameter optimization using five SRs from the Suda dataset, then compared five checklist input formats (Markdown, JSON, XML, plain text, and manuscript-only control) using ten development-phase LLMs on ten further SRs from the Suda dataset, and finally validated the locked Markdown pipeline using nineteen LLMs on ten SRs from the Tsuge dataset as additional frontier models became available during the study period. RESULTS: Supplying structured PRISMA 2020 checklists yielded 78.7-79.7% accuracy versus 45.2% for manuscript-only input, with paired aggregate analyses showing that structured formats outperformed manuscript-only input while structured formats did not differ significantly from one another. In the validation sample, accuracy ranged from 68.5% to 86.0% with distinct sensitivity-specificity trade-offs. Using Qwen3-Max on the full dataset (n = 120), we achieved 95.1% sensitivity and 49.3% specificity. DISCUSSION: Structured checklist provision substantially improves LLM-based PRISMA assessment. However, given the observed proportion of false positives, human expert verification remains essential before editorial decisions.
Artificial intelligence-enhanced nurse navigation for monitoring and care of long COVID.
Bandeira Barboza AP, Monteiro RL, Muschi AL … +18 more , Dos Santos Silva AR, Silva DAD, Silveira RC, Alencar Silva CS, de Castro GS, Santana JV, Soares Silva RDS, Cerri GG, Rodrigues Pereira AJ, Beco MA, Romitelli G, Veneziani JR, Santos Silva EMD, Barboza ADS, Mattar LL, Sabino EC, Ribeiro RS, Costa SF
Int J Med Inform · 2026 Jun · PMID 42364466 · Publisher ↗
INTRODUCTION: Long COVID is a multisystem condition with challenging diagnosis. Nurse-navigation, a patient-centered intervention, can enhance education and care access. Analyzing patient-nurse text message exchanges usi...
INTRODUCTION: Long COVID is a multisystem condition with challenging diagnosis. Nurse-navigation, a patient-centered intervention, can enhance education and care access. Analyzing patient-nurse text message exchanges using natural language processing (NLP) enables automated extraction of clinical information, potentially supporting early identification of long COVID. We aimed to evaluate a digital nurse navigation platform integrating a predictive model for long COVID identification as a triage-assisting tool and to assess user acceptance. METHODS: This observational study included patients and healthcare professionals diagnosed with COVID-19 from January to July 2024. Participants received nurse-navigation support for 16 weeks with monthly interactions via a WhatsApp-integrated platform. Structured sociodemographic and clinical data were combined with text-message insights using NLP techniques such as term frequency-inverse document frequency (TF-IDF), and analyzed using language models (Gemini 1.5 Pro, BERTimbau) and probabilistic linkage. The dataset was split into 70% training and 30% testing, and eight machine learning models were evaluated. Performance metrics included accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUROC). User satisfaction was assessed with the Net Promoter Score (NPS). RESULTS: Among 177 participants, 141 (78%) were female, with an overall mean age of 51 years. A total of 7,016 messages were processed. Long COVID was identified in 60 participants (33%), most frequently reporting memory loss, dyspnea, cognitive fatigue, and hair loss. Participants received structured education, and 20 were referred for further evaluation. The XGBoost-minor model achieved the highest classification performance with an accuracy of 72%, sensitivity of 38%, specificity of 88%, PPV of 63%, NPV of 74%, and AUROC 0.59. Predictive factors included age, COVID-19 episodes, vaccination, comorbidities, and respiratory symptoms. The NPS was 92, indicating strong endorsement. CONCLUSION: An AI-enhanced triage process within nurse navigation represents a promising and scalable strategy to support the identification and monitoring of patients at risk for long COVID.
Chain-of-verification prompting for NIH stroke scale extraction using small and frontier large language models.
Lott J, Stubbert B, McShannon D … +3 more , Bellissimo J, Patel D, Dietrich N
Int J Med Inform · 2026 Jun · PMID 42364465 · Publisher ↗
BACKGROUND: The National Institutes of Health Stroke Scale (NIHSS) is critical to acute stroke care but is often documented in unstructured notes. Large language models (LLMs) can enable automated extraction, though smal...
BACKGROUND: The National Institutes of Health Stroke Scale (NIHSS) is critical to acute stroke care but is often documented in unstructured notes. Large language models (LLMs) can enable automated extraction, though smaller models often underperform relative to frontier systems. Chain-of-Verification (CoVe) prompting introduces a structured self-verification step that may improve performance. METHODS: We evaluated eight LLMs on 312 discharge summaries. Small models included LLaMA 3.2 3B, Ministral 3B, Gemma 3 4B, and Qwen 3 4B. Frontier models included GPT-5.2, Gemini 3 Pro, Claude Opus 4.5, and Grok 4. Each model was tested under a baseline and CoVe prompt. Outcomes were subscore exact-match accuracy, subscore mean absolute error (MAE), total score exact-match accuracy, and total score MAE. RESULTS: At baseline, small models achieved 53.2 ± 10.0% subscore accuracy and subscore MAE 0.84 ± 0.22, compared with 88.5 ± 10.1% and 0.15 ± 0.16 in frontier models (both p < 0.001). Total exact accuracy was low in both groups (7.7 ± 12.9% vs 35.9 ± 32.4%). CoVe significantly improved small-model performance (subscore accuracy 65.0 ± 10.9%; subscore MAE 0.55 ± 0.21; total MAE 4.84 ± 2.30 vs 7.19 ± 3.54 at baseline; all p < 0.001), although total exact accuracy remained modest (9.6 ± 15.7%). Frontier models showed no significant group-level change with CoVe. CONCLUSION: CoVe prompting substantially improves NIHSS extraction in small LLMs while producing negligible effects in frontier models. Although smaller model performance remains insufficient for standalone clinical deployment, CoVe prompting offers a promising avenue for further exploration.
Pilot evaluation of smart glasses for communication workflow and task performance in prehospital care.
Zhang Z, Bai E, Xu Y … +7 more , Ozkaynak M, Kutzin J, Hanna M, Nudell N, Ruble C, Zapcic-Desrochers M, Adelgais K
Int J Med Inform · 2026 Jun · PMID 42364464 · Publisher ↗
INTRODUCTION: Communication between emergency medical services (EMS) clinicians and online medical control (OLMC) physicians is essential for effective prehospital decision-making but is currently limited by verbal-only...
INTRODUCTION: Communication between emergency medical services (EMS) clinicians and online medical control (OLMC) physicians is essential for effective prehospital decision-making but is currently limited by verbal-only tools (e.g., phone or radio). OBJECTIVE: This pilot study aimed to evaluate the impact of smart glasses on EMS-OLMC communication, workflow, and clinical task performance. METHODS: We conducted a pilot randomized crossover trial with 16 simulated EMS sessions across two agencies in the United States. EMS clinicians managed two scenarios using either smart glasses or standard communication (radio/phone) to consult with OLMC physicians. Task completeness and physician prompts were independently assessed by paired paramedic and physician reviewers using a standardized checklist, and both simulation and teleconsultation durations were analyzed to evaluate workflow impact. RESULTS: Smart glasses affected both workflow and clinical performance. Workflow analysis showed significantly longer simulation and teleconsultation durations when smart glasses were used compared with phone sessions (p < 0.05). Clinically, smart glasses yielded higher completion rates for physician-prompted critical tasks (96.0%) compared with radio/.48%, p < 0.05) and were associated with more physician prompts overall (2.1 vs. 1.6 per session). CONCLUSION: Smart glasses have the potential to enhance EMS-OLMC communication and improve completion of physician-guided critical tasks, though these benefits come with longer interactions. These findings highlight important workflow considerations for integrating smart glasses into EMS practice.
Patient-centered design requirements for a virtual reality supportive care system in lung cancer: a qualitative study.
ElKefi S, Wallen J
Int J Med Inform · 2026 Jun · PMID 42361438 · Publisher ↗
PURPOSE: This study aimed to characterize lung cancer patients' preferences for virtual reality (VR) as a supportive care and educational tool, focusing on desired content, format, and delivery features. METHODS: We cond...
PURPOSE: This study aimed to characterize lung cancer patients' preferences for virtual reality (VR) as a supportive care and educational tool, focusing on desired content, format, and delivery features. METHODS: We conducted a qualitative study with 20 lung cancer patients (≥18 years; diagnosed within the past three years; initiated treatment). Semi-structured interviews were audio-recorded, transcribed verbatim, and analyzed using thematic analysis with an iterative inductive-deductive approach. Themes were organized into three pre-specified domains relevant to intervention design: content, format, and delivery. Descriptive statistics summarized participant characteristics. KEY FINDINGS: Patients viewed VR as most valuable when it leveraged immersion to reduce anxiety and enhance engagement rather than replicating conventional education. Preferred content included tailored, diagnosis-specific education, treatment rationale and expectations, symptom and side-effect management, supportive and palliative care, and resource navigation, with optional peer content. Format preferences emphasized immersive, calming environments, audio-video delivery with minimal text, interactivity, and credible human presenters. Delivery preferences favored home use with optional clinic onboarding, short sessions (15-30 min), flexible frequency, and opt-in caregiver involvement. IMPLICATIONS: These findings suggest that future lung cancer VR systems may benefit from flexible, modular, and patient-controlled designs that prioritize immersion, personalization, credibility, and autonomy. Prototype usability testing and feasibility studies are needed to evaluate how these patient-informed design requirements perform in clinical and home-based settings.
Evaluation of the ROX index for predicting intubation in ICU patients.
Angelucci A, Aliverti A, Pradella I … +5 more , Reni C, Roncalli G, Tacconelli M, Cecconi M, Greco M
Int J Med Inform · 2026 Jun · PMID 42361437 · Publisher ↗
BACKGROUND: The ROX index was proposed as a decision-support tool to assess the effectiveness of high-flow oxygen therapy (HFOT) in patients with acute hypoxemic respiratory failure (AHRF) affected by pneumonia. OBJECTIV...
BACKGROUND: The ROX index was proposed as a decision-support tool to assess the effectiveness of high-flow oxygen therapy (HFOT) in patients with acute hypoxemic respiratory failure (AHRF) affected by pneumonia. OBJECTIVE: The purpose of this work was to assess the discriminative power of the ROX index across heterogeneous intensive care unit (ICU) populations. As a secondary, hypothesis-generating objective, we explored whether ROX-based risk stratification may provide a standardized reference for describing variability in observed intubation practices across datasets and centers. METHODS: Patients affected by AHRF and receiving oxygen support were identified from two large public ICU databases (MIMIC-IV and eICU). Oxygen support was stratified based on recorded flow rates, i.e., LPMO ≥ 6 for conventional oxygen therapy (COT) and ≥ 30 for HFOT. All AHRF patients were initially considered, regardless of the underlying pathology, with a subgroup analysis performed in patients with pneumonia. ROX index predictions were compared with actual intubation rates in different datasets, and alternative thresholds were explored using Youden's method. RESULTS: In the primary three-category analysis, ROX risk strata produced only modest likelihood ratios (LRs) for observed intubation. In the merged cohorts, high-risk ROX categories showed LR values ranging from 1.36 to 2.06, whereas low-risk categories showed LR values close to 1, ranging from 0.85 to 0.90. Binary cut-off analyses confirmed limited discrimination, with AUROC values between 0.56 and 0.64. CONCLUSIONS: When applied across heterogeneous real-world populations, the ROX index shows limited discriminative ability for predicting intubation and should not be used as a standalone decision tool. However, it may serve as a standardized reference to explore variability in intubation practices across centers, particularly in retrospective analyses.
Clinical note comparison and data retrieval via embedding vectors: model selection, metrics, and convergence.
Dahlberg A, Tapiola O, Luisto R … +3 more , Puranen T, Sanmark E, Vartiainen V
Int J Med Inform · 2026 Jun · PMID 42349271 · Publisher ↗
BACKGROUND: Embedding models are an integral part of generative AI architectures, transforming text into embedding vectors that represent semantic content in numerical form. Despite their central role, their performance...
BACKGROUND: Embedding models are an integral part of generative AI architectures, transforming text into embedding vectors that represent semantic content in numerical form. Despite their central role, their performance in clinical settings remains underexplored. We evaluated embedding models across two tasks: semantic difference detection in clinical notes, and data retrieval from patient records. METHODS: Eight models were applied to synthetic discharge summaries in English, Swedish, and Finnish. Semantic sensitivity was assessed by introducing controlled perturbations (deletion, modification, and paraphrasing) at three levels of severity; cosine similarity, L and Euclidean distances were computed between the vectors of the original and perturbed texts. Partial vectors were compared to explore dimensionality reduction. Two models with the biggest contrast in semantic difference detection were evaluated on retrieval of relevant information from real Finnish vascular surgery records. RESULTS: Embedding vectors captured semantic differences in clinical notes: content deletion and modification produced larger increases in vector distance than paraphrasing. On average, models detected the direction of semantic change correctly, but case-level performance varied considerably. Qwen3-Embedding-8B produced no case-level (directional) errors whereas multilingual-E5-large produced them the most (12.2%). In retrieval this contrast transferred only partially and was task-dependent: sufficiency scores favoured Qwen3-Embedding-8B for the vascular-diagnosis question (2.25 vs 1.15 out of 5) but were comparable for the antithrombotic-medication question (3.25 vs 3.17 out of 5). For some models, as few as 0.6-1.2% of dimensions sufficed to replicate full-vector accuracy; principal component analysis and coordinate-level analysis did not account for this finding. CONCLUSIONS: Our results show that the choice of embedding model is important: performance differences between models can be large enough to determine whether clinically relevant information reaches the end user, and model weaknesses can be both task-specific and context-dependent.
From internal accuracy to clinical readiness: a systematic review of artificial intelligence-based mental health studies in Southeast Asia.
Hidayat EY, Muda AK
Int J Med Inform · 2026 Jun · PMID 42349270 · Publisher ↗
BACKGROUND: Artificial intelligence (AI) is increasingly applied to mental health research, but internal performance does not establish clinical or public health readiness. OBJECTIVE: To synthesise this regional AI liter...
BACKGROUND: Artificial intelligence (AI) is increasingly applied to mental health research, but internal performance does not establish clinical or public health readiness. OBJECTIVE: To synthesise this regional AI literature and assess validation, reporting transparency, reproducibility, implementation-readiness, and clinical readiness. METHODS: Scopus, PubMed, and IEEE Xplore were searched from inception to March 26, 2026. Eligible records were English-language peer-reviewed articles or conference proceedings applying AI, machine learning, or deep learning to mental health or psychiatric outcomes. RESULTS: Ninety-nine studies were included. Evidence was concentrated in Indonesia, Malaysia, and Thailand, and more than half addressed depression-spectrum conditions. Studies mainly used questionnaires, clinical-tabular data, social media, or digital-trace data for classification, detection, or severity stratification. Validation was almost entirely internal; no study reported external or independent validation. Calibration, uncertainty, error analysis, fairness or bias assessment, code or data availability, and implementation context were rarely reported. No study met the criteria for high clinical readiness. CONCLUSIONS: The regional evidence base is expanding, but remains concentrated at the model-development and internal-validation stages. Future work should prioritise representative datasets, transparent reporting, privacy-preserving external validation across settings and languages, and evaluation linked to mental health services or public health workflows. The proposed clinical translation roadmap supports the development of reproducible, workflow-aligned, and clinically meaningful AI applications.
Knowledge-guided counterfactual explanations for diabetes risk decision support: A directional intervention taxonomy.
Thieu V
Int J Med Inform · 2026 Jun · PMID 42349269 · Publisher ↗
BACKGROUND: Counterfactual (CF) explanations help patients and clinicians understand which features could be changed to lower a predicted diabetes-risk score, supporting risk communication in ML-based decision support. E...
BACKGROUND: Counterfactual (CF) explanations help patients and clinicians understand which features could be changed to lower a predicted diabetes-risk score, supporting risk communication in ML-based decision support. Existing CF methods, however, distinguish features only as mutable or immutable; they do not encode the direction of clinically appropriate intervention (for example, physical activity should always be recommended to increase, never to decrease), and observational health-survey data such as BRFSS contain selection biases that CF generators can inadvertently exploit, producing guideline-inconsistent recommendations such as "reduce healthcare access to lower predicted diabetes risk". OBJECTIVES: To produce counterfactual explanations that clinicians can review as candidate intervention recommendations aligned with clinical guidelines, by encoding clinical domain knowledge as a per-query constraint layer over an off-the-shelf CF generator. The approach is a practical alternative to structural causal modelling when SCMs are not identifiable from observational health-survey data. METHODS: A five-class intervention-direction taxonomy (immutable, monotonic_up, monotonic_down, bidirectional, conditional) is encoded as a lightweight per-feature knowledge tuple and translated per query by a rule-based reasoner into the DiCE constraint API (features_to_vary, permitted_range). Evaluation uses BRFSS 2021 (n=236,378) with an XGBoost classifier (test AUC =0.8233), 200 high-risk patients, six ablations, and external validation on BRFSS 2015 (n=253,680). RESULTS: On a taxonomy-consistency measure of actionability, the per-query mode raises the score from 0.6655 to 0.9880 (+48.5% relative) by eliminating wrong-direction and immutable violations as defined by the taxonomy; validity, an independent metric defined on the classifier rather than on the taxonomy, also improves from 0.752 to 0.808 (+7.5% relative). A per-feature breakdown attributes approximately 64% of the eliminated wrong-direction violations to suppression of three healthcare-utilisation indicators and behavioural self-report features, and the remaining 36% to directional redirection. External validation on BRFSS 2015 confirms both classifier transfer (AUC 0.827, Brier 0.0975) and per-query mechanism transfer (wrong-direction violations 0.000, validity 0.829, validity gain +0.031). CONCLUSIONS: A clinical intervention taxonomy translated per query into the constraint interface of an off-the-shelf CF generator closes a substantial fraction of the actionability gap in current CF tooling and makes generated counterfactuals reviewable by clinicians as candidate intervention recommendations consistent with clinical guideline directions. The contribution is methodological, and the actionability score is reported as a taxonomy-consistency proxy rather than a clinically validated endpoint. The counterfactuals are intended as decision-support candidates for clinician review, not as personalised medical advice. Implications are discussed for future clinician-facing risk-communication tools and EHR-integrated decision-support research in chronic-disease screening and prevention, subject to clinician review and deployment-population validation.
About
- Frequency
- Sun
- Papers found
- 200
- RSS feed
- Subscribe