Document Type
Article
Publication Date
2-2-2026
Publication Title
npj Health Systems
Volume
3
Pages
15
Publisher Name
Springer Nature
Publisher Location
London, United Kingdom
Abstract
Intensive care units (ICU) produce numerous progress notes that may contain stigmatizing language that perpetuate negative biases and punitive approaches against patients. Patients with substance use disorders are particularly vulnerable to stigma. This study examined the performance of Large Language Models (LLMs) in the identification of stigmatizing language. We annotated a dataset with over 77,000 stigmatizing and non-stigmatizing notes from the MIMIC-III database. We utilized Meta's Llama-3 8B Instruct LLM to run the following experiments for stigma detection: zero-shot; in-context learning; in-context learning with a selective retrieval; supervised fine-tuning (SFT); and keyword search. All approaches were evaluated on a held-out test set and external validation (University of Wisconsin Health System). SFT had the best performance with 97.2% accuracy, followed by in-context learning. The LLMs with in-context learning and SFT provided appropriate reasoning for false positives during human review. Both approaches identified clinical notes with stigmatizing language that were missed during annotation. SFT achieved 97.9% accuracy on external validation dataset. LLMs, particularly SFT and in-context learning, effectively identify stigmatizing language in ICU notes with high accuracy while explaining their reasoning in an asynchronous fashion and demonstrated the ability to identify novel stigmatizing language, not explicitly in training data nor existing guidelines.
Recommended Citation
Sethi, R., Caskey, J., Gao, Y., Churpek, M. M., Miller, T. A., Mayampurath, A., Salisbury-Afshar, E., Afshar, M., & Dligach, D. (2026). Detecting Stigmatizing Language in Clinical Notes with Large Language Models for Addiction Care. npj Health Systems, 3, 15. https://doi.org/10.1038/s44401-026-00069-0
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.
Copyright Statement
© Springer Nature, 2026
Included in
Biomedical Informatics Commons, Computer Sciences Commons, Health Information Technology Commons
Comments
Author Posting © The Author(s), 2026. This article is posted here by permission of Springer Nature for personal use and non-commercial redistribution. This article was published open access in npj Health Systems, Vol. 3, Article 15, (February 2026), https://doi.org/10.1038/s44401-026-00069-0.
Open access article. Published as Article number 15 in npj Health Systems, Volume 3, 2026. Funded by NIH R01LM012973, R01DA051464, and PCORI ME-2024C2-37484.