Document Type
Article
Publication Date
4-29-2020
Publication Title
BMC Medical Informatics and Decision Making
Volume
20
Pages
1-11
Publisher Name
BioMed Central Ltd
Abstract
Background: Automated de-identification methods for removing protected health information (PHI) from the source notes of the electronic health record (EHR) rely on building systems to recognize mentions of PHI in text, but they remain inadequate at ensuring perfect PHI removal. As an alternative to relying on de-identification systems, we propose the following solutions: (1) Mapping the corpus of documents to standardized medical vocabulary (concept unique identifier [CUI] codes mapped from the Unified Medical Language System) thus eliminating PHI as inputs to a machine learning model; and (2) training character-based machine learning models that obviate the need for a dictionary containing input words/n-grams. We aim to test the performance of models with and without PHI in a use-case for an opioid misuse classifier.
Methods: An observational cohort sampled from adult hospital inpatient encounters at a health system between 2007 and 2017. A case-control stratified sampling (n = 1000) was performed to build an annotated dataset for a reference standard of cases and non-cases of opioid misuse. Models for training and testing included CUI codes, character-based, and n-gram features. Models applied were machine learning with neural network and logistic regression as well as expert consensus with a rule-based model for opioid misuse. The area under the receiver operating characteristic curves (AUROC) were compared between models for discrimination. The HosmerLemeshow test and visual plots measured model fit and calibration.
Results: Machine learning models with CUI codes performed similarly to n-gram models with PHI. The top performing models with AUROCs > 0.90 included CUI codes as inputs to a convolutional neural network, max pooling network, and logistic regression model. The top calibrated models with the best model fit were the CUIbased convolutional neural network and max pooling network. The top weighted CUI codes in logistic regression has the related terms ‘Heroin’ and ‘Victim of abuse’.
Conclusions: We demonstrate good test characteristics for an opioid misuse computable phenotype that is void of any PHI and performs similarly to models that use PHI. Herein we share a PHI-free, trained opioid misuse classifier for other researchers and health systems to use and benchmark to overcome privacy and security concerns.
Recommended Citation
Sharma, Brihat; Dligach, Dmitriy; Swope, Kristin; Salisbury-Afshar, Elizabeth; Karnik, Niranjan S.; Joyce, Cara; and Afshar, Majid, "Publicly available machine learning models for identifying opioid misuse from the clinical notes of hospitalized patients" (2020). Biostatistics Collaborative Core: Faculty Publications and Other Works. 40.
https://ecommons.luc.edu/biostatistics_facpubs/40
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright Statement
© The Author(s), 2020.
Comments
Author Posting © The Author(s), 2020. This article is posted here by permission of BioMed Central for personal use and redistribution. This article was published open access in BMC Medical Informatics and Decision Making, Vol. 20 (April 29, 2020), https://doi.org/10.1186/s12911-020-1099-y.