Identifying Symptom Information in Clinical Notes Using Natural Language Processing.

BackgroundPatient portals are widely adopted in the United States and allow millions of patients access to their electronic health records (EHRs), including their EHR clinical notes. A patient’s ability to understand the information in the EHR is dependent on their overall health literacy. Although many tests of health literacy exist, none specifically focuses on EHR note comprehension.ObjectiveThe aim of this paper was to develop an instrument to assess patients’ EHR note comprehension.MethodsWe identified 6 common diseases or conditions (heart failure, diabetes, cancer, hypertension, chronic obstructive pulmonary disease, and liver failure) and selected 5 representative EHR notes for each disease or condition. One note that did not contain natural language text was removed. Questions were generated from these notes using Sentence Verification Technique and were analyzed using item response theory (IRT) to identify a set of questions that represent a good test of ability for EHR note comprehension.ResultsUsing Sentence Verification Technique, 154 questions were generated from the 29 EHR notes initially obtained. Of these, 83 were manually selected for inclusion in the Amazon Mechanical Turk crowdsourcing tasks and 55 were ultimately retained following IRT analysis. A follow-up validation with a second Amazon Mechanical Turk task and IRT analysis confirmed that the 55 questions test a latent ability dimension for EHR note comprehension. A short test of 14 items was created along with the 55-item test.ConclusionsWe developed ComprehENotes, an instrument for assessing EHR note comprehension from existing EHR notes, gathered responses using crowdsourcing, and used IRT to analyze those responses, thus resulting in a set of questions to measure EHR note comprehension. Crowdsourced responses from Amazon Mechanical Turk can be used to estimate item parameters and select a subset of items for inclusion in the test set using IRT. The final set of questions is the first test of EHR note comprehension.

Objectives: A controlled vocabulary for chief complaint (CC) will facilitate clinical guidelines, emergency department (ED) surveillance, and third-party claims review. In an earlier pilot study, automated efforts to develop a CC vocabulary were conducted using natural language processing (NLP) of free text ED CC terms. The resulting standardized terms were then compared to the controlled vocabularies in the Unified Medical Language System (UMLS); only 14% of the ED CC free text terms matched a UMLS concept. The objective of this study was to apply and evaluate a combination of automated and manual methods for processing ED CC terms, in order to improve the match rate with UMLS concepts. Methods: In this retrospective secondary data analysis, we used corpus linguistics methods to collocate free text CC data from 3 university tertiary referral center EDs (rural, urban, and suburban) for all ED visits during 1/01. First, we evaluated how many CC terms matched a UMLS concept prior to any processing. The terms that did not match a UMLS concept were then manually examined by the investigator, who used domain knowledge to identify patterns in the data. NLP techniques were then applied, moving from simple to more aggressive techniques. The resulting CC terms were again compared to the UMLS to determine the match rate with standard concepts. Results: There were 6,900 visits, and 6,054 unique CC terms recorded during the 1/01. Prior to any processing, 22% of the terms matched a UMLS concept. 3 patterns in the data were identified: punctuation (N/V/D), acronyms (BRBPR), and modifiers (mild/moderate/severe). After addressing the patterns with NLP techniques, 35% of the remaining CC terms then matched a UMLS concept. Conclusions: We increased the match rate of CC terms with UMLS concepts with a combination of manual and automated techniques. Additional review by domain experts and further NLP are needed to identify concepts for the nonmatched ED CC terms. This study provides the foundation for a controlled ED CC vocabulary.

BackgroundThe Unified Medical Language System (UMLS) has been a critical tool in biomedical and health informatics, and the year 2021 marks its 30th anniversary. The UMLS brings together many broadly used vocabularies and standards in the biomedical field to facilitate interoperability among different computer systems and applications.ObjectiveDespite its longevity, there is no comprehensive publication analysis of the use of the UMLS. Thus, this review and analysis is conducted to provide an overview of the UMLS and its use in English-language peer-reviewed publications, with the objective of providing a comprehensive understanding of how the UMLS has been used in English-language peer-reviewed publications over the last 30 years.MethodsPubMed, ACM Digital Library, and the Nursing & Allied Health Database were used to search for studies. The primary search strategy was as follows: UMLS was used as a Medical Subject Headings term or a keyword or appeared in the title or abstract. Only English-language publications were considered. The publications were screened first, then coded and categorized iteratively, following the grounded theory. The review process followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.ResultsA total of 943 publications were included in the final analysis. Moreover, 32 publications were categorized into 2 categories; hence the total number of publications before duplicates are removed is 975. After analysis and categorization of the publications, UMLS was found to be used in the following emerging themes or areas (the number of publications and their respective percentages are given in parentheses): natural language processing (230/975, 23.6%), information retrieval (125/975, 12.8%), terminology study (90/975, 9.2%), ontology and modeling (80/975, 8.2%), medical subdomains (76/975, 7.8%), other language studies (53/975, 5.4%), artificial intelligence tools and applications (46/975, 4.7%), patient care (35/975, 3.6%), data mining and knowledge discovery (25/975, 2.6%), medical education (20/975, 2.1%), degree-related theses (13/975, 1.3%), digital library (5/975, 0.5%), and the UMLS itself (150/975, 15.4%), as well as the UMLS for other purposes (27/975, 2.8%).ConclusionsThe UMLS has been used successfully in patient care, medical education, digital libraries, and software development, as originally planned, as well as in degree-related theses, the building of artificial intelligence tools, data mining and knowledge discovery, foundational work in methodology, and middle layers that may lead to advanced products. Natural language processing, the UMLS itself, and information retrieval are the 3 most common themes that emerged among the included publications. The results, although largely related to academia, demonstrate that UMLS achieves its intended uses successfully, in addition to achieving uses broadly beyond its original intentions.

Most providers have experienced increased documentation demands with the use of electronic health records (EHRs). We sought to identify efficiency strategies that providers use to complete clinical documentation tasks in ambulatory care. Two observers performed ethnographic observations and interviews with 22 ambulatory care providers in a U.S. Veterans Affairs Medical Center. Observation notes and interview transcripts were coded for recurrent strategies relating to completion of the EHR progress notes. Findings included: the use of paper artifacts for handwritten notations; electronic templates for automation of certain parts of the note; use of shorthand and phrases rather than narrative writing; copying and pasting from previous EHR notes; directly entering information into the EHR note during the patient encounter; reliance on memory; and pre-populating an EHR note prior to seeing the patient. We discuss the findings in the context of distributed cognition to understand how clinical information is propagated and represented toward completion of a progress note. The study findings have important implications for improving and streamlining clinical documentation related to human factors workload management strategies.

BackgroundHeadache frequency, defined as the number of days with any headache in a month (or four weeks), remains a key parameter in the evaluation of treatment response to migraine preventive medications. However, due to the variations and inconsistencies in documentation by clinicians, significant challenges exist to accurately extract headache frequency from the electronic health record (EHR) by traditional natural language processing (NLP) algorithms.MethodsThis was a retrospective cross-sectional study with human subjects identified from three tertiary headache referral centers-Mayo Clinic Arizona, Florida, and Rochester. All neurology consultation notes written by more than 10 headache specialists between 2012 to 2022 were extracted and 1915 notes were used for model fine-tuning (90%) and testing (10%). We employed four different NLP frameworks: (1) ClinicalBERT (Bidirectional Encoder Representations from Transformers) regression model (2) Generative Pre-Trained Transformer-2 (GPT-2) Question Answering (QA) Model zero-shot (3) GPT-2 QA model few-shot training fine-tuned on Mayo Clinic notes; and (4) GPT-2 generative model few-shot training fine-tuned on Mayo Clinic notes to generate the answer by considering the context of included text.ResultsThe GPT-2 generative model was the best-performing model with an accuracy of 0.92[0.91 – 0.93] and R2 score of 0.89[0.87, 0.9], and all GPT2-based models outperformed the ClinicalBERT model in terms of the exact matching accuracy. Although the ClinicalBERT regression model had the lowest accuracy 0.27[0.26 – 0.28], it demonstrated a high R2 score 0.88[0.85, 0.89], suggesting the ClinicalBERT model can reasonably predict the headache frequency within a range of ≤ ± 3 days, and the R2 score was higher than the GPT-2 QA zero-shot model or GPT-2 QA model few-shot training fine-tuned model.ConclusionWe developed a robust model based on a state-of-the-art large language model (LLM)-a GPT-2 generative model that can extract headache frequency from EHR free-text clinical notes with high accuracy and R2 score. It overcame several challenges related to different ways clinicians document headache frequency that were not easily achieved by traditional NLP models. We also showed that GPT2-based frameworks outperformed ClinicalBERT in terms of accuracy in extracting headache frequency from clinical notes. To facilitate research in the field, we released the GPT-2 generative model and inference code with open-source license of community use in GitHub.

The automated processing of Electronic Health Records (EHRs) poses a significant challenge due to their unstructured nature, rich in valuable, yet disorganized information. Natural Language Processing (NLP), particularly Named Entity Recognition (NER), has been instrumental in extracting structured information from EHR data. However, existing literature primarly focuses on extracting handcrafted clinical features through NLP and NER methods without delving into their learned representations. In this work, we explore the untapped potential of these representations by considering their contextual richness and entity-specific information. Our proposed methodology extracts representations generated by a transformer-based NER model on EHRs data, combines them using a hierarchical attention mechanism, and employs the obtained enriched representation as input for a clinical prediction model. Specifically, this study addresses Overall Survival (OS) in Non-Small Cell Lung Cancer (NSCLC) using unstructured EHRs data collected from an Italian clinical centre encompassing 838 records from 231 lung cancer patients. Whilst our study is applied on EHRs written in Italian, it serves as use case to prove the effectiveness of extracting and employing high level textual representations that capture relevant information as named entities. Our methodology is interpretable because the hierarchical attention mechanism highlights the information in EHRs that the model considers the most crucial during the decision-making process. We validated this interpretability by measuring the agreement of domain experts on the importance assigned by the hierarchical attention mechanism to EHRs information through a questionnaire. Results demonstrate the effectiveness of our method, showcasing statistically significant improvements over traditional manually extracted clinical features.

Health information technology has been used in long-term care to improve outcomes and reduce cost. In Tiger Pace, an aging in place facility from Columbia, MO, we deployed sensor networks together with an electronic health record (EHR) system to provide early illness recognition. In this paper, we describe a methodology for early illness based on non-wearable sensor data and concepts extracted from nursing notes using Natural Language Processing (NLP). The methodology is inspired from genomie sequence annotation using BLAST. First, we extract a set of Unified Medical Language System (UMLS) concepts from each nursing note using Metamap, a NLP tool provided by UMLS. Then, we associate each daily sensor sequence with the medical concepts related to the nursing notes issued that day for that patient. Finally, to infer the health concepts for an unknown day, we compute the similarity between its sensor sequence and those available in the database. The challenges presented by this method are finding the most suitable multi-attribute time sequence similarity and aggregation of the retrieved concepts. On a pilot dataset from three Tiger Place residents, with a total of 1685 sensor days and 358 nursing records, we obtained an average precision of 0.34 and a recall of 0.52.

As more data is generated from medical attendances and as Artificial Neural Networks gain momentum in research and industry, computer-aided medical prognosis has become a promising technology. A common approach to perform automated prognoses relies on textual clinical notes extracted from Electronic Health Records (EHRs). Data from EHRs are fed to neural networks that produce a set with the most probable medical problems to which a patient is subject in her/his clinical future, including clinical conditions, mortality, and readmission. Following this research line, we introduce a methodology that takes advantage of the unstructured text found in clinical notes by applying preprocessing, concepts extraction, and fine-tuned neural networks to predict the most probable medical problems to follow in a patient's clinical trajectory. Different from former works that focus on word embeddings and raw sets of extracted concepts, we generate a refined set of Unified Medical Language System (UMLS) concepts by applying a similarity threshold filter and a list of acceptable concept types. In our prediction experiments, our method demonstrated AUC-ROC performance of 0.91 for diagnosis codes, 0.93 for mortality, and 0.72 for readmission, determining an efficacy that rivals state-of-the-art works. Our findings contribute to the development of automated prognosis systems in hospitals where text is the main source of clinical history.

Cancer is in the midst of leading causes of death. In 2018, around 1,735,350 new cases of cancer were estimated and 609,640 people will die from cancer in the United States. A wealth of cancer-relevant information is conserved in a variety of types of healthcare records, for example, the electronic health records (EHRs). However, part of the critical information is organized in the free narrative text which hampers machine to interpret the information underlying the text. The development of artificial intelligence provides a variety of solutions to this plight. For example, the technology of natural language processing (NLP) has emerged bridging the gap between free text and structured representation of cancer information. Recently, several researchers have published their work on unearthing cancer-related information in EHRs based on the NLP technology. Apart from the traditional NLP methods, the development of deep learning helps EHRs mining go further.

Financial stress, one of the social determinants, is common among cancer patients because of high out-ofpocket costs for treatment, as well as indirect costs. The National Academy of Medicine (NAM) has advised providers to recognize and discuss cost concerns with patients in order to enhance shared decision-making for treatment and exploration of financial assistant programs. However, financial stress is rarely assessed in clinical practice or research, thus, under-coded and under-documented in clinical practice. Natural language processing (NLP) offers great potential that can automatically extract and process data on financial stress from clinical free text existing in the patient electronic health record (EHR). We developed and evaluated an NLP approach to identify financial stress from clinical narratives for patients with prostate cancer. Of 4,195 eligible prostate cancer patients, we randomly sampled 3,138 patients (75%) as a training dataset (150,990 documents) to develop a financial stress lexicon and NLP algorithms iteratively. The remaining 1,057 patients (25%) were used as a test dataset (55,516 documents) to evaluate the NLP algorithm performance. The common terms representing financial stress were "financial concerns," "unable to afford," "insurance issue," "unemployed," and "financial assistance." Negations were used to exclude false mentions of financial stress. Applying both pre- and post-negation, the NLP algorithm identified 209 patients (6.0%) from the training sample and 66 patients (6.2%) with 161 notes from the test sample as having documented financial stress. Two independent domain experts manually reviewed all 161 notes with NLP identified positives and randomly selected 161 notes with NLP-identified negatives, the NLP algorithm yielded 0.86 for precision, 1 for recall, and 0.9.2 for F-score. Financial stress information is not commonly documented in the EHR, neither in structured format nor in clinical narratives. However, natural language processing can accurately extract financial stress information from clinical notes when such narrative information is available.

77 Results of a Novel National Emergency Department Chief Complaint Database

Researchers analyze data, information, and knowledge through many sources, formats, and methods. The dominant data format includes text and images. In the healthcare industry, professionals generate a large quantity of unstructured data. The complexity of this data and the lack of computational power causes delays in analysis. However, with emerging deep learning algorithms and access to computational powers such as graphics processing unit (GPU) and tensor processing units (TPUs), processing text and images is becoming more accessible. Deep learning algorithms achieve remarkable results in natural language processing (NLP) and computer vision. In this study, we focus on NLP in the healthcare industry and collect data not only from electronic medical records (EMRs) but also medical literature and social media. We propose a framework for linking social media, medical literature, and EMRs clinical notes using deep learning algorithms. Connecting data sources requires defining a link between them, and our key is finding concepts in the medical text. The National Library of Medicine (NLM) introduces a Unified Medical Language System (UMLS) and we use this system as the foundation of our own system. We recognize social media’s dynamic nature and apply supervised and semi-supervised methodologies to generate concepts. Named entity recognition (NER) allows efficient extraction of information, or entities, from medical literature, and we extend the model to process the EMRs’ clinical notes via transfer learning. The results include an integrated, end-to-end, web-based system solution that unifies social media, literature, and clinical notes, and improves access to medical knowledge for the public and experts.

Electronic Health Records (EHRs) capture all information regarding the treatment of a patient. In the case of patients with chronic diseases that needs periodic interdepartmental evaluations, the volume of EHR becomes very large and clinicians find it difficult to find all the required information within the constrained time. In such scenarios, automated EHR summarization can be very useful. Summary of EHR can help healthcare provider to see the most important clinical decisions and history details in brief. This will help in saving time, reducing medical errors like prescribing an existing medication and can help in better clinical decision making. This paper reviews the existing approaches for extractive summarization and proposes a technique to provide extractive summary for a single EHR note.

ObjectivesUrinary tract infection (UTI) is common in home care but not easily captured with standard assessment. This study aimed to examine the value of nursing notes in detecting UTI signs and symptoms in home care. DesignThe study developed a natural language processing (NLP) algorithm to automatically identify UTI-related information in nursing notes. Setting and ParticipantsHome care visit notes (n = 1,149,586) and care coordination notes (n = 1,461,171) for 89,459 patients treated in the largest nonprofit home care agency in the United States during 2014. MeasuresWe generated 6 categories of UTI-related information from literature and used the Unified Medical Language System (UMLS) to identify a preliminary list of terms. The NLP algorithm was tested on a gold standard set of 300 clinical notes annotated by clinical experts. We used structured Outcome and Assessment Information Set data to extract the frequency of UTI-related emergency department (ED) visits or hospitalizations and explored time-patterns in documentation of UTI-related information. ResultsThe NLP system achieved very good overall performance (F measure = 0.9, 95% CI: 0.87-0.93) based on the test results obtained by using the notes for patients admitted to the ED or hospital due to UTI. UTI-related information was significantly more prevalent (P < .01 for all the tests) in home care episodes with UTI-related ED admission or hospitalization vs the general patient population; 81% of home care episodes with UTI-related hospitalization or ED admission had at least 1 category of UTI-related information vs 21.6% among episodes without UTI-related hospitalization or ED admission. Frequency of UTI-related information documentation increased in advance of UTI-related hospitalization or ED admission, peaking within a few days before the event. Conclusions and ImplicationsInformation in nursing notes is often overlooked by stakeholders and not integrated into predictive modeling for decision-making support, but our findings highlight their value in early risk identification and care guidance. Health care administrators should consider using NLP to extract clinical data from nursing notes to improve early detection and treatment, which may lead to quality improvement and cost reduction.

In the last years, the need to de-identify privacy-sensitive information within Electronic Health Records (EHRs) has become increasingly felt and extremely relevant to encourage the sharing and publication of their content in accordance with the restrictions imposed by both national and supranational privacy authorities. In the field of Natural Language Processing (NLP), several deep learning techniques for Named Entity Recognition (NER) have been applied to face this issue, significantly improving the effectiveness in identifying sensitive information in EHRs written in English. However, the lack of data sets in other languages has strongly limited their applicability and performance evaluation. To this aim, a new de-identification data set in Italian has been developed in this work, starting from the 115 COVID-19 EHRs provided by the Italian Society of Radiology (SIRM): 65 were used for training and development, the remaining 50 were used for testing. The data set was labelled following the guidelines of the i2b2 2014 de-identification track. As additional contribution, combined with the best performing Bi-LSTM + CRF sequence labeling architecture, a stacked word representation form, not yet experimented for the Italian clinical de-identification scenario, has been tested, based both on a contextualized linguistic model to manage word polysemy and its morpho-syntactic variations and on sub-word embeddings to better capture latent syntactic and semantic similarities. Finally, other cutting-edge approaches were compared with the proposed model, which achieved the best performance highlighting the goodness of the promoted approach.