• Home
  • Search
  • Generating contextual embeddings for emergency department chief complaints.
  • Cite Icon23
  • https://doi.org/10.1093/jamiaopen/ooaa022Copy DOI Icon

Generating contextual embeddings for emergency department chief complaints.

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

ObjectiveWe learn contextual embeddings for emergency department (ED) chief complaints using Bidirectional Encoder Representations from Transformers (BERT), a state-of-the-art language model, to derive a compact and computationally useful representation for free-text chief complaints.Materials and methodsRetrospective data on 2.1 million adult and pediatric ED visits was obtained from a large healthcare system covering the period of March 2013 to July 2019. A total of 355 497 (16.4%) visits from 65 737 (8.9%) patients were removed for absence of either a structured or unstructured chief complaint. To ensure adequate training set size, chief complaint labels that comprised less than 0.01%, or 1 in 10 000, of all visits were excluded. The cutoff threshold was incremented on a log scale to create seven datasets of decreasing sparsity. The classification task was to predict the provider-assigned label from the free-text chief complaint using BERT, with Long Short-Term Memory (LSTM) and Embeddings from Language Models (ELMo) as baselines. Performance was measured as the Top-k accuracy from k = 1:5 on a hold-out test set comprising 5% of the samples. The embedding for each free-text chief complaint was extracted as the final 768-dimensional layer of the BERT model and visualized using t-distributed stochastic neighbor embedding (t-SNE).ResultsThe models achieved increasing performance with datasets of decreasing sparsity, with BERT outperforming both LSTM and ELMo. The BERT model yielded Top-1 accuracies of 0.65 and 0.69, Top-3 accuracies of 0.87 and 0.90, and Top-5 accuracies of 0.92 and 0.94 on datasets comprised of 434 and 188 labels, respectively. Visualization using t-SNE mapped the learned embeddings in a clinically meaningful way, with related concepts embedded close to each other and broader types of chief complaints clustered together.DiscussionDespite the inherent noise in the chief complaint label space, the model was able to learn a rich representation of chief complaints and generate reasonable predictions of their labels. The learned embeddings accurately predict provider-assigned chief complaint labels and map semantically similar chief complaints to nearby points in vector space.ConclusionSuch a model may be used to automatically map free-text chief complaints to structured fields and to assist the development of a standardized, data-driven ontology of chief complaints for healthcare institutions.

Similar Papers
  • Conference Article
  • Citations3

Engineering Document Summarization Using Sentence Representations Generated by Bidirectional Language Model

  • Aug 17, 2021
  • Yunjian Qiu +1
  • Research Article
  • Citations20

Oversampling effect in pretraining for bidirectional encoder representations from transformers (BERT) to localize medical BERT and enhance biomedical BERT

  • May 05, 2024
  • Artificial Intelligence In Medicine
  • Shoya Wada +6
  • Research Article

Cybershield Monitor Using Deep Learning and BERT Model

  • Feb 24, 2025
  • INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT
  • A Nandhini
  • Research Article
  • Citations4

Artificial Intelligence for CT and MRI Protocoling: A Meta-Analysis of Traditional Machine Learning, BERT, and Large Language Models.

  • Oct 29, 2025
  • AJR. American journal of roentgenology
  • Ethan Sacoransky +2
  • Research Article

Spam Detection Based on BERT Model: A Descriptive Study

  • Jan 01, 2023
  • Theory and practice of science and technology
  • Hailin Xu
  • PDF
  • Research Article
  • Citations25

Identification of asthma control factor in clinical notes using a hybrid deep learning model

  • Nov 01, 2021
  • BMC Medical Informatics and Decision Making
  • Bhavani Singh Agnikula Kshatriya +6
  • Research Article
  • Citations38

Modified Bidirectional Encoder Representations From Transformers Extractive Summarization Model for Hospital Information Systems Based on Character-Level Tokens (AlphaBERT): Development and Performance Evaluation

  • Apr 29, 2020
  • JMIR Medical Informatics
  • Yen-Pin Chen +4
  • Research Article

Sentence Vectors Enable GRU And Multi-Headed Attention-Based Video Captioning Framework For Specifying Events In MSVD Video Dataset

  • Jan 10, 2026
  • INTERNATIONAL JOURNAL OF ADVANCES IN SIGNAL AND IMAGE SCIENCES
  • K Annapoorneshwari Shetty +1
  • Research Article

T-­BERT:臺灣語言模型–以臺灣在地語言預訓練BERT模型

  • Jan 01, 2020
  • 鍾明諺
  • Abstract

77 Results of a Novel National Emergency Department Chief Complaint Database

  • Sep 29, 2022
  • Annals of Emergency Medicine
  • J Seidenfeld +3
  • Abstract
  • Citations5

Bert model fine-tuning for text classification in knee OA radiology reports

  • Apr 01, 2020
  • Osteoarthritis and Cartilage
  • L Chen +5
  • Research Article

RESEARCH OF THE PROCESS OF VISUAL ART TRANSMISSION IN MUSIC AND THE CREATION OF COLLECTIONS FOR PEOPLE WITH VISUAL IMPAIRMENTS

  • Apr 02, 2025
  • Municipal economy of cities
  • I Karavan +4
  • Conference Article

Predictive Sentiment Analysis of Skincare Product Reviews: A Natural Language Processing Approach with BERT-Enhanced Sequential Model (BESM)

  • Dec 01, 2025
  • Sky Nurimba +4
  • Research Article
  • Citations1

Natural language processing models for predicting treatment outcomes in internet-delivered cognitive behavioral therapy

  • Oct 10, 2025
  • Internet Interventions
  • Nils Hentati Isacsson +4
  • Research Article
  • Citations9

Multifaceted Natural Language Processing Task-Based Evaluation of Bidirectional Encoder Representations From Transformers Models for Bilingual (Korean and English) Clinical Notes: Algorithm Development and Validation.

  • Oct 30, 2024
  • JMIR medical informatics
  • Kyungmo Kim +11
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.