Evaluation of a Meta-1-based automatic indexing method for medical documents

Analyzing polysemous concepts from a clinical perspective: Application to auditing concept categorization in the UMLS

The Unified Medical Language System metathesaurus and medical semantic network (National Library of Medicine, Meta 1.3, 1993) currently contains 270,797 medical terms derived from multiple controlled vocabularies, including ICD-9, ACR, MeSH, CPT, and SNOMED. We briefly describe the structure and content of Meta 1.3, focusing on its potential value to the radiological community.

Efficient clinical information retrieval tasks like entity recognition, relation suggestions, summarization etc, require a comprehensive, extensive, unambiguous and well structured medical knowledge base. One of the largest Metathesaurus, UMLS (Unified Medical Language System), is a repository of biomedical dictionaries developed by the US National Library of Medicine (NLM) and widely used in medical domain. UMLS Metathesaurus includes the dictionaries like SNOMED-CT, National Center for Biotechnology Information (NCBI) taxonomy, Medical Subject Headings (MeSH), Online Mendelian Inheritance in Man (OMIM) etc. As it has integrated data from different sources, it contains different kinds of ambiguity, which is problematic for all clinical information retrieval tasks that use it. In this paper, we describe our methodology of curating the UMLS metathesaurus to create a centralized knowledge base that can be used as a knowledge base for a variety of clinical NLP systems. We have also developed a process of updating the curated centralized knowledge base with a newer version of UMLS such that there is no need to repeat the whole process. We have also presented the comparative results of a Clinical Entity Recognition (CER) using our curated centralized knowledge base and original UMLS database.

This paper describes the development and testing of the Medical Concept Mapper, a tool designed to facilitate access to online medical information sources by providing users with appropriate medical search terms for their personal queries. Our system is valuable for patients whose knowledge of medical vocabularies is inadequate to find the desired information, and for medical experts who search for information outside their field of expertise. The Medical Concept Mapper maps synonyms and semantically related concepts to a user's query. The system is unique because it integrates our natural language processing tool, i.e., the Arizona (AZ) Noun Phraser, with human-created ontologies, the Unified Medical Language System (UMLS) and WordNet, and our computer generated Concept Space, into one system. Our unique contribution results from combining the UMLS Semantic Net with Concept Space in our deep semantic parsing (DSP) algorithm. This algorithm establishes a medical query context based on the UMLS Semantic Net, which allows Concept Space terms to be filtered so as to isolate related terms relevant to the query. We performed two user studies in which Medical Concept Mapper terms were compared against human experts' terms. We conclude that the AZ Noun Phraser is well suited to extract medical phrases from user queries, that WordNet is not well suited to provide strictly medical synonyms, that the UMLS Metathesaurus is well suited to provide medical synonyms, and that Concept Space is well suited to provide related medical terms, especially when these terms are limited by our DSP algorithm.

Problem lists are tools to improve patient management. In the medical record, they connect diagnoses to therapy, prognosis, and psychosocial issues. Computer-based problem lists enhance paper-based approaches by enabling cost-containment and quality assurance applications, but they require clinically expressive controlled vocabularies. Because existing controlled vocabularies do not represent problem statements at a clinically useful level, we derived a new canonical problem statement vocabulary through semi-automated analysis and distillation of provider-entered problem lists collected over 6 years from 74,696 patients. We combined automated and manual methods to condense 891,770 problem statements entered by 1961 care providers at Grady Memorial Hospital in Atlanta, Georgia, to 15,534 Canonical Clinical Problem Statement System (CCPSS) terms. The nature and frequency of problem statements were characterized, interrelations among them were enumerated, and a database capturing the epidemiology of problems was created. The authors identified 23,503 problem relations (co-occurrences, sign-symptom complexes, and differential diagnoses) and 22,690 modifier words that further categorized "canonical" problems. To assess completeness, CCPSS content was compared with that of the 1997 Unified Medical Language System Metathesaurus (containing terms from 44 clinical vocabularies). Unified Medical Language System terms expressed 25% of individual CCPSS terms exactly (71% of problems by frequency), 27% partially, and 48% poorly or not at all. Clinicians judged that CCPSS terms completely captured their clinical intent for 84% of 686 randomly selected free-text problem statements. The CCPSS represents clinical concepts at a level exceeding that of previous approaches. A similar national approach could create a standardized, useful, shared resource for clinical practice.

We explored two strategies for query expansion utilizing medical subject headings (MeSH) ontology to improve the effectiveness of medical image retrieval systems. In order to achieve greater effectiveness in the expansion, the search text was analyzed to identify which terms were most amenable to being expanded. To perform the expansions we utilized the hierarchical structure by which the MeSH descriptors are organized. Two strategies for selecting the terms to be expanded in each query were studied. The first consisted of identifying the medical concepts using the unified medical language system metathesaurus. In the second strategy the text of the query was divided into n-grams, resulting in sequences corresponding to MeSH descriptors. For the evaluation of the system, we used the collection made available by the ImageCLEF organization in its 2011 medical image retrieval task. The main measure of efficiency employed for evaluating the techniques developed was the mean average precision (MAP). Both strategies exceeded the average MAP score in the ImageCLEF 2011 competition (0.1644). The n-gram expansion strategy achieved a MAP of 0.2004, which represents an improvement of 21.89% over the average MAP score in the competition. On the other hand, the medical concepts expansion strategy scored 0.2172 in the MAP, representing a 32.11% improvement. This run won the text-based medical image retrieval task in 2011. Query expansion exploiting the hierarchical structure of the MeSH descriptors achieved a significant improvement in image retrieval systems.

We have developed a semantic network of biological terminology to aid in the retrieval and integration of biological information from a variety of disparate information sources. Our semantic network strives to provide a categorization of biological concepts and relationships among these concepts. The semantic network will impart a knowledge structure through which computers can "reason " and draw conclusions about biological data objects and will provide a federated view of the many disparate databases of interest to biologists. In the development of our system, we have included the concepts from several established controlled vocabularies, chief among them being the National Library of Medicine's Unified Medical language System (UMLS). While the UMLS Metathesaurus provides an excellent controlled vocabulary, we have found their semantic network lacking in sufficient detail to be useful as a tool for categorization of biological concepts in databases. We would like to provide a categorization of concepts that provides finer detail than their semantic network without the considerable size and complexity of their Metathesaurus. Our complete semantic network consists of 183 semantic types and 69 relationships.

With the recent increase in the number of news videos and in their value, their automatic indexing has been anticipated. Among various proposals for this problem, the use of textual data has been actively considered, and some applications of this type have actually been developed. Nonetheless, most of these approaches are textual data oriented, and a few of them consider the correspondences between their indexes (keywords) and image contents which are essentials for an image database.This paper proposes an automatic video indexing method which takes advantage of the characteristic of news videos: similar images have similar contents. The method tags indexes selectively according to typical image content. The method has been applied to actual news videos, and indexing has been successful for between 25% and 93% of “typical shots,” and for between 75% and 100% of “captionized shots.” © 2001 Scripta Technica, Syst Comp Jpn, 32(9): 32–41, 2001

BackgroundHow to treat a disease remains to be the most common type of clinical question. Obtaining evidence-based answers from biomedical literature is difficult. Analogical reasoning with embeddings from deep learning (embedding analogies) may extract such biomedical facts, although the state-of-the-art focuses on pair-based proportional (pairwise) analogies such as man:woman::king:queen (“queen = −man +king +woman”).ObjectiveThis study aimed to systematically extract disease treatment statements with a Semantic Deep Learning (SemDeep) approach underpinned by prior knowledge and another type of 4-term analogy (other than pairwise).MethodsAs preliminaries, we investigated Continuous Bag-of-Words (CBOW) embedding analogies in a common-English corpus with five lines of text and observed a type of 4-term analogy (not pairwise) applying the 3CosAdd formula and relating the semantic fields person and death: “dagger = −Romeo +die +died” (search query: −Romeo +die +died). Our SemDeep approach worked with pre-existing items of knowledge (what is known) to make inferences sanctioned by a 4-term analogy (search query −x +z1 +z2) from CBOW and Skip-gram embeddings created with a PubMed systematic reviews subset (PMSB dataset). Stage1: Knowledge acquisition. Obtaining a set of terms, candidate y, from embeddings using vector arithmetic. Some n-gram pairs from the cosine and validated with evidence (prior knowledge) are the input for the 3cosAdd, seeking a type of 4-term analogy relating the semantic fields disease and treatment. Stage 2: Knowledge organization. Identification of candidates sanctioned by the analogy belonging to the semantic field treatment and mapping these candidates to unified medical language system Metathesaurus concepts with MetaMap. A concept pair is a brief disease treatment statement (biomedical fact). Stage 3: Knowledge validation. An evidence-based evaluation followed by human validation of biomedical facts potentially useful for clinicians.ResultsWe obtained 5352 n-gram pairs from 446 search queries by applying the 3CosAdd. The microaveraging performance of MetaMap for candidate y belonging to the semantic field treatment was F-measure=80.00% (precision=77.00%, recall=83.25%). We developed an empirical heuristic with some predictive power for clinical winners, that is, search queries bringing candidate y with evidence of a therapeutic intent for target disease x. The search queries -asthma +inhaled_corticosteroids +inhaled_corticosteroid and -epilepsy +valproate +antiepileptic_drug were clinical winners, finding eight evidence-based beneficial treatments.ConclusionsExtracting treatments with therapeutic intent by analogical reasoning from embeddings (423K n-grams from the PMSB dataset) is an ambitious goal. Our SemDeep approach is knowledge-based, underpinned by embedding analogies that exploit prior knowledge. Biomedical facts from embedding analogies (4-term type, not pairwise) are potentially useful for clinicians. The heuristic offers a practical way to discover beneficial treatments for well-known diseases. Learning from deep learning models does not require a massive amount of data. Embedding analogies are not limited to pairwise analogies; hence, analogical reasoning with embeddings is underexploited.

Significance. Early detection of axial spondyloarthritis (axSpA) is a complex clinical task. Quality improvement of axSpA diagnostics in primary care settings is possible with the help of decision-making information systems based on the ontological approach application. The key stage of the decision-making system development consists of the elaboration of a set of clinical terms. This set should fully describe clinical area or sub-area under study. One of the essential requirements is compliance of the clinical terms used with the existing clinical nomenclatures. Currently, the largest set of clinical terms is the Unified Medical Language System (UMLS) metathesaurus. The majority of UMLS terms is presented in English only. The development of tools for the analysis of unstructured texts and recognition of clinically relevant UMLS entities make it possible to elaborate a set of terms describing axSpA diagnostic aspects. This will also help to compile a list of UMLS terminology nomenclatures for their priority adaptation and expert translation into Russian. The purpose of this study is to develop an automated system for recognizing clinically relevant UMLS terms in texts of the English-language articles. Material and methods. The research material has included English terms (11.2 million) aggregated from 76 nome6nclatures of the current UMLS (2022AB) version. In addition, the study has used texts of PubMed clinical abstracts in English. Queries to the UMLS graph model, semantic algorithms for unstructured texts and machine-learning methods have been applied for data collection and analysis. Results. The study has elaborated a set of high-accuracy regular expressions (F1-score=98%) for metadata elimination from the text corpus. Then the authors have identified patterns for searching clinically relevant terms in the aggregated set of UMLS concepts. Using a logistic regression algorithm, the authors have trained a binary classification model. Input data for the created classificatory are information about an UMLS term. Output data are a label indicating the presence or absence of clinical relevance. Conclusion. The binary classification model has been validated individually and double-tested on different data samples. Values of the accuracy, sensitivity and specificity of metrics equal to 91%, 90% and 91%, respectively, for the validation sample (a number of axSpA terms). In addition, this model has been tested on the sets of terms aggregated for any two diseases. Values of the accuracy metric equal to 91% and 90%, respectively. With the help of the developed machine-learning model, the study has estimated that UMLS contains 1.5 million unique terms applicable to describing a clinical picture. In addition, lists of priority UMLS data sources and thematic groups have been compiled. These clinically relevant UMLS terms should be adapted and translated into Russian as soon as possible.

Modern healthcare interoperability demands objective methods for quantitatively evaluating the coverage and granularity of biomedical terminology systems to support evidence-based selection and integration decisions. We introduce novel metrics—structural size (an integrated measure of width and depth), mapping burden ratio (a measure of relative granularity between systems), and content overlap—to quantitatively evaluate the semantic integration potentials of five major terminology systems: SNOMED CT; Logical Observation Identifiers Names and Codes (LOINC); International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM); Gene Ontology (GO); and Current Procedural Terminology. The Unified Medical Language System Metathesaurus was employed to establish semantic equivalency between concepts from different systems. SNOMED CT exhibited superior granularity across most clinical domains, with some exceptions (ICD-10-CM in “Qualifier value,” GO in “Observable entity,” and LOINC in “Staging and scales.”) These findings address the challenge of semantic degradation in health information exchange by quantifying the degree to which meaning might be lost when translating between terminology systems. The proposed metrics empower healthcare organizations to develop targeted extensions or integration strategies that maintain semantic consistency across systems, providing objective tools for terminology system selection, integration planning, and semantic interoperability assessment.Supplementary InformationThe online version contains supplementary material available at 10.1038/s41598-025-17737-0.

This paper describes preliminary work evaluating automated semantic indexing of radiology imaging reports to represent images stored in the Image Engine multimedia medical record system at the University of Pittsburgh Medical Center. The authors used the SAPHIRE indexing system to automatically identify important biomedical concepts within radiology reports and represent these concepts with terms from the 1998 edition of the U.S. National Library of Medicine's Unified Medical Language System (UMLS) Metathesaurus. This automated UMLS indexing was then compared with manual UMLS indexing of the same reports. Human indexing identified appropriate UMLS Metathesaurus descriptors for 81% of the important biomedical concepts contained in the report set. SAPHIRE automatically identified UMLS Metathesaurus descriptors for 64% of the important biomedical concepts contained in the report set. The overall conclusions of this pilot study were that the UMLS metathesaurus provided adequate coverage of the majority of the important concepts contained within the radiology report test set and that SAPHIRE could automatically identify and translate almost two thirds of these concepts into appropriate UMLS descriptors. Further work is required to improve both the recall and precision of this automated concept extraction process.

In this paper, we present a knowledge-assisted approach to index and retrieve large volume of medical images. Both images and associated texts are indexed using medical concepts from the Unified Medical Language System (UMLS) meta-thesaurus. We propose a structured learning framework for modular acquisition of medical semantics from images with complementary global and local image indexing schemes. Two fusion approaches are also developed to improve text retrieval using the UMLS-based image indexing: a simple post-query fusion and a visual modality filtering to remove visually aberrant images according to the query modality concepts. On the ImageCLEFmed 2005 database, our framework outperformed our previous result which ranked top in the ImageCLEFmed 2005 Medical Image Retrieval task benchmark.

BackgroundRadiology reporting is a clinically oriented form of documentation that reflects critical information for patients about their health care processes. Realizing its importance, many medical institutions have started providing radiology reports in patient portals. The gain, however, can be limited because of medical language barriers, which require a way for customizing these reports for patients. The open-access, collaborative consumer health vocabulary (CHV) is a terminology system created for such purposes and can be the basis of lexical simplification processes for clinical notes.ObjectiveThe aim of this study was to examine the comprehensibility and suitability of CHV in simplifying radiology reports for consumers. This was done by characterizing the content coverage and the lexical similarity between the terms in the reports and the CHV-preferred terms.MethodsThe overall procedure was divided into the following two main stages: (1) translation and (2) evaluation. The translation process involved using MetaMap to link terms in the reports to CHV concepts. This is followed by replacing the terms with CHV-preferred terms using the concept names and sources table (MRCONSO) in the Unified Medical Language System (UMLS) Metathesaurus. In the second stage, medical terms in the reports and general terms that are used to describe medical phenomena were selected and evaluated by comparing the words in the original reports with the translated ones. The evaluation includes measuring the content coverage, investigating lexical similarity, and finding trends in missing concepts.ResultsOf the 792 terms selected from the radiology reports, 695 of them could be mapped directly to CHV concepts, indicating a content coverage of 88.5%. A total of 51 of the concepts (53%, 51/97) that could not be mapped are names of human anatomical structures and regions, followed by 28 anatomical descriptions and pathological variations (29%, 28/97). In addition, 12 radiology techniques and projections represented 12% of the unmapped concepts, whereas the remaining six concepts (6%, 12/97) were physiological descriptions. The rate of lexical similarity between the CHV-preferred terms and the terms in the radiology reports was approximately 72.6%.ConclusionsThe CHV covered a high percentage of concepts found in the radiology reports, but unmapped concepts are associated with areas that are commonly found in radiology reporting. CHV terms also showed a high percentage of lexical similarity with terms in the reports, which contain a myriad of medical jargon. This suggests that many CHV terms might not be suitable for lay consumers who would not be facile with radiology-specific vocabulary. Therefore, further patient-centered content changes are needed of the CHV to increase its usefulness and facilitate its integration into consumer-oriented applications.

Data mining is a process of discovering useful information from a database and analysis of extracted information. Text mining uses many techniques of data mining. It primarily deals with unstructured data. Web mining is an extension of text mining since it deals with unstructured data. Data mining relates to find data from “static databases” which contains “structured” data where as, web mining plays with data that are “dynamic” and “unstructured”. In this papers our goal is to mine biomedical data from hypertext documents (e.g., mining data from web contents) using text mining techniques with the help of “biomedical ontology”. Web data repositories are the hypertext documents. Texts in the Hypertext documents are unstructured and they contain Hypertext Markup Language (HTML) tags, scripting languages, images, audios, videos, URLs etc. We collect a number of documents using Google crawler and preprocess the hypertext documents and extract the text data. Next, we identify whether a word is a biomedical entity or not by using a biomedical database the “Unified Medical Language System (UMLS) metathesaurus”. The mapping of biomedical entity from the metathesaurus will be done based on keyword query. Then we apply the result to re-rank the web documents to find most relevant documents. We conclude that the more occurrence of a biomedical entity in a page, the more relevant the page is, and thus, we can re-rank the documents to find the most relevant documents by using text mining technique.