A method to extract essential keywords from a tweet using NLP tools

Applications in the World Wide Web aggregate vast amounts of information from different data sources. The aggregation process is often implemented with Extract, Transform and Load (ETL) processes. Usually ETL processes require information for aggregation available in structured formats, e. g. XML or JSON. In many cases the information is provided in natural language text which makes the application of ETL processes impractical. Due to the fact that information is provided in natural language, Information Extraction (IE) systems have been evolved. They make use of Natural Language Processing (NLP) tools to derive meaning from natural language text. State-of-the-art NLP tools apply Machine Learning methods. These NLP tools perform on newspapers with good quality, but they drop accuracy in other domains. However, to improve the quality for IE systems in specific domains often NLP tools are trained on domain specific text which is a time consuming process. This paper introduces an approach using a Continuous Integration pipeline for organizing and monitoring the annotation process on domain specific corpora.

We present SimpleApprenant, a platform aiming to improve French L2 learners’ knowledge of Multi Word Expressions (MWEs). SimpleApprenant integrates an MWE database annotated with the Common European Framework of Reference for languages (CEFR) level and several Natural Language Processing (NLP) tools: a spelling checker, a parser, and a set of transformation rules. NLP tools and resources are used to build training and writing exercises to improve MWE knowledge and writing skills of French L2 learners. We present the user scenarios, the platform’s architecture, as well as the preliminary evaluation of its NLP tools.

Objective: The prevalence of moderate to severe asymptomatic carotid stenosis (ie, atherosclerotic narrowing of the extracranial carotid arteries) is generally approximately 6% and 2%, respectively. Most prior studies of carotid stenosis risk factors have been small. This study describes the development and validation of a natural language processing (NLP) tool to identify carotid stenosis and uses it to identify significant risk factors, presence, and severity of carotid stenosis. Methods: We created an NLP tool to extract the ratio of peak systolic velocity of the internal carotid artery to the common carotid artery (ICA/CCA ratio) in veterans receiving carotid duplex ultrasound examinations in the Veteran's Health Administration from 2001 to 2020. Among those who had at least one valid ICA/CCA ratio, we identified carotid stenosis severity (<50%, 50%-69%, ≥70%) based on the ICA/CCA ratio (<2, ≥2 to <4, ≥4) and assessed the association between presence and severity of carotid stenosis and clinical and demographic characteristics, including age, sex, self-identified race and ethnicity, smoking status, body mass index, systolic and diastolic blood pressures, indicator variables for pre-existing hypertension, coronary heart disease, and type 2 diabetes, and selected laboratory measures (ie, hemoglobin A1c, low-density lipoprotein cholesterol, high-density lipoprotein cholesterol, triglyceride, and creatinine). Results: The harmonic F1 score of the NLP tool was 0.907 for the right value, 0.882 for the left value, and 0.920 for the maximum value. Among the 290,517 veterans in the cohort, the median age was 68.2 years. Black patients had 16% decreased risk of more severe carotid stenosis (odds ratio: 0.84, 95% confidence interval: 0.81-0.87, P < .001). All patient-level risk factors except high-density lipoprotein cholesterol were significantly associated with carotid stenosis severity. Conclusions: The NLP tool performed well, and the study performed with our NLP-created cohort largely validates the risk factors identified by previous smaller studies, demonstrating the utility of big data and NLP in carotid stenosis research.

The US Food and Drug Administration Adverse Event Reporting System (FAERS) contains over 24 million individual case safety reports (ICSRs). In this research project, we evaluated a natural language processing (NLP) tool’s ability to extract four demographic variables (gender, weight, ethnicity, race) from ICSR narratives. Specificity of the NLP algorithm was over 94% for all demographics, while sensitivity varied between the demographics: 98.6% (gender), 45.5% (weight), 100% (ethnicity), and 85.3% (race). Among ICSRs missing weight, ethnicity, and race in the structured field, few cases had this information in the narrative (&amp;gt;95% missing); consequently, the positive predictive value (PPV) for these three demographics had wide 95% confidence intervals. After NLP implementation, the total number of ICSRs missing gender was reduced by 33% (i.e., NLP identified 472 thousand reports having a gender value in the narrative that was not in the structured field), while the total number of ICSRs missing weight, ethnicity, or race was reduced by less than 4%. This study demonstrated that the implementation of an NLP tool can provide meaningful improvements in the availability of gender information for pharmacovigilance activities conducted with FAERS data. In contrast, NLP tools targeting the extraction of weight, ethnicity, or race from free-text fields have minimal impact largely because the information was infrequently provided by the reporter. Further gains in completeness of these fields must originate from increases in provision of demographic information from the reporter rather than informatic solutions.

Evaluation of Natural Language Processing (NLP) systems to annotate drug product labeling with MedDRA terminology

1 IntroductionOrganizations focus on automate their processes in order to improve efficiency, reduce costs, and/or reduce human beings' errors in an easy and rapid manner. Business process management (BPM) methods provide a solution to this issue. In this context, information systems like CRM, ERP, SCM, etc. have known an increasing demand. The main problem consists of the length of the business process specifications. If new regulations appear, these specifications must be adapted. Manually extraction of visual models is time consuming. During time, a series of solutions were proposed. The literature shows a crowd of approaches that extracts data models [1], [2], [3], and process models [4], [5] from Natural Language (NL) text. This paper aims to analyze the Natural Language Processing (NLP) techniques and tools used in order to provide different types of visual representations.On the last few years, approaches based on NLP have been developed in order to automate process conversion from NL text. NLP plays an important role in NL text analysis as NLP tries to understand speech and text as humans beings would do. Colloquialism, abbreviations or typos make this task a challenging one. NLP has the origins in 1950s when Alan Turing proposed the Turing test [6], by introducing the imitation game. Since then, the literature shows a plethora of NLP tools [7], [8], [9], [10] using several machine learning techniques, our focus being on those applied on data models and process models discovery from NL text, starting with the first language parser [11] to the actual ones like NLTk [7], [8], ANTLR1, etc.Linguistic analysis is closely tied to NLP. Liddy [12] highlights 7 levels of linguistic analysis: a) Phonetic or Phonological level: how words are pronounced, b) Morphological level: prefixes, suffixes and roots analysis, c) Lexical level: word level analysis including lexical meaning and Part-Of-Speech (POS) analysis, d) Syntactic level: grammatical analysis of words in a sentence, e) Semantic level: determining the possible meanings of sentences, f) Discourse level: interpreting structure and meaning for texts larger than a sentence, g) Pragmatic level: understanding the purpose of a language.Some of the problems approached by NLP are: POS tagging, parsing, Named Entity Recognition (NER), chunking, Semantic Role Labeling (SRL). Anaphora resolution [13] refers to the interpretation of the link between the anaphor and its antecedents.The remainder of the paper is organized as follows: Section 2 briefly outlines the NLP domain, describing the levels of linguistic analysis and the main NLP approaches of analyzing NL requirements. Section 3 focuses on data and process models extraction from NL text. This section makes an introduction to Object Oriented Analysis and Business Process Modeling and analyses the existing tools that discover data and process models from text. Subsequently, Section 4 summarizes the results of this work and the conclusions are drawn in Section 5.2NLPA detailed review on NLP is given in [14] and in [15]. Jones [14] divides the history of NLP into four phases: the first starts at the beginning of the 1940s and lasts to the late 1960s, the second begins from the end of 60s and lasts to the end of 70s, the third is represented by late 80s, where the fourth phase starts in the late of 90s. Next, we will detail each phase as they were defined in [14] and [15]. First phase treated machine translation issues, while the second focused on artificial intelligence. The third phase can be called grammatico-logical phase, which is followed by the lexical phase. A fifth phase is proposed in [16] where formal theories and statistical data are combined. Since this study was published first in 1994 and then it was re-organized in 2001 we can add the sixth phase: from 2000 until present where NLP techniques are combined in order to contribute to visual models extraction.Software requirements are usually written in NL which is asymmetric and irregular [17]. …

The aim of this study was to test the ability of a commercially available natural language processing (NLP) tool to accurately extract examination quality-related and large polyp information from colonoscopy reports with varying report formats. Colonoscopy quality reporting often requires manual data abstraction. NLP is another option for extracting information; however, limited data exist on its ability to accurately extract examination quality and polyp findings from unstructured text in colonoscopy reports with different reporting formats. NLP strategies were developed using 500 colonoscopy reports from Kaiser Permanente Northern California and then tested using 300 separate colonoscopy reports that underwent manual chart review. Using findings from manual review as the reference standard, we evaluated the NLP tool's sensitivity, specificity, positive predictive value (PPV), and accuracy for identifying colonoscopy examination indication, cecal intubation, bowel preparation adequacy, and polyps ≥10 mm. The NLP tool was highly accurate in identifying examination quality-related variables from colonoscopy reports. Compared with manual review, sensitivity for screening indication was 100% (95% confidence interval: 95.3%-100%), PPV was 90.6% (82.3%-95.8%), and accuracy was 98.2% (97.0%-99.4%). For cecal intubation, sensitivity was 99.6% (98.0%-100%), PPV was 100% (98.5%-100%), and accuracy was 99.8% (99.5%-100%). For bowel preparation adequacy, sensitivity was 100% (98.5%-100%), PPV was 100% (98.5%-100%), and accuracy was 100% (100%-100%). For polyp(s) ≥10 mm, sensitivity was 90.5% (69.6%-98.8%), PPV was 100% (82.4%-100%), and accuracy was 95.2% (88.8%-100%). NLP yielded a high degree of accuracy for identifying examination quality-related and large polyp information from diverse types of colonoscopy reports.

Purpose This study explores the potential of Natural Language Processing (NLP) tools, specifically ChatGPT-4o and Data Analyst, in supporting health commissions with disability assessments. Methods Nine realistic patient scenarios were created to reflect typical cases in disability evaluations, encompassing conditions like stroke, cerebral palsy, traumatic injuries, and hereditary disorders, each involving varied motor and functional impairments. These scenarios were input into ChatGPT-4o and Data Analyst. Their outputs were evaluated using a 5-point Likert scale on alignment with expert guidelines, completeness, and lack of false information. Interrater reliability was established before scoring. Results Both models produced generally high-quality narrative reports, ChatGPT-4o was rated “very good” and Data Analyst “good,” with no statistically significant difference in overall scores. However, ChatGPT-4o failed to calculate correct disability percentages in 55.6%, and Data Analyst failed in 88.9% of scenarios. Conclusions While NLP tools can assist in generating structured disability reports, they currently lack the precision needed for calculating disability percentages reliably. Expert oversight remains essential for decision-making in disability assessments.

Biographical writing is one of the earliest and most extensive forms of Arabic literature. Some scholars tend to assume that classical Arabic biographies, widely known as Tarāǧim, arose in conjunction with the study of the reliability of the Hadith transmitters (the reciters of the Prophet Mohammad's sayings) which lead to a proliferation of biographical material collected and used to assess the transmitter's trustworthiness . However, a scrutiny of the well-known classical Arabic biographical dictionaries such as Siyaru 'A`lāmi an-Nubalā' `The Lives of the Noble Figures' for Adh-Dhahabī shows that they extend their entries to other classes of persons important to the development of particular fields such as Islamic jurisprudents, rulers, poets, philosophers or physicians. The main contribution of Arabic biographical dictionaries is the cumulative value of the thousands of life histories which construct a picture of the Islamic society in different eras. An Arabic biographical dictionary, therefore, is predominantly used by scholars to look up an eminent person's achievements and historical background. In this project, however, we explore Arabic biographies as a prosopography, rather than a biography in the strict sense. We introduce a novel method for a better understanding of Arabic biographical dictionaries by creating a network of relations among different persons. We utilise Natural Language Processing (NLP) tools to create a topological network from the unstructured data of 45,500 biographical entries collected from different dictionaries. We aim to illustrate how network analysis leveraged by NLP tools can provide scholars with innovative methods for discovering complex constellation of relations between prominent and non-prominent figures spanning over several eras and from different fields of knowledge. We also use graph visualisation as a means to effectively communicate and explore such complex constellations. Each network visualisation is purposefully designed to be as simple and robust as possible to offer scholars a way to move relatively fluidly between the large scale of biographical entries and to easily interpret the minute ties between persons of different walks of life. We make both our data and code publicly available for researchers to replicate the experiment. It can be found at:https://github.com/sadanyh/Relational-Network-for-Arabic-Tarajem

Historical data poses unique challenges to natural language processing (NLP) and information retrieval (IR) tools, including digitization errors, lack of annotated data, and diachronic-specific issues. However, the increasing recognition of the value in historical documents has promoted efforts to semantically enrich and optimize their analysis. This article contributes to this endeavour by enriching the Corpus de Textos Antigos through NLP tools and techniques to enhance its usability and support research. The corpus undergoes linguistic annotation, including part-of-speech tagging, lemma annotation and named entity recognition (NER). Subsequently, the article delves into the tasks of entity disambiguation and entity linking, which involve identifying and disambiguating named entities by referring to a knowledge base (KB). Addressing the challenges posed by factors such as text state, epoch and the chosen KB, the article presents insights into related work, annotation results and the linguistic interest of a medieval annotated corpus for named entities. It concludes by discussing the challenges and providing avenues for future research in this domain.

Towards the first Maithili part of speech tagger: Resource creation and system development

Introduction:Nowadays in Panama, there is a lot of patient information stored in textual form which cannot be manipulated to manage adequate knowledge. There are multiple resources created to represent knowledge, including specialized glossaries, ontologies, among others. The ontologies are an important part within the scope of the recovery and organization of the information and the semantic web. Also in recent works they are used in applications of natural language processing (NLP), as a knowledge base.Aim:This research was conducted with the aim of creating a methodology that allows from a text written in NL, extract the necessary elements using NLP tools and with them create a knowledge base represented by one domain ontology and extract knowledge to help medical specialists.Material and Methods:In this study we carried out a methodology that allows the extraction of knowledge of patient clinical records, general medicine and palliative care, in order to show relevant knowledge elements to specialists. The methodology was validated with a data corpus of approximately 200 patient records.Conclusion:We have created a knowledge representation methodology, combining NLP techniques and tools and the automatic instantiation of an ontology, which can serve as a software agent for other applications or used to visualize the patient’s clinical information. The study was validated using the traditional metrics of information retrieval systems precision, recall, F-measure obtaining excellent results, and can be used as a software agent or methodology for the development of information extraction software systems in the medical domain.

The fashion department at SMK Negeri 1 Warureja faces significant challenges in understanding and applying Artificial Intelligence (AI)-based Natural Language Processing (NLP) among its students and teachers. Despite the potential benefits of NLP technology for the fashion industry, the limited knowledge and skills concerning NLP tools at SMK Negeri 1 Warureja impede effective utilization. This Community Service Program (CSP) is strategically designed to address these gaps by providing targeted training on NLP tools such as ChatGPT, Chatbot, and Brand24.com within the fashion context. This CSP activity applies the technique of training an initial evaluation through a pre-test assessed participants' baseline understanding prior to the training. During the practical tutorial phase, participants received hands-on training with NLP tools, including practical guidance to ensure real-world application of these concepts. Evaluation results indicated a significant improvement in participants' understanding and skills. The post-test average of 80.47 points showed a significant improvement of 32.19 points over the pre-test average of 48.28 points.

In the age of knowledge, Natural Language Processing (NLP) express its demand by a huge range of utilization. Previously NLP was dealing with statically data. Contemporary time NLP is doing considerably with the corpus, lexicon database, pattern reorganization. Considering Deep Learning (DL) method recognize artificial Neural Network (NN) to nonlinear process, NLP tools become increasingly accurate and efficient that begin a debacle. Multi-Layer Neural Network obtaining the importance of the NLP for its capability including standard speed and resolute output. Hierarchical designs of data operate recurring processing layers to learn and with this arrangement of DL methods manage several practices. In this paper, this resumed striving to reach a review of the tools and the necessary methodology to present a clear understanding of the association of NLP and DL for truly understand in the training. Efficiency and execution both are improved in NLP by Part of speech tagging (POST), Morphological Analysis, Named Entity Recognition (NER), Semantic Role Labeling (SRL), Syntactic Parsing, and Coreference resolution. Artificial Neural Networks (ANN), Time Delay Neural Networks (TDNN), Recurrent Neural Network (RNN), Convolution Neural Networks (CNN), and Long-Short-Term-Memory (LSTM) dealings among Dense Vector (DV), Windows Approach (WA), and Multitask learning (MTL) as a characteristic of Deep Learning. After statically methods, when DL communicate the influence of NLP, the individual form of the NLP process and DL rule collaboration was started a fundamental connection.

To evaluate the diagnostic performance of a natural language processing (NLP) model in detecting incidental lung nodules (ILNs) in unstructured chest computed tomography (CT) reports. All unstructured consecutive reports of chest CT scans performed at a tertiary hospital between 2020 and 2021 were retrospectively reviewed (n = 21,542) to train the NLP tool. Internal validation was performed using reference readings by two radiologists of both CT scans and reports, using a different external cohort of 300 chest CT scans. Second, external validation was performed in a cohort of all random unstructured chest CT reports from 57 different hospitals conducted in May 2022. A review by the same thoracic radiologists was used as the gold standard. The sensitivity, specificity, and accuracy were calculated. Of 21,542 CT reports, 484 mentioned at least one ILN (mean age, 71 ± 17.6 [standard deviation] years; women, 52%) and were included in the training set. In the internal validation (n = 300), the NLP tool detected ILN with a sensitivity of 100.0% (95% CI, 97.6 to 100.0), a specificity of 95.9% (95% CI, 91.3 to 98.5), and an accuracy of 98.0% (95% CI, 95.7 to 99.3). In the external validation (n = 977), the NLP tool yielded a sensitivity of 98.4% (95% CI, 94.5 to 99.8), a specificity of 98.6% (95% CI, 97.5 to 99.3), and an accuracy of 98.6% (95% CI, 97.6 to 99.2). Twelve months after the initial reports, 8 (8.60%) patients had a final diagnosis of lung cancer, among which 2 (2.15%) would have been lost to follow-up without the NLP tool. NLP can be used to identify ILNs in unstructured reports with high accuracy, allowing a timely recall of patients and a potential diagnosis of early-stage lung cancer that might have been lost to follow-up.