Decoding the Patient Narrative: Natural Language Processing and Deep Learning for Improved Clinical Text Analysis

1 IntroductionOrganizations focus on automate their processes in order to improve efficiency, reduce costs, and/or reduce human beings' errors in an easy and rapid manner. Business process management (BPM) methods provide a solution to this issue. In this context, information systems like CRM, ERP, SCM, etc. have known an increasing demand. The main problem consists of the length of the business process specifications. If new regulations appear, these specifications must be adapted. Manually extraction of visual models is time consuming. During time, a series of solutions were proposed. The literature shows a crowd of approaches that extracts data models [1], [2], [3], and process models [4], [5] from Natural Language (NL) text. This paper aims to analyze the Natural Language Processing (NLP) techniques and tools used in order to provide different types of visual representations.On the last few years, approaches based on NLP have been developed in order to automate process conversion from NL text. NLP plays an important role in NL text analysis as NLP tries to understand speech and text as humans beings would do. Colloquialism, abbreviations or typos make this task a challenging one. NLP has the origins in 1950s when Alan Turing proposed the Turing test [6], by introducing the imitation game. Since then, the literature shows a plethora of NLP tools [7], [8], [9], [10] using several machine learning techniques, our focus being on those applied on data models and process models discovery from NL text, starting with the first language parser [11] to the actual ones like NLTk [7], [8], ANTLR1, etc.Linguistic analysis is closely tied to NLP. Liddy [12] highlights 7 levels of linguistic analysis: a) Phonetic or Phonological level: how words are pronounced, b) Morphological level: prefixes, suffixes and roots analysis, c) Lexical level: word level analysis including lexical meaning and Part-Of-Speech (POS) analysis, d) Syntactic level: grammatical analysis of words in a sentence, e) Semantic level: determining the possible meanings of sentences, f) Discourse level: interpreting structure and meaning for texts larger than a sentence, g) Pragmatic level: understanding the purpose of a language.Some of the problems approached by NLP are: POS tagging, parsing, Named Entity Recognition (NER), chunking, Semantic Role Labeling (SRL). Anaphora resolution [13] refers to the interpretation of the link between the anaphor and its antecedents.The remainder of the paper is organized as follows: Section 2 briefly outlines the NLP domain, describing the levels of linguistic analysis and the main NLP approaches of analyzing NL requirements. Section 3 focuses on data and process models extraction from NL text. This section makes an introduction to Object Oriented Analysis and Business Process Modeling and analyses the existing tools that discover data and process models from text. Subsequently, Section 4 summarizes the results of this work and the conclusions are drawn in Section 5.2NLPA detailed review on NLP is given in [14] and in [15]. Jones [14] divides the history of NLP into four phases: the first starts at the beginning of the 1940s and lasts to the late 1960s, the second begins from the end of 60s and lasts to the end of 70s, the third is represented by late 80s, where the fourth phase starts in the late of 90s. Next, we will detail each phase as they were defined in [14] and [15]. First phase treated machine translation issues, while the second focused on artificial intelligence. The third phase can be called grammatico-logical phase, which is followed by the lexical phase. A fifth phase is proposed in [16] where formal theories and statistical data are combined. Since this study was published first in 1994 and then it was re-organized in 2001 we can add the sixth phase: from 2000 until present where NLP techniques are combined in order to contribute to visual models extraction.Software requirements are usually written in NL which is asymmetric and irregular [17]. …

Customer sentiment analytics has emerged as a critical tool for retail businesses aiming to understand consumer perceptions, improve customer experience, and enhance decision-making. This paper explores the application of Natural Language Processing (NLP) and deep learning techniques in analyzing unstructured textual data such as customer reviews, social media posts, and feedback to extract actionable insights for the retail industry. Traditional sentiment analysis methods often rely on keyword-based approaches, which lack the contextual understanding necessary to interpret nuanced emotions and implicit meanings. In contrast, NLP combined with advanced deep learning models, such as Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN), and Transformers (e.g., BERT), enables more accurate sentiment classification by capturing semantic relationships, context, and sentiment polarity. The study presents a comprehensive framework that leverages data preprocessing, tokenization, embedding layers (e.g., Word2Vec, GloVe), and model training pipelines to analyze large volumes of customer-generated content. Experimental results from real-world retail datasets demonstrate the superiority of deep learning models over traditional machine learning techniques in sentiment prediction accuracy, especially in multilingual and noisy data scenarios. Moreover, the paper discusses the integration of sentiment scores with customer segmentation and product performance metrics to drive personalized marketing, inventory adjustments, and service enhancements. Challenges such as data labeling, domain adaptation, and interpretability of deep models are also addressed, along with strategies to overcome them using transfer learning, attention mechanisms, and explainable AI tools. The research concludes that customer sentiment analytics using NLP and deep learning is not merely a technological advancement but a strategic enabler for data-driven retail transformation. By systematically capturing and interpreting customer voices, retail businesses can proactively respond to market demands, enhance loyalty, and gain a competitive edge in an increasingly digital consumer landscape.

Sentiment analysis (or opinion mining), is a natural language processing (NLP) technique that harnesses Information from text data that is naturally subjective. Opinions, emotions, and attitudes are things that this information can contain. In recent years, there has been an increasing focus on sentiment analysis because of the explosive growth of unstructured data on the internet. Sentiment analysis tasks have seen state of the art performance using deep learning models. However, these models can be computationally intensive and high dimensional can produce problems of overfitting when working with relatively limited data. In order to overcome these limitations, a new hybrid model is proposed using a combination of LSTM network for feature extraction and Random Forest classifier for the final sentiment classification improving the accuracy of the sentiment analysis, while reducing computational cost. In this paper, the proposed hybrid model is designed and implemented, and evaluated on a publicly available dataset of movie reviews.

Named Entity Recognition is a well-known research direction in the area of deep learning. It takes an essential role in natural language processing. The goal of the Named Entity Recognition is to identify and separate the named entities, such as a person, location, or name, from the entire text. In addition, the deep learning model has achieved remarkable achievements in many other areas, and the deep learning-based named entity recognition method has reached an F score of over 90. The paper summarizes the development of named entity recognition and puts forward a recurrent neural network-based named entity recognition algorithm. The result shows that improving the performance of the Named Entity Recognition model by simply enriching the number and variety of the input datasets or providing the model with substantial computing resources for training is nearly impossible without a significant breakthrough. There still requires another way to improve the NER model in future research.

PurposeTo conduct a head-to-head comparison between deep learning (DL) and radiomics models across institutions for predicting microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and to investigate the model robustness and generalizability through rigorous internal and external validation.MethodsThis retrospective study included 2304 preoperative images of 576 HCC lesions from two centers, with MVI status determined by postoperative histopathology. We developed DL and radiomics models for predicting the presence of MVI using B-mode ultrasound, contrast-enhanced ultrasound (CEUS) at the arterial, portal, and delayed phases, and a combined modality (B + CEUS). For radiomics, we constructed models with enlarged vs. original regions of interest (ROIs). A cross-validation approach was performed by training models on one center’s dataset and validating the other, and vice versa. This allowed assessment of the validity of different ultrasound modalities and the cross-center robustness of the models. The optimal model combined with alpha-fetoprotein (AFP) was also validated. The head-to-head comparison was based on the area under the receiver operating characteristic curve (AUC).ResultsThirteen DL models and 25 radiomics models using different ultrasound modalities were constructed and compared. B + CEUS was the optimal modality for both DL and radiomics models. The DL model achieved AUCs of 0.802–0.818 internally and 0.667–0.688 externally across the two centers, whereas radiomics achieved AUCs of 0.749–0.869 internally and 0.646–0.697 externally. The radiomics models showed overall improvement with enlarged ROIs (P < 0.05 for both CEUS and B + CEUS modalities). The DL models showed good cross-institutional robustness (P > 0.05 for all modalities, 1.6–2.1% differences in AUC for the optimal modality), whereas the radiomics models had relatively limited robustness across the two centers (12% drop-off in AUC for the optimal modality). Adding AFP improved the DL models (P < 0.05 externally) and well maintained the robustness, but did not benefit the radiomics model (P > 0.05).ConclusionCross-institutional validation indicated that DL demonstrated better robustness than radiomics for preoperative MVI prediction in patients with HCC, representing a promising solution to non-standardized ultrasound examination procedures.

Deep learning subsumes algorithms that automatically learn compositional representations. The ability of these models to generalize well has ushered in tremendous advances in many fields such as natural language processing (NLP). Recent research in the software engineering (SE) community has demonstrated the usefulness of applying NLP techniques to software corpora. Hence, we motivate deep learning for software language modeling, highlighting fundamental differences between state-of-the-practice software language models and connectionist models. Our deep learning models are applicable to source code files (since they only require lexically analyzed source code written in any programming language) and other types of artifacts. We show how a particular deep learning model can remember its state to effectively model sequential data, e.g., Streaming software tokens, and the state is shown to be much more expressive than discrete tokens in a prefix. Then we instantiate deep learning models and show that deep learning induces high-quality models compared to n-grams and cache-based n-grams on a corpus of Java projects. We experiment with two of the models' hyper parameters, which govern their capacity and the amount of context they use to inform predictions, before building several committees of software language models to aid generalization. Then we apply the deep learning models to code suggestion and demonstrate their effectiveness at a real SE task compared to state-of-the-practice models. Finally, we propose avenues for future work, where deep learning can be brought to bear to support model-based testing, improve software lexicons, and conceptualize software artifacts. Thus, our work serves as the first step toward deep learning software repositories.

Many researchers in various disciplines have focused on extracting meaningful information from social media platforms in recent years. Identification of behaviors and emotions from user posts is examined under the heading of sentiment analysis (SA) studies using the natural language processing (NLP) techniques. In this study, a novel TCNN-Bi-LSTM model using the two-stage convolutional neural network (TCNN) and bidirectional long short-term memory (Bi-LSTM) architectures was proposed. While TCNN layers enable the extraction of strong local features, the output of these layers feeds the Bi-LSTM model that remembers forward-looking information and capture long-term dependencies. In this study, first, preprocessing steps were applied to the raw dataset. Thus, strong features were extracted from the obtained quality dataset using the FastText word embedding technique that pre-trained with location-based and sub-word information features. The experimental results of the proposed method are promising compared to the baseline deep learning and machine learning models. Also, experimental results show that while the FastText data embedding technique achieves the best performance compared to other word embedding techniques in all deep learning classification models, it has not had the same outstanding success in machine learning models. This study aims to investigate the sentiments of tweets about the COVID-19 vaccines and comments on these tweets among Twitter users by using the power of Twitter data. A new dataset collected from Twitter was constructed to be used in experimental results. This study will facilitate detecting inappropriate, incomplete, and erroneous information about vaccination. The results of this study will enable society to broaden its perspective on the administered vaccines. It can also assist the government and healthcare agencies in planning and implementing the vaccination's promotion on time to achieve the herd immunity provided by the vaccination.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Sentiment analysis is a subfield of Natural Language Processing (NLP) which tries to process a text to extract opinions or attitudes towards topics or entities. Recently, the use of deep learning methods for sentiment analysis has received noticeable attention from researchers. Generally, different deep learning methods have shown superb performance in sentiment analysis problem. However, deep learning models are different in nature and have different strengths and limitations. For example, convolutional neural networks are useful for extracting local structures from data, while recurrent models are able to learn order dependence in sequential data. In order to combine the advantages of different deep models, in this paper we have proposed a novel approach for aspect-based sentiment analysis which utilizes deep ensemble learning. In the proposed method, we first build four deep learning models, namely CNN, LSTM, BiLSTM and GRU. Then the outputs of these models are combined using stacking ensemble approach where we have used logistic regression as meta-learner. The results of applying the proposed method on the real datasets show that our method has increased the accuracy of aspect-based prediction by 5% to 20% compared to the basic deep learning methods.

This Chapter discusses the application of Natural Language Processing (NLP) in the legal domain for entity identification and the generation of hierarchy-based applications. NLP techniques, such as Named Entity Recognition (NER) and information extraction, are utilized to analyse legal documents, extract key information, and understand contextual relationships. The integration of NLP facilitates tasks such as legal document summarization, contract analysis, automated document generation, and legal research, thereby enhancing the efficiency and accuracy of legal processes. The creation of hierarchical structures within documents, language translation, sentiment analysis, and compliance monitoring further contribute to the comprehensive utilization of NLP in the legal field. The abstract highlights the transformative impact of NLP in streamlining legal workflows, improving information retrieval, and ensuring compliance with evolving legal standards. Most judicial systems in the world, centered around the Supreme Court as the highest-level court, grapples with a substantial caseload, receiving over 7,500 cases annually but hearing fewer than 150. The Supreme Court's unique role in interpreting the constitutionality of laws and establishing precedents underscores its significant influence. Challenges arise from the overwhelming caseload, leading to increased use of plea bargains and potential concerns about justice. To address these issues, a proposed research initiative suggests employing machine learning and natural language processing to accelerate Supreme Court decisions, aiming to uncover pivotal aspects that greatly influence the outcomes, and enhance our comprehension on India legal system's functioning and constraints. This chapter presents a methodology for sentiment-based summarization of legal documents using Natural Language Processing (NLP) techniques. The process involves pre-processing the text, conducting sentiment analysis to label sentences as positive, negative, or neutral, and then summarizing the document while incorporating the identified sentiments. Special attention is paid to legal-specific considerations such as terminology and context.

Harnessing attention mechanisms in a comprehensive deep learning approach for induction motor fault diagnosis using raw electrical signals

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.

The AI voice assistant mobile application was developed to aid drivers in operating their mobile phones while driving without touching their phones. The literature review examines multiple innovative artificial technologies involved in applications with voice assistants in natural language processing (NLP) techniques. The methodology used involved a qualitative approach, and the design science paradigm was used for the development of the voice assistant for smartphones with NLP techniques. NLP techniques that were applied in the development of the AI voice assistant are smart synthesis, data flow sequence, core and interface accessing, part of speech tagging, named entity recognition, conference resolution, and porter stemming. Some of the operations that are achieved by the application include arithmetic calculations based on voice commands and returning the computer result via voice, searching the internet based on user voice input, and providing a response via voice assistance.

In secondary analysis of electronic health records, a crucial task consists in correctly identifying the patient cohort under investigation. In many cases, the most valuable and relevant information for an accurate classification of medical conditions exist only in clinical narratives. Therefore, it is necessary to use natural language processing (NLP) techniques to extract and evaluate these narratives. The most commonly used approach to this problem relies on extracting a number of clinician-defined medical concepts from text and using machine learning techniques to identify whether a particular patient has a certain condition. However, recent advances in deep learning and NLP enable models to learn a rich representation of (medical) language. Convolutional neural networks (CNN) for text classification can augment the existing techniques by leveraging the representation of language to learn which phrases in a text are relevant for a given medical condition. In this work, we compare concept extraction based methods with CNNs and other commonly used models in NLP in ten phenotyping tasks using 1,610 discharge summaries from the MIMIC-III database. We show that CNNs outperform concept extraction based methods in almost all of the tasks, with an improvement in F1-score of up to 26 and up to 7 percentage points in area under the ROC curve (AUC). We additionally assess the interpretability of both approaches by presenting and evaluating methods that calculate and extract the most salient phrases for a prediction. The results indicate that CNNs are a valid alternative to existing approaches in patient phenotyping and cohort identification, and should be further investigated. Moreover, the deep learning approach presented in this paper can be used to assist clinicians during chart review or support the extraction of billing codes from text by identifying and highlighting relevant phrases for various medical conditions.

We developed a multimodal ultrasound (US) deep learning (DL) fusion model to automatically classify early fibrosis in patients with chronic kidney disease (CKD). This prospective study included patients with CKD who underwent continuous gray-scale US, superb microvascular imaging, and strain elastography from May to November 2022. According to the pathological tubular atrophy and interstitial fibrosis score, patients were divided into minimal and mild groups (affected area ≤10% and 11 − 25% of the total cortical volume, respectively). The dataset was divided into training (70%) and test (30%) sets. A DL model combining the features of the three US modes was developed to predict early fibrosis in patients with CKD. We compared these findings with the area under the receiver operating characteristic curve (AUC) of the clinical model by analyzing the receiver operating characteristic curve in the test set. The AUC of single-mode DL based on gray-scale US, superb microvascular imaging, and strain elastography was 0.682, 0.745, and 0.648, respectively, while that of the multimodal US DL model was 0.86. The accuracy, specificity, and sensitivity of the multimodal US DL model were 0.779, 0.767, and 0.796, respectively, and the negative and positive predictive values were 0.842 and 0.706, respectively. The AUC of the multimodal US DL model was significantly better than that of the single-mode DL and clinical models. The DL algorithm developed using multimodal US images can effectively predict early fibrosis in patients with CKD with significantly greater accuracy than single-mode DL or clinical models.