A Year of Papers Using Biomedical Texts.

Medical natural language processing (NLP) systems are a key enabling technology for transforming Big Data from clinical report repositories to information used to support disease models and validate intervention methods. However, current medical NLP systems fall considerably short when faced with the task of logically interpreting clinical text. In this paper, we describe a framework inspired by mechanisms of human cognition in an attempt to jump the NLP performance curve. The design centers on a hierarchical semantic compositional model (HSCM), which provides an internal substrate for guiding the interpretation process. The paper describes insights from four key cognitive aspects: semantic memory, semantic composition, semantic activation, and hierarchical predictive coding. We discuss the design of a generative semantic model and an associated semantic parser used to transform a free-text sentence into a logical representation of its meaning. The paper discusses supportive and antagonistic arguments for the key features of the architecture as a long-term foundational framework.

Temporal reasoning with medical data—A review with emphasis on medical natural language processing

Big data analytics (BDA) is an advanced analytic technique used with very large and diverse sets of data from different sources. Natural language processing (NLP) is a technology that interfaces with different fields such as computer science, linguistics, and human-computer interactions. Over the past few years, there is a growing number of firms, which are using different BDA and NLP applications in their businesses. Only a few of the research have investigated different dimensions of NLP and BDA and their impacts on the overall organizational performance. There is a growing interest among researchers and practitioners in understanding the consequences for firms that adopt BDA and NLP applications. In this context, the aim of this article is to determine the factors for the usage of BDA and NLP applications in business. With the help of dynamic capability view theory and existing literature, a theoretical model was developed conceptually. Later, the model was validated using structural equation modeling approach considering 1287 samples from 23 firms, primarily based in Asia and Europe, which use NLP and BDA applications. The article finds that NLP and BDA applications help the firms to improve their operational efficiency, which in turn improves the overall firm performance.

The domain of Natural Language Processing covers various tasks, such as classification, text generation, and language model. The data processed using word embeddings, or vectorizers, is then trained using Machine Learning and Deep Learning algorithms. In order to observe the tradeoff between both these types of algorithms, with respect to data available, accuracy obtained and other factors, a binary classification is undertaken to distinguish between insincere and regular questions on Quora. A dataset called Quora Insincere Questions Classification was used to train various machine learning and deep learning models. A Bidirectional-Long Short Term Network (LSTM) was trained, with the text processed using Global Vectors for Word Representation (GloVe). Machine Learning algorithms such as Extreme Gradient Boosting classifier, Gaussian Naive Bayes, and Support Vector Classifier (SVC), by using the TF-IDF vectorizer to process the text. This paper also presents an evaluation of the above algorithms on the basis of precision, recall, f1 score metrics.

In the domain of Natural Language Generation and Processing, a lot of work is being done for text generation. As the machines become able to understand the text and language, it leads to a significant reduction in human involvement. Many sequence models show great work in generating human like text, but the amount of research work done to check the extent up to which their results match the man-made texts are limited in number. In this paper, the text is generated using Long Short Term Memory networks (LSTMs) and Generative Pretrained Transformer-2 (GPT-2). The text by neural language models based on LSTMs and GPT-2 follows Zipf’s law and Heap’s law, two statistical representations followed by every natural language generated text. One of the main findings is about the influence of parameter Temperature on the text produced. The LSTM generated text improves as the value of Temperature increases. The comparison between GPT-2 and LSTM generated text also shows that text generated using GPT-2 is more similar to natural text than that generated by LSTMs.

Machine learning techniques have long been the foundations of speech processing. Bayesian classiflcation, decision trees, unsupervised clustering, the EM algorithm, maximum entropy, etc. are all part of existing speech recognition systems. The success of statistical speech recognition has led to the rise of statistical and empirical methods in natural language processing. Indeed, many of the machine learning techniques used in language processing, from statistical part-of-speech tagging to the noisy channel model for machine translation have roots in work conducted in the speech fleld. However, advances in learning theory and algorithmic machine learning approaches in recent years have led to signiflcant changes in the direction and emphasis of the statistical and learning centered research in natural language processing and made a mark on natural language and speech processing. Approaches such as memory based learning, a range of linear classiflers such as Boosting, SVMs and SNoW and others have been successfully applied to a broad range of natural language problems, and these now inspire new research in speech retrieval and recognition. We have seen an increasingly close collaboration between voice and language processing researchers in some of the shared tasks such as spontaneous speech recognition and understanding, voice data information extraction, and machine translation. The purpose of this special issue was to invite speech and language researchers to communicate with each other, and with the machine learning community on the latest machine learning advances in their work. The call for papers was met with great enthusiasm from the speech and natural language community. Thirty six submissions were received; each paper was reviewed by at least three reviewers. Only ten papers were selected re∞ecting not only some of the best work on machine learning in the areas of natural language and spoken language processing but also what we view as a collection of papers that represent current trends in these areas of research both from the perspective of

Medical entity normalization is a fundamental task in medical natural language processing and clinical applications. The task aims to map medical mentions to standard entities in a given knowledge base. In this paper, we focus on Chinese medical procedure entity normalization. This task brings an extra multi-implication challenge that a mention may link to multiple standard entities. To perform the task, we propose a novel deep neural multi-task learning framework to jointly model implication number prediction and entity normalization. Our model utilizes the multi-head attention mechanism to provide mutual benefits between the two tasks. Experimental results show that our method achieves comparable performance compared with the baseline methods.

Malay social media text is a text written on social media networks like Twitter. Commonly, this text comprises non-standard words, filled with dialects, foreign languages, word abbreviations, grammatical neglect, spelling errors, and many more. It is well known that this type of text is difficult to process due to its high noise and distinct text structure. Such problems can be resolved using rigorous text normalization, which is critical before any technique can be implemented and evaluated on social media text. In this paper, an improved normalization method towards Malay social media text was proposed by converting non-standard Malay words using a rule-based model. The method normalizes common language words often used by Malaysian users, such as non-standard Malay (like dialect and slangs), Romanized Arabic, and English words. Thus, a Malay text normalizer was proposed using a set of rules that extend across different domains of natural language processing (NLP) and is expected to address the challenges of processing Malay social media text. This study implements the proposed Malay text normalizer in a Part-of-Speech (POS) tagging application to evaluate the normalizer’s performance. The implementation demonstrates a substantial improvement in the POS tagging efficiency over several pre-processing stages, with an improvement of accuracy up to 31.8%. The increase of accuracy in the POS tagging indicates two main points. First, the Malay text normalizer’s rules improve the performance of a Malay text normalizer on social media text. Second, our proposed Malay text normalizer has successfully improved the POS tagging percentage and demonstrates the importance of normalized pre-processing in any NLP application.

The artificial intelligence (AI), especially deep learning models, is highly compatible with medical images and natural language processing and is expected to be applied to pathological image analysis and other medical fields [...].

FDR-TransUNet: A novel encoder-decoder architecture with vision transformer for improved medical image segmentation

Accurate patient outcome prediction in the intensive care unit (ICU) can potentially lead to more effective and efficient patient care. Deep learning models are capable of learning from data to accurately predict patient outcomes, but they typically require large amounts of data and computational resources. Transfer learning (TL) can help in scenarios where data and computational resources are scarce by leveraging pretrained models. While TL has been widely used in medical imaging and natural language processing, it has been rare in electronic health record (EHR) analysis. Furthermore, domain adaptation (DA) has been the most common TL method in general, whereas inductive transfer learning (ITL) has been rare. To the best of our knowledge, DA and ITL have never been studied in-depth in the context of EHR-based ICU patient outcome prediction. This study investigated DA, as well as rarely researched ITL, in EHR-based ICU patient outcome prediction under simulated, varying levels of data scarcity. Two patient cohorts were used in this study: (1) eCritical, a multicenter ICU data from 55,689 unique admission records from 48,672 unique patients admitted to 15 medical-surgical ICUs in Alberta, Canada, between March 2013 and December 2019, and (2) Medical Information Mart for Intensive Care III, a single-center, publicly available ICU data set from Boston, Massachusetts, acquired between 2001 and 2012 containing 61,532 admission records from 46,476 patients. We compared DA and ITL models with baseline models (without TL) of fully connected neural networks, logistic regression, and lasso regression in the prediction of 30-day mortality, acute kidney injury, ICU length of stay, and hospital length of stay. Random subsets of training data, ranging from 1% to 75%, as well as the full data set, were used to compare the performances of DA and ITL with the baseline models at various levels of data scarcity. Overall, the ITL models outperformed the baseline models in 55 of 56 comparisons (all P values <.001). The DA models outperformed the baseline models in 45 of 56 comparisons (all P values <.001). ITL resulted in better performance than DA in terms of the number of times and the margin with which it outperformed the baseline models. In 11 of 16 cases (8 of 8 for ITL and 3 of 8 for DA), TL models outperformed baseline models when trained using 1% data subset. TL-based ICU patient outcome prediction models are useful in data-scarce scenarios. The results of this study can be used to estimate ICU outcome prediction performance at different levels of data scarcity, with and without TL. The publicly available pretrained models from this study can serve as building blocks in further research for the development and validation of models in other ICU cohorts and outcomes.

Deep learning has emerged as a powerful tool in various domains, revolutionising machine learning research. However, one persistent challenge is the scarcity of labelled training data, which hampers the performance and generalisation of deep learning models. To address this limitation, researchers have developed innovative methods to overcome data scarcity and enhance deep model learning capabilities. Two prevalent techniques that have gained significant attention are transfer learning and self-supervised learning. Transfer learning leverages knowledge learned from pre-training on a large-scale dataset, such as ImageNet, and applies it to a target task with limited labelled data. This approach allows models to benefit from the learned representations and effectively transfer knowledge to new tasks, resulting in improved learning performance and generalisation. On the other hand, self-supervised learning focuses on training models using pretext tasks that do not require manual annotation, allowing them to learn valuable representations from large amounts of unlabelled data. These learned representations can then be fine-tuned for downstream tasks, mitigating the need for extensive labelled data. In recent years, transfer and self-supervised learning have found applications in various fields, including medical image processing, video recognition, and natural language processing. These approaches have demonstrated remarkable achievements, enabling breakthroughs in areas such as disease diagnosis, object recognition, and language understanding. However, while these methods offer numerous advantages, they also have limitations. For example, transfer learning may face domain mismatch issues between the pre-training and target domains, while self-supervised learning requires careful design of pretext tasks to ensure meaningful representations. This review paper explores the recent applications of these pre-training methods in various fields within the past three years. It delves into the advantages and limitations of each approach, assesses the performance of models employing these techniques, and identifies potential directions for future research. By providing a comprehensive review of current pre-training methods, this article offers guidance for selecting the best technique for specific deep learning applications to address the data scarcity issue.

Medical Visual Question Answering (MedVQA) is a dual research field that intersects medical imaging and natural language processing, for better interpretability and accessibility of medical image data.Medical image quality is paramount for accurate diagnostics and subsequent medical visual question answering (MedVQA) tasks. This research focuses on applying and then analyzing results of different denoising methods on VQA-RAD, Medical VQA dataset to enhance quality of images. This study explores effectiveness of different traditional and deep learning based methods to reduce noise within medical images, thereby improving the accuracy and reliability of MedVQA task. We applied different traditional denoising filtering methods such as, gaussian filter, median filter, average filter, bilateral filter and convolutional autoencoder (CAE) based on deep learning on a VQA-RAD dataset to compare effectiveness of each denoising methods to improve image quality. Through comprehensive experiments and evaluations, this paper demonstrates that the convolutional autoencoder is potentially enhancing quality of medical images with an emphasis on preserving essential diagnostic information while suppressing unwanted noise with compare to other traditional denoising filters. The denoised images are then employed as input to improve accuracy for MedVQA tasks. The results of this research will help in optimizing medical imaging pipelines, ultimately benefiting clinical decision-making and healthcare outcomes.

Short text matching is a key problem in natural language processing (NLP), which can be applied in journalism, military, and other fields. In this paper, we propose an optimized Chinese short text matching algorithm based on external knowledge (OTE). OTE can effectively eliminate semantic ambiguity in Chinese text by integrating the HowNet external knowledge base. We use SoftLexicon to optimize the word lattice graph to provide more comprehensive multi-granularity information and integrate the LaserTagger model and EDA for data augmentation. Experimental results show that OTE has an average accuracy improvement of 1.5% in three datasets compared with existing models.

Natural Language Processing (NLP) involves the use of algorithms and models and various computational techniques to analyze, process, and generate natural language data, including speech and text. NLP helps computers interact with humans in a more natural way, which has become increasingly important as more humancomputer interactions take place. NLP allows machines to process and analyze voluminous unstructured data, including social media posts, newspaper articles, reviews from customers, emails, and others. It helps organizations extract insights, automate tasks, and improve decision-making by enabling machines to understand and generate human-like language. A linguistic background is essential for understanding NLP. Linguistic theories and models help in developing NLU systems, as NLP specialists need to understand the structure and rules of language. NLU systems are organized into different components, including language modelling, parsing, and semantic analysis. NLU systems may be assessed through the use of metrics that includes measures like precision and recall, as well as indicators that convey meaningful information that include F1 score and others. Semantics and knowledge representation are central to NLU, as they involve understanding the meaning of words and sentences and representing this information in a way that machines can use. Approaches to knowledge representation include semantic networks, ontologies, and vector embeddings. Language modelling is an essential step in NLP that sees usage in applications like speech recognition, text generation, and text completion and also in areas such as machine translation. Ambiguity Resolution remains a major challenge in NLP, as language is often ambiguous and context-dependent. Some common applications of NLP include sentiment analysis, chatbots, virtual assistants, machine translation, speech recognition, text classification, text summarization, and information extraction. In this chapter, we show the applicability of a popular unsupervised learning technique, viz., clustering through K-Means. The efficiency provided by the K-Means algorithm can be improved through the use of an optimization loop. The prospects for NLP are promising, with an increasing demand for AI-powered language technologies in various industries, including healthcare, finance, and e-commerce. There is also a growing need for ethical and responsible AI systems that are transparent and accountable.