A language-independent hybrid approach for multi-word expression extraction

We present SimpleApprenant, a platform aiming to improve French L2 learners’ knowledge of Multi Word Expressions (MWEs). SimpleApprenant integrates an MWE database annotated with the Common European Framework of Reference for languages (CEFR) level and several Natural Language Processing (NLP) tools: a spelling checker, a parser, and a set of transformation rules. NLP tools and resources are used to build training and writing exercises to improve MWE knowledge and writing skills of French L2 learners. We present the user scenarios, the platform’s architecture, as well as the preliminary evaluation of its NLP tools.

Machine Translation (MT) systems are now being improved with the use of an ongoing methodology known as Neural Machine Translation (NMT). Natural language processing (NLP) researchers have shown that NMT systems are unable to deal with out-of-vocabulary (OOV) words and multi-word expressions (MWEs) in the text. OOV terms are those that are not currently included in the vocabulary that is used by the NMT system. MWEs are phrases that consist of a minimum of two terms but are treated as a single unit. MWEs have great importance in NLP, linguistic theory, and MT systems. In this article, OOV words and MWEs are handled for the Punjabi to English NMT system. A parallel corpus for Punjabi to English containing MWEs was developed and used to train the different models of NMT. Punjabi is a low-resource language as it lacks the availability of a large parallel corpus for building various NLP tools, and this is an attempt to improve the accuracy of Punjabi in the English NMT system by using named entities and MWEs in the corpus. The developed NMT models were assessed using human evaluation through adequacy, fluency and overall rating as well as automated assessment tools such as the bilingual evaluation study (BLEU) and translation error rate (TER) score. Results show that using word embedding (WE) and MWEs corpus increased the accuracy of translation for the Punjabi to English language pair. The best BLEU score obtained was 15.45 for the small test set, 43.32 for the medium test set, and 34.5 for the large test set, respectively. The best TER rate score obtained was 57.34% for the small test set, 37.29% for the medium test set, and 53.79% for the large test set, repectively.

<span lang="EN-US">In the realm of natural language processing (NLP), a diverse array of language models has emerged, catering to a wide spectrum of tasks, ranging from speaker recognition and auto-correction to sentiment analysis and stock prediction. The significance of language models in enabling the execution of these NLP tasks cannot be overstated. This study proposes an approach to enhance accuracy by leveraging a hybrid language model, combining the strengths of long short-term memory (LSTM) and gated recurrent unit (GRU). LSTM excels in preserving long-term dependencies in data, while GRU's simpler gating mechanism expedites the training process. The research endeavors to evaluate four variations of this hybrid model: LSTM, GRU, <a name="_Hlk158992630"></a>bidirectional long short-term memory (Bi-LSTM), and a combination of LSTM with GRU. These models are subjected to rigorous testing on two distinct datasets: one focused on IBM stock price prediction, and the other on Jigsaw toxic comment classification (sentiment analysis). This work represents a significant stride towards democratizing NLP capabilities, ensuring that even in resource-constrained settings, NLP models can exhibit improved performance. The anticipated implications of these findings span a wide spectrum of real-world applications and hold the potential to stimulate further research in the field of NLP. </span>

Parts of Speech (pos) tagging corresponds to techniques of tagging accurate pos tags of each word of a sentence. It is a pivotal and indispensable part of sentiment analysis, synthesizing text to speech, analyzing textual data and many more Natural Language Processing (NLP) tasks. Though there are many researches done in the field of languages like English, Chinese etc., it is still comparatively unexplored for Bangla. An excellent pos tagger in Bangla language can accelerate NLP of Bangla textual data. In this study, we have developed a novel hybrid Deep Learning (DL) model with two parallel input networks with Convolutional Neural Network (CNN), and Bidirectional Long Short Term Memory (BiLSTM), and an output network of CNN. CNN works well regarding the local dependencies of words in sentences, and BiLSTM works well with the global dependencies of words regarding the context in a sentence. No other work has been done with local features of textual data along with long term word dependencies in Bangla pos tagging studies. The model can work with any external Word Embeddings. In this study, it is implemented with both Embedding Layer and Word2Vec layer. In the evaluation process, Mathews Correlation Coefficient (MCC), Precision-Recall curve are used along with accuracy and F1-score due to the dataset's imbalanced characteristic. Our proposed model accompanied by Word2Vec layer has outperformed Vanilla Recurrent Neural Network (RNN), Gated Recurrent Units (GRU), Long Short Term Memory (LSTM), BiLSTM networks, and also one recent study using the same dataset with the highest accuracy of 0.974, F1-score of 0.883, and MCC value of 0.936.

The COVID-19 pandemic has had a devastating effect on many people, creating severe anxiety, fear, and complicated feelings or emotions. After the initiation of vaccinations against coronavirus, people's feelings have become more diverse and complex. Our aim is to understand and unravel their sentiments in this research using deep learning techniques. Social media is currently the best way to express feelings and emotions, and with the help of Twitter, one can have a better idea of what is trending and going on in people's minds. Our motivation for this research was to understand the diverse sentiments of people regarding the vaccination process. In this research, the timeline of the collected tweets was from December 21 to July21. The tweets contained information about the most common vaccines available recently from across the world. The sentiments of people regarding vaccines of all sorts were assessed using the natural language processing (NLP) tool, Valence Aware Dictionary for sEntiment Reasoner (VADER). Initializing the polarities of the obtained sentiments into three groups (positive, negative, and neutral) helped us visualize the overall scenario; our findings included 33.96% positive, 17.55% negative, and 48.49% neutral responses. In addition, we included our analysis of the timeline of the tweets in this research, as sentiments fluctuated over time. A recurrent neural network- (RNN-) oriented architecture, including long short-term memory (LSTM) and bidirectional LSTM (Bi-LSTM), was used to assess the performance of the predictive models, with LSTM achieving an accuracy of 90.59% and Bi-LSTM achieving 90.83%. Other performance metrics such as precision,, F1-score, and a confusion matrix were also used to validate our models and findings more effectively. This study improves understanding of the public's opinion on COVID-19 vaccines and supports the aim of eradicating coronavirus from the world.

In natural language processing, multiword expressions (MWEs) play a significant role in understanding the context and meaning of a sentence. A MWE comprises two or more words that are handled as if they were one. MWE has the property that the constituent words are consistent and are often used in related contexts. Hindi is used as a case study. We employed three properties: linguistic or syntactical pattern, a relationship between constituent words, and context similarity and proposed a three-phase hybrid approach to extract MWE from unstructured Hindi text. Experimental analysis and comparison of results on the TDIL dataset show the superiority of the proposed hybrid method over the context-based method and association-based methods.

Applications in the World Wide Web aggregate vast amounts of information from different data sources. The aggregation process is often implemented with Extract, Transform and Load (ETL) processes. Usually ETL processes require information for aggregation available in structured formats, e. g. XML or JSON. In many cases the information is provided in natural language text which makes the application of ETL processes impractical. Due to the fact that information is provided in natural language, Information Extraction (IE) systems have been evolved. They make use of Natural Language Processing (NLP) tools to derive meaning from natural language text. State-of-the-art NLP tools apply Machine Learning methods. These NLP tools perform on newspapers with good quality, but they drop accuracy in other domains. However, to improve the quality for IE systems in specific domains often NLP tools are trained on domain specific text which is a time consuming process. This paper introduces an approach using a Continuous Integration pipeline for organizing and monitoring the annotation process on domain specific corpora.

This study examines the efficacy of three distinct machine learning models for sentiment analysis: Bidirectional Long Short-Term Memory (Bi-LSTM), Long Short-Term Memory (LSTM), and Logistic Regression. One of the most important tasks in natural language processing is sentiment analysis, which entails categorizing reviews as positive or negative. Fifty thousand sentiment-labeled movie reviews make up the dataset. To determine the most precise and effective technique for sentiment categorization, we put these models into practice and evaluated their respective performances. The highest accuracy of 89.42% is achieved by Logistic Regression, with precision of 88.35%, recall of 91.01%, and F1-score of 89.66%. The LSTM model, well-known for capturing temporal dependencies in text, obtained an accuracy of 86.23%, precision of 89.60%, recall of 82.22%, and an F1-score of 85.75%. The Bi-LSTM model, which processes input sequences in both forward and backward directions to better capture context, demonstrated accuracy of 87.65%, precision of 88.67%, recall of 86.54%, and an F1-score of 87.60%. The results show that although Logistic Regression performs better in accuracy, the Bi-LSTM model offers a substantial trade-off between precision and recall, making it a viable option for sentiment analysis applications. Despite its minor accuracy lag, the LSTM model provides important insights into the sequential relationships of the text data.

The US Food and Drug Administration Adverse Event Reporting System (FAERS) contains over 24 million individual case safety reports (ICSRs). In this research project, we evaluated a natural language processing (NLP) tool’s ability to extract four demographic variables (gender, weight, ethnicity, race) from ICSR narratives. Specificity of the NLP algorithm was over 94% for all demographics, while sensitivity varied between the demographics: 98.6% (gender), 45.5% (weight), 100% (ethnicity), and 85.3% (race). Among ICSRs missing weight, ethnicity, and race in the structured field, few cases had this information in the narrative (>95% missing); consequently, the positive predictive value (PPV) for these three demographics had wide 95% confidence intervals. After NLP implementation, the total number of ICSRs missing gender was reduced by 33% (i.e., NLP identified 472 thousand reports having a gender value in the narrative that was not in the structured field), while the total number of ICSRs missing weight, ethnicity, or race was reduced by less than 4%. This study demonstrated that the implementation of an NLP tool can provide meaningful improvements in the availability of gender information for pharmacovigilance activities conducted with FAERS data. In contrast, NLP tools targeting the extraction of weight, ethnicity, or race from free-text fields have minimal impact largely because the information was infrequently provided by the reporter. Further gains in completeness of these fields must originate from increases in provision of demographic information from the reporter rather than informatic solutions.

Word embeddings are essential to natural language processing tasks because they contain a single word’s syntactic and semantic information. Word embeddings have been developed widely for numerous spoken languages across the globe like English. The research community needs to pay more attention to the Urdu language despite its significant number of speakers, which amounts to approximately 231.3 million individuals. Urdu is a complex language because word boundaries in Urdu are unspecified, as it does not employ delimiters between words. The compound word, a multiword expression, is a more complex word consisting of many strings or independent base words. Traditionally, compound words are identified during the word segmentation using bigram or trigram approaches. The challenge with these techniques is that they do not produce meaningful words. This study uses morphological rule-based compound words in Urdu text documents. For text representation, a self-trained morphological rule-based compound word embedding (Cword2vec) based on the word2vec model is proposed for Urdu text sentiment analysis. The performance of self-trained morphological rule-based compound word embedding was then evaluated using four well-known deep learning models, i.e., long short-term memory (LSTM), bidirectional LSTM (BiLSTM), convolutional neural networks (CNN), and convolutional LSTM (C-LSTM) for sentiment analysis. We also compare the performance of morphological rule-based compound words with traditional compound word identification techniques such as bigrams and trigrams. Regardless of the classification model, word embedding using our proposed morphological rule-based compound words outperformed in terms of precision, recall, F1 score, and accuracy than bigrams and trigrams.

Text simplification is a vital work for comprehending patent claims due to its complex syntactic structures and lengthy sentences. Therefore, almost all patent analysis practitioners cannot be able to directly and intuitively understand patent essence even through some common natural language processing (NLP) tools are applied to parse these patent claim paragraph or sentences. Universal text analysis tools above is almost useless, or even crashed when applied to some complex paragraphs of patent claims. Therefore, it is necessary to propose a patent text oriented simplification approach to help patent researchers grasp the essence of patent quickly and intuitively. Motivated by the above reason, we in this article propose a simplification method based on deep learning to segment patent claim into shorter and comprehensible sentences for downstream tasks of patent analysis. The proposed approach contains two stages: on one stage, we use a machine learning approach of conditional random field (CRF) to decompose syntactically complex paragraphs into coarse‐grained level sentences with simplified structures and complete semantics; on another stage, a deep Learning architecture of bidirectional long‐short term memory (Bi‐LSTM)‐CRF is applied to segment coarse‐grained and lengthy sentences of former stage into fined‐grained and shorter sentences. Compared with a series of baselines, our patent segmentation architecture based on deep learning of Bi‐LSTM‐CRF achieves higher performance than any other methods on the evaluation measures of precision, recall, and F1.

Medical named entity recognition based on dilated convolutional neural network

Text classification is a fundamental task in natural language processing (NLP). Context semantics can greatly improve the accuracy of text classification tasks. Although there are some popular methods in obtaining semantics, current context semantic analysis techniques, due to limited accuracy, are still a great bottleneck for text classification. This paper introduces a novel model, the densely connected Bidirectional LSTM with Max-pooling of CNN network (Dense-BiLSTM-MP), which greatly enhances the context of semantic information. In this model, a densely connected bidirectional long short-term memory (BiLSTM) model, as well as multiple max-pooling layers of convolutional network, are applied to obtain an increasingly enhanced assessment of context, and extract the key features, respectively. Experiments were conducted on four public datasets: YELP, 20NewsGroup, THUNews and AG. The experimental results show that the proposed model outperforms state of the art methods on several datasets. Furthermore, discussions on the Dense-BiLSTM-MP model’s performance in short texts and long texts were given, respectively.

Authorship attribution is a critical task in natural language processing that involves identifying the author of a given text based on writing style, linguistic patterns, and structural features. This research presents a deep learning-based approach combining Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) networks to accurately attribute authorship. Using the Reuters-50-50 dataset, we extract syntactic and structural information such as part-of-speech tags, punctuation frequency, and average sentence length, which help capture the unique stylistic traits of individual authors. The text is cleaned, transformed into numerical vectors, and used to train the proposed model. Experimental results demonstrate that the hybrid CNN-BiLSTM architecture achieves high accuracy of 96% in identifying authors from unseen text samples. The model also performs well across other metrics such as precision, recall, and F1-score, showing its robustness and effectiveness in capturing deep textual patterns. This work contributes to the fields of authorship verification, plagiarism detection, and digital forensics, offering a scalable and reliable solution for text-based author identification.

In the ever-expanding digital landscape, the abundance of user-generated content on consumer platforms such as Booking and TripAdvisor offers a rich source of information for both travellers and hoteliers. Sentiment analysis, a fundamental research task of natural language processing (NLP) is used for mining sentiments and opinions within this vast reservoir of text reviews. A more specific type of sentiment analysis, i.e., aspect-based sentiment analysis (ABSA), is used when processing customer reviews is required. In ABSA, we aim to capture aspect-level sentiments and intricate relationships between various aspects within reviews. This article proposes a novel approach to ABSA by introducing a novel technique of word sense disambiguation (WSD) and integrating it with the Transformer architecture bidirectional encoder representations from Transformers (BERT) and graph convolutional networks (GCNs). The proposed approach resolves the intriguing ambiguities of the words and represents the review data as a complex graph structure, facilitating the modeling of intricate relationships between different aspects. The combination of bidirectional long short-term memory (BiLSTM) and GCN proves effective in capturing inter-dependencies among various aspects, providing a nuanced understanding of customer sentiments. The experiments are conducted on the RABSA dataset (an enhanced and richer hotel review data collection), and results demonstrate that our approach outperforms previous baselines, showcasing the effectiveness of integrating WSD in ABSA. Furthermore, an ablation study confirms the significant contribution of the WSD module to the overall performance. Moreover, we explore different similarity measures and find that cosine similarity yields the best results when identifying the real sense of a word in a given sentence using WordNet. The findings of our work and future work related to our work create lots of interest for people in the tourism and hospitality industry. This research gives another boost to the concept of the potential of NLP techniques in sentiment analysis. It emphasizes that if we combine the potential of NLP techniques along with state-of-the-art machine learning frameworks, we can shape the future of this field.