Emotion Detection from Text via Ensemble Classification Using Word Embeddings

Vision-Language Pretrained (VLP) models have achieved impressive performance on multimodal tasks, including text-image retrieval, based on dense representations. Meanwhile, Learned Sparse Retrieval (LSR) has gained traction in text-only settings due to its interpretability and efficiency with fast term-based lookup via inverted indexes. Inspired by these advantages, recent work has extended LSR to the multimodal domain. However, these methods often rely on computationally expensive contrastive pre-training, or distillation from a frozen dense model, which limits the potential for mutual enhancement. To address these limitations, we propose a simple yet effective framework that enables bi-directional learning between dense and sparse representations through Self-Knowledge Distillation. This bi-directional learning is achieved using an integrated similarity score-a weighted sum of dense and sparse similarities-which serves as a shared teacher signal for both representations. To ensure efficiency, we fine-tune the final layer of the dense encoder and the sparse projection head, enabling easy adaptation of any existing VLP model. Experiments on MSCOCO and Flickr30k demonstrate that our sparse retriever not only outperforms existing sparse baselines, but also achieves performance comparable to-or even surpassing-its dense counterparts, while retaining the benefits of sparse models.

Dense feature representations are recently popular for human action recognition based on video. They achieve better performance than sparse local representations. However, dense representations usually lead to high-dimensional data which contain plenty of redundancy information. In order to solve this issue, a novel compact representation based on global Gist feature and 2DPCA-2DLDA is proposed. Firstly Global grid based Gist features are extracted from image sequences and described as Gist matrices. The Gist matrices reflect global structure distribution of local grids, and at the same time depict frequency characteristics of the local grids. Secondly, bilateral dimensionality reduction is developed to our proposed Gist matrices. The most discriminative spatial structure and frequency characteristic of local grids are separately selected with 2DPCA and 2DLDA. Finally, the derived structure and frequency properties are simultaneously embedded into a low dimensional compact global descriptor. Experimental results on UCF Sports dataset validate that the proposed method makes a good compromise between feature discrimination and dimensions. And the action recognition accuracy is promising.

In this paper, we propose an effective saliency detection method based on dense and sparse representation in-terms of an optimized background template. Firstly, the input image is divided into compact and uniform super-pixels. Then, the optimized background template is produced by introducing boundary conductivity measurement to improve the dense and sparse representation of the image's super-pixels in terms of the optimized background, where the reconstruction error represents a saliency measure. Based on the optimized template, two saliency maps are generated by dense and sparse representation. Finally, the Bayesian framework used to integrate the two saliency maps to obtain the final one. Experimental results show that the proposed method performs favorably against eight state-of-the-art methods. In addition, the proposed method is shown to be more effective in highlighting the challenging salient objects that touch the image boundary.

Humans are accustomed to autonomously associating prior knowledge with the text in a query when answering questions. However, for machines lacking cognition and common sense, a query is merely a combination of some words. Although we can enrich the semantic information of the given query through language representation or query expansion (QE) , the information contained in the query is still insufficient. In this paper, we propose an effective passage retrieval method named query context expansion-based retrieval (QCER) for open-domain question answering (OpenQA) . QCER associates a query with domain information by adding contextual association information based on the pseudo-relevance feedback (PRF) . QCER uses a dense reader to select top-n expansion terms for QE. We implement QCER by appending reader predictions, theoretically present in candidate passages, as contextual information to the initial query to form the new query. QCER with sparse representations (BM25) can improve retrieval efficiency and accelerate query convergence so that the reader can find the desired answer using fewer relevant passages, e.g., 10 passages, as soon as possible. Moreover, QCER can be easily combined with dense passage retrieval (DPR) to achieve even better performance, as sparse and dense representations are often complementary. Remarkably, we demonstrate that QCER achieves state-of-the-art performance in three tasks, passage retrieval, passage reading, and passage reranking, on the Natural Questions (NQ) and TriviaQA (Trivia) datasets under an extractive QA setup.

A disability is a significant problem that has posed and proceeds to pose a challenge. Disability is frustrating because it is noted as a constraint, mental, physical, and cognitive handicap, which prevents the individual’s involvement and growth. Therefore, significant effort is brought into eliminating these types of restrictions. These plans deal with the problems that disabled people face. People with disabilities are frequently required to depend on others to fulfil their needs. Machine learning (ML) is outshining in making smart cities and providing a protected environment for disabled people. Emotional detection is an essential field of study that presents numerous recognized inputs. Emotion is phrased differently through facial and speech gestures, expressions, and written medium. Emotion detection in a text document is a content-based classification task using deep learning (DL) techniques, intricate methods, and natural language processing (NLP). This study proposes a Novel Hybrid Attention-Based Deep Learning for Textual Emotion Recognition Using Natural Language Processing Technologies (HADLTER-NLPT) technique. The HADLTER-NLPT technique aims to recognize emotions from textual data, improving assistive technologies and emotional understanding for disabled persons. Initially, the HADLTER-NLPT model performs text pre-processing at different levels to clean and normalize the input text. The Word2Vec model converts the textual data into dense vector representations that capture semantic meaning for the word embedding process. Furthermore, the hybrid attention-based long short-term memory (HA-LSTM) classifier effectively recognizes emotional expressions from text. The oscillating chaotic sunflower optimization (OCSFO) approach is employed for hyperparameter tuning to optimize the performance of the HA-LSTM approach. An extensive experimental study is performed on the HADLTER-NLPT method under Emotion detection from the text dataset. The performance validation of the HADLTER-NLPT method portrayed a superior accuracy value of 98.86% over existing models.

BackgroundElectronic Medical Record (EMR) comprises patients’ medical information gathered by medical stuff for providing better health care. Named Entity Recognition (NER) is a sub-field of information extraction aimed at identifying specific entity terms such as disease, test, symptom, genes etc. NER can be a relief for healthcare providers and medical specialists to extract useful information automatically and avoid unnecessary and unrelated information in EMR. However, limited resources of available EMR pose a great challenge for mining entity terms. Therefore, a multitask bi-directional RNN model is proposed here as a potential solution of data augmentation to enhance NER performance with limited data.MethodsA multitask bi-directional RNN model is proposed for extracting entity terms from Chinese EMR. The proposed model can be divided into a shared layer and a task specific layer. Firstly, vector representation of each word is obtained as a concatenation of word embedding and character embedding. Then Bi-directional RNN is used to extract context information from sentence. After that, all these layers are shared by two different task layers, namely the parts-of-speech tagging task layer and the named entity recognition task layer. These two tasks layers are trained alternatively so that the knowledge learned from named entity recognition task can be enhanced by the knowledge gained from parts-of-speech tagging task.ResultsThe performance of our proposed model has been evaluated in terms of micro average F-score, macro average F-score and accuracy. It is observed that the proposed model outperforms the baseline model in all cases. For instance, experimental results conducted on the discharge summaries show that the micro average F-score and the macro average F-score are improved by 2.41% point and 4.16% point, respectively, and the overall accuracy is improved by 5.66% point.ConclusionsIn this paper, a novel multitask bi-directional RNN model is proposed for improving the performance of named entity recognition in EMR. Evaluation results using real datasets demonstrate the effectiveness of the proposed model.

In this paper we focus on the problem of automatically detecting the absence of the fastening bolts that secure the rails to the sleepers. The proposed visual inspection system uses images acquired from a digital line scan camera installed under a train. The general performances of the system, in terms of speed and detection rate, are mainly influenced by the adopted features for representing images and by their number. In this paper we use overcomplete dictionaries of waveforms, called frames, which allow dense and sparse representations of images and analyze the performances of the system with respect to the sparsity of the representation. Sparse means a representation with only few no vanishing components. In particular we show that, in the case of Gabor dictionaries, dense representations provide the highest detection rate. Moreover, the number of no vanishing components of 1% of the total reduces of 10% the detection rate of the system, indicating that very sparse representations do not heavily influence the performances. We show the adopted techniques by using images acquired in real experimental conditions.

Automatic emotion detection is a key task in human machine interaction,where emotion detection makes system more natural. In this paper, we propose an emotion detection using deep learning algorithm. The proposed algorithm uses end to end CNN. To increase computational efficiency of the deep network, we make use of trained weight parameters of the MobileNet to initialize the weight parameters of our system. To make our system independent of the input image size, we place global average pooling layer On top of the last convolution layer of it. Proposed system is validated for emotion detection using two benchmark datasets viz. Cohn–Kanade+ (CK+) and Japanese female facial expression (JAFFE). The experimental results show that the proposed method outperforms the other existing methods for emotion detection.

At the core of any inference procedure, deep neural networks are dot product operations, which are the component that requires the highest computational resources. For instance, deep neural networks, such as VGG-16, require up to 15-G operations in order to perform the dot products present in a single forward pass, which results in significant energy consumption and thus limits their use in resource-limited environments, e.g., on embedded devices or smartphones. One common approach to reduce the complexity of the inference is to prune and quantize the weight matrices of the neural network. Usually, this results in matrices whose entropy values are low, as measured relative to the empirical probability mass distribution of its elements. In order to efficiently exploit such matrices, one usually relies on, inter alia, sparse matrix representations. However, most of these common matrix storage formats make strong statistical assumptions about the distribution of the elements; therefore, cannot efficiently represent the entire set of matrices that exhibit low-entropy statistics (thus, the entire set of compressed neural network weight matrices). In this paper, we address this issue and present new efficient representations for matrices with low-entropy statistics. Alike sparse matrix data structures, these formats exploit the statistical properties of the data in order to reduce the size and execution complexity. Moreover, we show that the proposed data structures can not only be regarded as a generalization of sparse formats but are also more energy and time efficient under practically relevant assumptions. Finally, we test the storage requirements and execution performance of the proposed formats on compressed neural networks and compare them to dense and sparse representations. We experimentally show that we are able to attain up to ×42 compression ratios, ×5 speed ups, and ×90 energy savings when we lossless convert the state-of-the-art networks, such as AlexNet, VGG-16, ResNet152, and DenseNet, into the new data structures and benchmark their respective dot product.

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations. It aims to provide effective, reproducible, and easy-to-use first-stage retrieval in a multi-stage ranking architecture. Our toolkit is self-contained as a standard Python package and comes with queries, relevance judgments, pre-built indexes, and evaluation scripts for many commonly used IR test collections. We aim to support, out of the box, the entire research lifecycle of efforts aimed at improving ranking with modern neural approaches. In particular, Pyserini supports sparse retrieval (e.g., BM25 scoring using bag-of-words representations), dense retrieval (e.g., nearest-neighbor search on transformer-encoded representations), as well as hybrid retrieval that integrates both approaches. This paper provides an overview of toolkit features and presents empirical results that illustrate its effectiveness on two popular ranking tasks. Around this toolkit, our group has built a culture of reproducibility through shared norms and tools that enable rigorous automated testing.

Removing constraints of 4D-STEM with a framework for event-driven acquisition and processing.

In Romanian language there are some resources for automatic text comprehension, but for Emotion Detection, not lexicon-based, there are none. To cover this gap, we extracted data from Twitter and created the first dataset containing tweets annotated with five types of emotions: joy, fear, sadness, anger and neutral, with the intent of being used for opinion mining and analysis tasks. In this article we present some features of our novel dataset, and create a benchmark to achieve the first supervised machine learning model for automatic Emotion Detection in Romanian short texts. We investigate the performance of four classical machine learning models: Multinomial Naive Bayes, Logistic Regression, Support Vector Classification and Linear Support Vector Classification. We also investigate more modern approaches like fastText, which makes use of subword information. Lastly, we fine-tune the Romanian BERT for text classification and our experiments show that the BERT-based model has the best performance for the task of Emotion Detection from Romanian tweets. Keywords: Emotion Detection, Twitter, Romanian, Supervised Machine Learning

Research and education in machine learning requires diverse, representative, and open datasets that contain sufficient samples to handle the necessary training, validation, and testing tasks. Currently, the Recommender Systems area includes a large number of subfields in which accuracy and beyond-accuracy quality measures are continuously being improved. To feed this research variety, it is both necessary and convenient to reinforce the existing datasets with synthetic ones. This paper proposes a Generative Adversarial Network (GAN)-based method to generate collaborative filtering datasets in a parameterized way by selecting their preferred number of users, items, samples, and stochastic variability. This parameterization cannot be performed using regular GANs. Our GAN model is fed with dense, short, and continuous embedding representations of items and users, instead of sparse, large, and discrete vectors, to ensure fast and accurate learning, as compared to the traditional approach based on large and sparse input vectors. The proposed architecture includes a DeepMF model to extract the dense user and item embeddings and a clustering process to convert the dense GAN generated samples to the discrete and sparse samples necessary to create each required synthetic dataset. The results from three different source datasets show adequate distributions and expected quality values and evolutions in the generated datasets compared to the source datasets. Synthetic datasets and source codes are available to researchers.

Effective urban planning, city digital twins, and informed policy formulation rely heavily on precise building use information. While existing research often focuses on broad categories of building use, there is a noticeable gap in the classification of buildings’ detailed use. This study addresses this gap by concurrently extracting both broad and detailed hierarchical information regarding building use. Our approach involves leveraging multiple data sources, including high spatial resolution remote sensing images (RS), digital surface models (DSM), street view images (SVI), and textual information from point of interest (POI) data. Given the complexity of mixed-use buildings, where different functions coexist, we treat building hierarchical use classification as a multi-label task, determining the presence of specific categories within a building. To maximize the utility of features across diverse modalities and their interrelationships, we introduce a novel multi-label multimodal Transformer-based feature fusion network. This network can simultaneously predict four broad categories and thirteen detailed categories, representing the first instance of utilizing these four modalities for building use classification. Experimental results demonstrate the effectiveness of our model, achieving a weighted average F1 score (WAF) of 91% for broad categories, 77% for detailed categories, and 84% for hierarchical categories. The macro average F1 scores (MAF) are 81%, 48%, and 56%, respectively. Ablation experiments highlight RS data as the cornerstone for hierarchical building use classification. DSM and POI provide slight supplementary information, while SVI data may introduce more noise than effective information. Our analysis of hierarchy consistency, supplementary, and exclusiveness between broad and detailed categories shows our model can effectively learn these relations. We compared two ways to obtain broad categories: classifying them directly and scaling up detailed categories, associating them with their broad counterparts. Experiments show that the WAF and MAF of the former are 3.8% and 6% higher than the latter. Notably, our research visualizes attention models for different modalities, revealing the synergy among them. Despite the model’s emphasis on SVI and POI data, the critical role of RS and DSM in building hierarchical use classification is underscored. By considering hierarchical use categories and accommodating mixed-use scenarios, our method provides more accurate and comprehensive insights into land use patterns.

Very often in articles on monomodal human-machine-interaction (HMI) it is pointed out that the results can strongly be improved if other modalities are taken into account. In this contribution we look at two different problems in HMI: the detection of emotion or user state and the question whether the user is currently interacting with the machine, himself or another person (On/Off-Focus). We present monomodal classification results for these two problems and discuss whether multimodal classification seems to be promising for the respective problem. Different fusion models are considered. The examples are taken from the German HMI projects "SmartKom" and "SmartWeb".