Sparse Coding Methods for Music Induced Emotion Recognition

Recently, some sparse coding methods with geometrical constraint have been proposed, in which local geometrical structure of the data points was preserved during sparse coding process. These methods have been applied to classification problems and gained much success. However, they failed to use label information which has been proved to be useful in supervised sparse coding and discriminant manifold learning. In this paper, we propose a discriminant sparse coding approach with geometrical constraint. Labels are used to learn an intrinsic graph and a penalty graph, and these graphs are then embedded into sparse coding framework as constraints. The local geometric structure within each class is preserved and the separability between different classes is enforced. As a result, the discrimination of sparse coding will be improved. Experiments on benchmark databases demonstrate the effectiveness of the proposed method.

Recognition of emotional content in music is an issue that arises recently. Music received by live applications are often exposed to noise, thus prone to reducing the recognition rate of the application. The solution proposed in this study is a robust music emotion recognition system for live applications. The proposed system consists of two major parts, i.e. subspace-based noise suppression and a hierarchical sparse representation classifier, which is based on sparse coding and a sparse representation classifier (SRC). The music is firstly enhanced by fast subspace based noise suppression. Nine classes of emotion are then used to construct a dictionary, and the vector of coefficients is obtained by sparse coding. The vector can be divided into nine parts, and each of which models a specific emotional class of a signal. Since the proposed descriptor can provide emotional content analysis of different resolutions for emotional music recognition, this work regards vectors of coefficients as feature representations. Finally, a sparse representation based classification method is employed for classification of music into four emotional classes. The experimental results confirm the highly robust performance of the proposed system in emotion recognition in live music.

BackgroundPatients with Alzheimer disease (AD) and amnesic mild cognitive impairment (aMCI) have deficits in emotion recognition. However, it has not yet been determined whether patients with AD and aMCI also experience difficulty in recognizing the emotions conveyed by music. This study was conducted to investigate whether musical emotion recognition is impaired or retained in patients with AD and aMCI.MethodsAll patients were recruited from the First Affiliated Hospital of Anhui Medical University between March 1, 2015 and January 31, 2017. Using the musical emotion recognition test, patients with AD (n = 16), patients with aMCI (n = 19), and healthy controls (HCs, n = 16) were required to choose one of four emotional labels (happy, sad, peaceful, and fearful) that matched each musical excerpt. Emotion recognition scores in three groups were compared using one-way analysis of variance (ANOVA) test. We also investigated the relationship between the emotion recognition scores and Mini-Mental State Examination (MMSE) using Pearson's correlation analysis test in patients with AD and aMCI.ResultsCompared to the HC group, both of the patient groups showed deficits in the recognition of fearful musical emotions (HC: 7.88 ± 1.36; aMCI: 5.05 ± 2.34; AD: 3.69 ± 2.02), with results of a one-way ANOVA confirming a significant main effect of group (F(2,50) = 18.70, P < 0.001). No significant differences were present among the three groups for the happy (F(2,50)=2.57, P = 0.09), peaceful (F(2,50) = 0.38, P = 0.09), or sad (F(2,50) = 2.50, P = 0.09) musical emotions. The recognition of fearful musical emotion was positively associated with general cognition, which was evaluated by MMSE in patients with AD and aMCI (r = 0.578, P < 0.001). The correlations between the MMSE scores and recognition of the remaining emotions were not significant (happy, r = 0.228, P = 0.11; peaceful, r = 0.047, P = 0.74; sad, r = 0.207, P = 0.15).ConclusionThis study showed that both patients with AD and aMCI had decreased ability to distinguish fearful emotions, which might be correlated with diminished cognitive function.

The development of computer technology, multimedia technology and communication technology has made the application of computers more and more widespread in people's lives. The next direction of computer development is to expect to be able to use computers to analyze, simulate and express human subjective thinking activities. As a part of people's lives, music has a strong emotional appeal. The stronger the musical emotion, the more audiences representing the song. Most of the computer technology related to music is still in the storage, tuning, compression, etc. of music, but with the development of computer analysis and automatic recognition technology, today's computer technology can already realize the cognition of music emotions through models and programs and creation. This article first briefly introduces computer analysis and automatic recognition technology, then analyzes and discusses the current representation model of music emotion recognition, and finally tries to design a music emotion recognition model design and proposes to introduce BP neural network into music emotion recognition technology to realize the automatic emotion analysis and recognition of the main music track.

With the development of video-sharing websites, P2P, micro-blog, mobile WAP websites, and so on, sensitive videos can be more easily accessed. Effective sensitive video recognition is necessary for web content security. Among web sensitive videos, this paper focuses on violent and horror videos. Based on color emotion and color harmony theories, we extract visual emotional features from videos. A video is viewed as a bag and each shot in the video is represented by a key frame which is treated as an instance in the bag. Then, we combine multi-instance learning (MIL) with sparse coding to recognize violent and horror videos. The resulting MIL-based model can be updated online to adapt to changing web environments. We propose a cost-sensitive context-aware multi- instance sparse coding (MI-SC) method, in which the contextual structure of the key frames is modeled using a graph, and fusion between audio and visual features is carried out by extending the classic sparse coding into cost-sensitive sparse coding. We then propose a multi-perspective multi- instance joint sparse coding (MI-J-SC) method that handles each bag of instances from an independent perspective, a contextual perspective, and a holistic perspective. The experiments demonstrate that the features with an emotional meaning are effective for violent and horror video recognition, and our cost-sensitive context-aware MI-SC and multi-perspective MI-J-SC methods outperform the traditional MIL methods and the traditional SVM and KNN-based methods.

Sparse Coding is a widely used method to represent an image. However, sparse coding and its improved algorithms have the problem of complex computation and long running time and so on. For these problems, we propose an image classification method based on hash codes and space pyramid, which encodes local feature points with hash codes instead of sparse coding. Firstly, extract the local feature points from the images. Second, learn binary auto-encoder hashing functions, which map the local feature points into hash codes. Third, perform binary k-means cluster on the binary hash codes and generate the binary visual vocabularies. Finally, Combine with spatial pyramid matching model, and represent the image by the histogram vector of space pyramid, which is used in image classification. Experimental results show that compared with other sparse coding methods, our method has the shorter time of learning vocabularies and faster encoder speed and higher classification accuracy.

In the recent years, there has been an increasing interest in using sparse representations for image processing and computer vision. The main reason behind their popularity is that they could provide a more robust and efficient way of reconstructing a target by means of a limited number of atoms in a dictionary. The common practice in sparse coding is to use dictionary atoms which live in Euclidean spaces. In recent years, some studies proposed to use region covariance based dictionary atoms to come up with more effective sparse coding schemes. The optimization schemes suggested by these studies are fundamentally different than those of the standard methods since covariance matrices live in a special Riemannian manifold. In this study, we propose to enrich such a sparse coding scheme proposed by Sivalingram et al. with a group sparsity constraint. The experimental results on a face recognition task reveals that considering group sparsity improves the recognition rate.

Music Emotion Recognition (MER) is a computational field of affective computing and audio signal processing. Although previous attempts used traditional machine learning algorithms, e.g., Support Vector Machines and k-Nearest Neighbors, for emotion classification in music, these methods are often challenged by the intricate temporal and spectral nature of sound signals, thereby constraining classification performance. To address this, the work proposes a new model, the Lion-Optimized CNN-BiGRU for Emotion Recognition (LOCBER), leveraging the strengths of Convolutional Neural Networks and Bidirectional Gated Recurrent Units for effective feature extraction and temporal sequence modeling. For additional performance optimization, LOCBER is optimized using the Lion Swarm Optimization (LSO) algorithm, which adjusts model parameters to minimize loss and achieve better accuracy and convergence. This study aims to improve music emotion recognition by integrating a CNN–BiGRU model with Lion Swarm Optimization, achieving 97.7% classification accuracy and outperforming conventional methods by 15.3%. LSO-based optimization achieved improved convergence and stability compared to traditional training processes. The LOCBER model benefits emotion-sensitive music recommendation systems, intelligent user interfaces, and music therapy applications. Experiments demonstrate that by fusion of state-of-the-art neural architectures with bio-inspired optimization, one can substantially improve the performance and validity of MER systems for real-world use.

The correlation structure of natural hand & finger movements suggests that their motion is controlled in a lower-dimensional space than would be possible given their mechanical nature. Yet, it is unclear whether this low dimensional embedding is relevant to how the brain represents motor actions and how we can decode it for Brain-Machine Interface applications. We collected large data set of natural hand movement kinematics and analysed it using a novel sparse coding and dictionary learning approach - Sparse Movement Decomposition (SMD), which captures the embedding of the data in terms of spatial and temporal structure. We show that our sparse codes over natural movement statistics give a more parsimonious representation than the simple correlation structure. This suggest that, like V1 neuron receptive fields can be predicted from sparse code over natural image statistics, motor control may be encoded in such a manner. We further show how our sparse coding can help understand the temporal structure of behaviour, and thus our technique may be used for behavioural fingerprinting in diagnostics and for more naturalistic neuroprosthetic control.

Sparse coding (SC) method has been shown to deliver successful result in a variety of computer vision applications. However, it does not consider the underlying structure of the data in the feature space. On the other hand, locality constrained linear coding (LLC) utilizes locality constraint to project each input data into its local-coordinate system. Based on the recent success of LLC, we propose a novel locality-constrained sparse coding (LSC) method to overcome the limitation of the SC. In experiments, the proposed algorithms were applied to head pose estimation applications. Experimental results demonstrated that the LSC method is better than state-of-the-art methods.

Emotion recognition of music objects is a promising and important research issues in the field of music information retrieval. Usually, music emotion recognition could be considered as a training/classification problem. However, even given a benchmark (a training data with ground truth) and using effective classification algorithms, music emotion recognition remains a challenging problem. Most previous relevant work focuses only on acoustic music content without considering individual difference (i.e., personalization issues). In addition, assessment of emotions is usually self-reported (e.g., emotion tags) which might introduce inaccuracy and inconsistency. Electroencephalography (EEG) is a non-invasive brain-machine interface which allows external machines to sense neurophysiological signals from the brain without surgery. Such unintrusive EEG signals, captured from the central nervous system, have been utilized for exploring emotions. This paper proposes an evidence-based and personalized model for music emotion recognition. In the training phase for model construction and personalized adaption, based on the IADS (the International Affective Digitized Sound system, a set of acoustic emotional stimuli for experimental investigations of emotion and attention), we construct two predictive and generic models \(AN\!N_1\) (“EEG recordings of standardized group vs. emotions”) and \(AN\!N_2\) (“music audio content vs. emotion”). Both models are trained by an artificial neural network. We then collect a subject’s EEG recordings when listening the selected IADS samples, and apply the \(AN\!N_1\) to determine the subject’s emotion vector. With the generic model and the corresponding individual differences, we construct the personalized model H by the projective transformation. In the testing phase, given a music object, the processing steps are: (1) to extract features from the music audio content, (2) to apply \(AN\!N_2\) to calculate the vector in the arousal-valence emotion space, and (3) to apply the transformation matrix H to determine the personalized emotion vector. Moreover, with respect to a moderate music object, we apply a sliding window on the music object to obtain a sequence of personalized emotion vectors, in which those predicted vectors will be fitted and organized as an emotion trail for revealing dynamics in the affective content of music object. Experimental results suggest the proposed approach is effective.

Hierarchical discriminating sparse coding for weak fault feature extraction of rolling bearings

In this paper, a sub-dictionary based sparse coding method is proposed for image representation. The novel sparse coding method substitutes a new regularization item for L1-norm in the sparse representation model. The proposed sparse coding method involves a series of sub-dictionaries. Each sub-dictionary contains all the training samples except for those from one particular category. For the test sample to be represented, all the sub-dictionaries should linearly represent it apart from the one that does not contain samples from that label, and this sub-dictionary is called irrelevant sub-dictionary. This new regularization item restricts the sparsity of each sub-dictionary's residual, and this restriction is helpful for classification. The experimental results demonstrate that the proposed method is superior to the previous related sparse representation based classification.

Music is a powerful force that evokes human emotions. Several investigations of music emotion recognition (MER) have been conducted in recent years. This paper proposes a system for detecting emotion in music that is based on a deep Gaussian process (GP). The system consists of two parts-feature extraction and classification. In the feature extraction part, five types of features that are associated with emotions are selected for representing the music signal; these are rhythm, dynamics, timbre, pitch and tonality. A music clip is decomposed into frames and these features are extracted from each frame. Next, statistical values, such as mean and standard deviation, of frame-based features are calculated to generate a 38-dimensional feature vector. In the classification part, a deep GP is utilized for emotion recognition. We treat classification problem from the perspective of regression. Finally, 9 classes of emotion are categorized by 9 one-versus-all classifiers. The experimental results demonstrate that the proposed system performs well in emotion recognition.

In this study, we address emotion recognition using unsupervised feature learning from speech data, and test its transferability to music. Our approach is to pre-train models using speech in English and Mandarin, and then fine-tune them with excerpts of music labeled with categories of emotion. Our initial hypothesis is that features automatically learned from speech should be transferable to music. Namely, we expect the intra-linguistic setting (e.g., pre-training on speech in English and fine-tuning on music in English) should result in improved performance over the cross-linguistic setting (e.g., pre-training on speech in English and fine-tuning on music in Mandarin). Our results confirm previous research on cross-domain transferability, and encourage research towards language-sensitive Music Emotion Recognition (MER) models.