Semi-supervised multi-label feature selection with adaptive structure learning and manifold learning

Unsupervised feature selection and NMF de-noising for robust Speech Emotion Recognition

Existing feature selection methods mainly target single-label learning and multi-label learning, and only a few algorithms are optimized for label distribution learning. In label distribution learning, the associated labels of each sample have different levels of importance. Therefore, multi-label feature selection algorithms cannot be directly applied to label distribution learning. Discretizing label distribution data into multi-label data will cause part of the supervision information to be lost. In most practical applications of label distribution learning, the feature space is undefined, and the features are in the form of flow features. To solve this problem, this paper applies fuzzy rough set theory and applies the flow feature framework to propose a dynamic label distribution feature selection algorithm that handles flow features. Experimental results show that the proposed method is more effective than six state-of-the-art feature selection algorithms on 12 datasets with respect to six representative evaluation metrics.

Feature selection for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures

With the increasing data dimensionality, feature selection has become a fundamental task to deal with high-dimensional data. Semi-supervised feature selection focuses on the problem of how to learn a relevant feature subset in the case of abundant unlabeled data with few labeled data. In recent years, many semi-supervised feature selection algorithms have been proposed. However, these algorithms are implemented by separating the processes of feature selection and classifier training, such that they cannot simultaneously select features and learn a classifier with the selected features. Moreover, they ignore the difference of reliability inside unlabeled samples and directly use them in the training stage, which might cause performance degradation. In this paper, we propose a joint semi-supervised feature selection and classification algorithm (JSFS) which adopts a Bayesian approach to automatically select the relevant features and simultaneously learn a classifier. Instead of using all unlabeled samples indiscriminately, JSFS associates each unlabeled sample with a self-adjusting weight to distinguish the difference between them, which can effectively eliminate the irrelevant unlabeled samples via introducing a left-truncated Gaussian prior. Experiments on various datasets demonstrate the effectiveness and superiority of JSFS.

Local positive and negative correlation-based k-labelsets for multi-label classification

In a conventional supervised learning paradigm, each data instance is associated with one single class label. Multilabel learning differs in the way that data instances may belong to multiple concepts simultaneously, which naturally appear in a variety of high impact domains, ranging from bioinformatics and information retrieval to multimedia analysis. It targets leveraging the multiple label information of data instances to build a predictive learning model that can classify unlabeled instances into one or multiple predefined target classes. In multilabel learning, even though each instance is associated with a rich set of class labels, the label information could be noisy and incomplete as the labeling process is both time consuming and labor expensive, leading to potential missing annotations or even erroneous annotations. The existence of noisy and missing labels could negatively affect the performance of underlying learning algorithms. More often than not, multilabeled data often has noisy, irrelevant, and redundant features of high dimensionality. The existence of these uninformative features may also deteriorate the predictive power of the learning model due to the curse of dimensionality. Feature selection, as an effective dimensionality reduction technique, has shown to be powerful in preparing high-dimensional data for numerous data mining and machine-learning tasks. However, a vast majority of existing multilabel feature selection algorithms either boil down to solving multiple single-labeled feature selection problems or directly make use of the imperfect labels to guide the selection of representative features. As a result, they may not be able to obtain discriminative features shared across multiple labels. In this article, to bridge the gap between a rich source of multilabel information and its blemish in practical usage, we propose a novel noise-resilient multilabel informed feature selection framework (MIFS) by exploiting the correlations among different labels. In particular, to reduce the negative effects of imperfect label information in obtaining label correlations, we decompose the multilabel information of data instances into a low-dimensional space and then employ the reduced label representation to guide the feature selection phase via a joint sparse regression framework. Empirical studies on both synthetic and real-world datasets demonstrate the effectiveness and efficiency of the proposed MIFS framework.

On social media, the user generated contents, e.g., Articles and images, can be assigned with multiple labels. In this paper, we focus on the problem of performing multi-label classification on social media data, where the user generated contents are associated with multiple labels. Multi-label learning studies the problem where each object is represented by a single instance and associated with a set of labels. Current multi-label learning algorithms mainly exploit label correlations globally, by assuming that the label correlations are shared by all the examples. In real applications, however, different examples may share different label correlations. In this paper, we propose a Local Pair wise Label Correlation (LPLC) method for social media content categorization. We try to exploit the strongest local pair wise label correlations between the ground truth labels for each training example by computing the maximum conditional probabilities. If two labels have strong correlation, there will be a larger conditional probability of one label given by another. In the training stage, we find the most correlated labels for ground truth labels of each training example. In the test stage, we make prediction through maximizing the posterior probability, which is estimated with the distribution of each label in the k nearest neighbors and their most correlated local pair wise label correlations. We compare our method with six well-established multi-label learning algorithms over nine data sets from different social media data domains and scales. Comparison results with the state-of-the-arts approaches manifest competitive performances of our method.

In multi-label learning, research on label correlation provides an effective solution to compress the hypothesis space of classifiers. However, the current works focus on the label correlation adapted to overall data, while ignoring the locally targeted information presented by some instances. The lack exploration on the distribution of local label correlation in multi-label instance space undoubtedly limits the in-depth application of label correlation in multi-label learning. Based on fuzzy rough set theory, the instance-level label correlation distribution is first proposed in this paper and applied to design a novel multi-label learner. For each multi-label instance, the local importance of features to label is quantitatively analyzed, by considering the decisive influence of input information on decision-making. According to coincidence degree between local feature weight distribution for different labels, the instance-level label correlation is constructed. In order to reflect the internal relationship between label variables objectively, the instance-level label correlation distribution is integrated into the empirical label relevance. On the basis, the label relevance matrix is used to define the constraints of optimization function in a new form. The relative position of sub-separating hyperplanes in input space is quantitatively characterized to reduce the complexity of multi-label classifier and improve learning performance. The experiment results on eighteen multi-label datasets illustrate the effectiveness of our algorithm. The impact of core parameters on performance is also dissected.

Deep learning has been a successful model which can effectively represent\nseveral features of input space and remarkably improve image recognition\nperformance on the deep architectures. In our research, an adaptive structural\nlearning method of Restricted Boltzmann Machine (Adaptive RBM) and Deep Belief\nNetwork (Adaptive DBN) have been developed as a deep learning model. The models\nhave a self-organize function which can discover an optimal number of hidden\nneurons for given input data in a RBM by neuron generation-annihilation\nalgorithm, and can obtain an appropriate number of RBM as hidden layers in the\ntrained DBN. The proposed method was applied to a concrete image benchmark data\nset SDNET 2018 for crack detection. The dataset contains about 56,000 crack\nimages for three types of concrete structures: bridge decks, walls, and paved\nroads. The fine-tuning method of the Adaptive DBN can show 99.7%, 99.7%, and\n99.4% classification accuracy for test dataset of three types of structures. In\nthis paper, our developed Adaptive DBN was embedded to a tiny PC with GPU for\nreal-time inference on a drone. For fast inference, the fine tuning algorithm\nalso removed some inactivated hidden neurons to make a small model and then the\nmodel was able to improve not only classification accuracy but also inference\nspeed simultaneously. The inference speed and running time of portable battery\ncharger were evaluated on three kinds of Nvidia embedded systems; Jetson Nano,\nAGX Xavier, and Xavier NX.\n

Existing multi-label learning algorithms mostly assume that all class labels in an obtained data set are completely accurate. This assumption is rarely valid. As the scale of the label space increases, owing to limited domain knowledge and differences among methods of data acquisition, some examples will be unlabeled or mislabeled. The introduction of such noise results in the poor performance of trained multi-label classifiers. This paper proposes a new method for multi-label learning with label noise, called low-rank-constraint-based multi-label learning with label noise (ML-LRC), by incorporating low-rank matrix recovery and features selection into the learning process. First, we assume that two strongly correlated labels share more samples than two weakly correlated labels. Based on this assumption, the ML-LRC applies low-rank constraints to the label matrix to mine the local correlations of class labels and remove noise from the observed label matrix. Second, we believe that each class label only correlates to some features. So we control the sparsity of the coefficient matrix to filter out label-specific features, which can reduce the complexity of the model and greatly improve the performance of the classifier. The accelerated proximal gradient method and alternating direction method of multipliers are adopted to solve the problem. Experiments conducted on four benchmark multi-label datasets demonstrate the competitive performance of our proposed method compared with several state-of-the-art methods.

To deal with the problem where each instance is associated with multiple labels, a lot of multi-label learning algorithms have been developed in recent years. Some approaches have been proposed to select label-specific features to utilize discriminate features for multi-label classification. Although label correlation has been considered in learning label-specific features, the critical correlation among instances was less taken into account. In this paper, we proposed a new approach called multi-label learning with label-specific features using correlation information (LSF-CI) to learn label-specific features for each label with the consideration of both correlation information in label space and correlation information in feature space. In the LSF-CI, the instance correlation in feature space is computed by a probabilistic neighborhood graph model, and label correlation in label space is computed by cosine similarity. For multi-label data, the LSF-CI has the capability to select Label-specific features for each label as well as classify an unseen instance into a set of relevant labels. To validate the effectiveness of LSF-CI, we conducted comprehensive experiments on eight multi-label datasets. The experimental results demonstrate that the LSF-CI is capable of selecting compact label-specific features, and achieving a competitive performance in comparison with the performances of the existing multi-label learning approaches.

Information theoretical based methods have attracted a great attention in recent years, and gained promising results to deal with multi-label data with high dimensionality. However, most of the existing methods are either directly transformed from heuristic single-label feature selection methods or inefficient in exploiting labeling information. Thus, they may not be able to get an optimal feature selection result shared by multiple labels. In this paper, we propose a general global optimization framework, in which feature relevance, label relevance (i.e., label correlation), and feature redundancy are taken into account, thus facilitating multi-label feature selection. Moreover, the proposed method has an excellent mechanism for utilizing inherent properties of multi-label learning. Specially, we provide a formulation to extend the proposed method with label-specific features. Empirical studies on twenty multi-label data sets reveal the effectiveness and efficiency of the proposed method. Our implementation of the proposed method is available online at: https://jiazhang-ml.pub/GRRO-master.zip.

Recent status-of-the-art methods for multivariate time series forecasting can be categorized into graph-based approach and global-local approach. The former approach uses graphs to represent the dependencies among variables and apply graph neural networks to the forecasting problem. The latter approach decomposes the matrix of multivariate time series into global components and local components to capture the shared information across variables. However, both approaches cannot capture the propagation delay of the dependencies among individual variables of a multivariate time series, for example, the congestion at intersection A has delayed effects on the neighboring intersection B. In addition, graph-based forecasting methods cannot capture the shared global tendency across the variables of a multivariate time series; and global-local forecasting methods cannot reflect the nonlinear inter-dependencies among variables of a multivariate time series. In this paper, we propose to combine the advantages of both approaches by integrating Adaptive Global-Local Graph Structure Learning with Gated Recurrent Units (AGLG-GRU). We learn a global graph to represent the shared information across variables. And we learn dynamic local graphs to capture the local randomness and nonlinear dependencies among variables. We apply diffusion convolution and graph convolution operations to global and dynamic local graphs to integrate the information of graphs and update gated recurrent unit for multivariate time series forecasting. The experimental results on seven representative real-world datasets demonstrate that our approach outperforms various existing methods.

Label correlation guided borderline oversampling for imbalanced multi-label data learning

Feature selection based on label distribution and fuzzy mutual information