Multi-label learning with fuzzy hypergraph regularization for protein subcellular location prediction.

Prediction of protein subcellular localization is an important but challenging problem, particularly when proteins may simultaneously exist at, or move between, two or more different subcellular location sites. Most of the existing protein subcellular localization methods are only used to deal with the single-location proteins. In the past few years, only a few methods have been proposed to tackle proteins with multiple locations. However, they only adopt a simple strategy, that is, transforming the multilocation proteins to multiple proteins with single location, which does not take correlations among different subcellular locations into account. In this paper, a novel method named random label selection (RALS) (multilabel learning via RALS), which extends the simple binary relevance (BR) method, is proposed to learn from multilocation proteins in an effective and efficient way. RALS does not explicitly find the correlations among labels, but rather implicitly attempts to learn the label correlations from data by augmenting original feature space with randomly selected labels as its additional input features. Through the fivefold cross-validation test on a benchmark data set, we demonstrate our proposed method with consideration of label correlations obviously outperforms the baseline BR method without consideration of label correlations, indicating correlations among different subcellular locations really exist and contribute to improvement of prediction performance. Experimental results on two benchmark data sets also show that our proposed methods achieve significantly higher performance than some other state-of-the-art methods in predicting subcellular multilocations of proteins. The prediction web server is available at http://levis.tongji.edu.cn:8080/bioinfo/MLPred-Euk/ for the public usage.

Protein subcellular localization aims at predicting the location of a protein within a cell using computational methods. Knowledge of subcellular localization of proteins indicates protein functions and helps in identifying drug targets. Prediction of protein subcellular localization is an important but challenging problem, particularly when proteins may simultaneously exist at, or move between, two or more different subcellular location sites. Most of the existing protein subcellular localization methods are only used to deal with the single-location proteins. To better reflect the characteristics of multiplex proteins, we formulate prediction of subcellular localization of multiplex proteins as a multi-label learning problem. We present and compare two multi-label learning approaches, which exploit correlations between labels and leverage label-specific features, respectively, to induce a high quality prediction model. Experimental results on six protein data sets under various organisms show that our described methods achieve significantly higher performance than any of the existing methods. Among the different multi-label learning methods, we find that methods exploiting label correlations performs better than those leveraging label-specific features.

To deal with the problem where each instance is associated with multiple labels, a lot of multi-label learning algorithms have been developed in recent years. Some approaches have been proposed to select label-specific features to utilize discriminate features for multi-label classification. Although label correlation has been considered in learning label-specific features, the critical correlation among instances was less taken into account. In this paper, we proposed a new approach called multi-label learning with label-specific features using correlation information (LSF-CI) to learn label-specific features for each label with the consideration of both correlation information in label space and correlation information in feature space. In the LSF-CI, the instance correlation in feature space is computed by a probabilistic neighborhood graph model, and label correlation in label space is computed by cosine similarity. For multi-label data, the LSF-CI has the capability to select Label-specific features for each label as well as classify an unseen instance into a set of relevant labels. To validate the effectiveness of LSF-CI, we conducted comprehensive experiments on eight multi-label datasets. The experimental results demonstrate that the LSF-CI is capable of selecting compact label-specific features, and achieving a competitive performance in comparison with the performances of the existing multi-label learning approaches.

Partial multi-label learning via specific label disambiguation

In multi-label learning, each object belongs to multiple class labels simultaneously. In the data explosion age, the size of data is often huge, i.e., large number of instances, features and class labels. The high dimension of both the feature and label spaces has posed great challenges to multi-label learning problems, e.g., high time and memory costs. In this paper, we propose a new framework for multi-label learning with a large number of class labels and features, i.e., M ulti- L abel L earning via F eature and L abel S pace D imension R eduction, namely MLL-FLSDR. Specifically, both the feature space and label space are reduced to low dimensional spaces respectively, in which the local structure of data points is utilized to constrain the geometrical structure on both the learned low dimensional spaces and guarantee the qualities of them. Then, an effective multi-label classifier is constructed from the low dimensional feature space to the latent label space. Last, the final prediction for new test data examples can be obtained by recovering from their prediction results in the latent label space with an encoding matrix learned in the previous stage. Extensive comparison experiments with the state-of-the-art approaches manifest the effectiveness of the proposed method MLL-FLSDR.

In multi-label image classification, each image is always associated with multiple labels and labels are usually correlated with each other. The intrinsic relation among labels can definitely contribute to classifier training. However, most previous studies on active learning for multi-label image classification purely mine label correlation based on observed label distribution. They ignore the mapping relation between examples and their labels. This mapping relation also implicates label relationship. Ignoring the mapping relation leads to an uncomprehensive label correlation estimation and results in a bad performance for classification. In this paper, we propose a novel multi-label active learning with low-rank mapping for image classification, called LMMAL, to solve this issue. More precisely, we train a low-rank mapping matrix to signify the mapping relation between the feature space and the label space of a certain multi-label dataset. Using this low-rank mapping relation, we exploit a full label correlation. Subsequently, an effective sampling strategy is designed by integrating this potential information with uncertainty to select the most informative example-label pairs. In addition, we extend LMMAL with automatic labeling (denoted as AL-LMMAL) to further reduce the annotation workload of active learning. Empirical results demonstrate the effectiveness of our approaches.

For automatic object detection tasks, large amounts of training images are usually labeled to achieve more reliable training of the object classifiers; this is cost-expensive since it requires hiring professionals to label large-scale training images. When a large number of object classes come into view, the issue of obtaining a large enough amount of the labeled training images becomes more critical. There are three potential solutions to reduce the burden for image labeling: (1) allowing people to provide the object labels loosely at the image level rather than at the object level (e.g., loosely-tagged images without identifying the exact object locations in the images); (2) harnessing large-scale collaboratively-tagged images that are available on the Internet; and, (3) developing new machine learning algorithms that can directly leverage large-scale collaboratively- or loosely-tagged images for achieving more effective training of a large number of object classifiers. Based on these observations, a multi-task multi-label multiple instance learning (MTML-MIL) algorithm is developed in this paper by leveraging both interobject correlations and large-scale loosely-labeled images for object classifier training. By seamlessly integrating multi-task learning, multi-label learning, and multiple instance learning, our MTML-MIL algorithm can achieve more accurate training of a large number of inter-related object classifiers (where an object network is constructed for determining the inter-related learning tasks directly in the feature space rather than in the label space). Our experimental results have shown that our MTML-MIL algorithm can achieve higher detection accuracy rates for automatic object detection.

Multi-label learning is an effective framework for learning with objects that have multiple semantic labels, and has been successfully applied into many real-world tasks. In contrast with traditional single-label learning, the cost of labeling a multi-label example is rather high, thus it becomes an important task to train an effectivemulti-label learning model with as few labeled examples as possible. Active learning, which actively selects the most valuable data to query their labels, is the most important approach to reduce labeling cost. In this paper, we propose a novel approach MADM for batch mode multi-label active learning. On one hand, MADM exploits representativeness and diversity in both the feature and label space by matching the distribution between labeled and unlabeled data. On the other hand, it tends to query predicted positive instances, which are expected to be more informative than negative ones. Experiments on benchmark datasets demonstrate that the proposed approach can reduce the labeling cost significantly.

Multi-label classification with local pairwise and high-order label correlations using graph partitioning

Partial multi-label learning (PML) models the scenario where each training instance is annotated with a set of candidate labels, and only some of the labels are relevant. The PML problem is practical in real-world scenarios, as it is difficult and even impossible to obtain precisely labeled samples. Several PML solutions have been proposed to combat with the prone misled by the irrelevant labels concealed in the candidate labels, but they generally focus on the smoothness assumption in feature space or low-rank assumption in label space, while ignore the negative information between features and labels. Specifically, if two instances have largely overlapped candidate labels, irrespective of their feature similarity, their ground-truth labels should be similar; while if they are dissimilar in the feature and candidate label space, their ground-truth labels should be dissimilar with each other. To achieve a credible predictor on PML data, we propose a novel approach called PML-LFC (Partial Multi-label Learning with Label and Feature Collaboration). PML-LFC estimates the confidence values of relevant labels for each instance using the similarity from both the label and feature spaces, and trains the desired predictor with the estimated confidence values. PML-LFC achieves the predictor and the latent label matrix in a reciprocal reinforce manner by a unified model, and develops an alternative optimization procedure to optimize them. Extensive empirical study on both synthetic and real-world datasets demonstrates the superiority of PML-LFC.

Using Nearest Feature Line and Tunable Nearest Neighbor methods for prediction of protein subcellular locations

Multi-label weak-label learning via semantic reconstruction and label correlations

In the fields of machine learning and data mining, label learning is a nascent area of research, and within this paradigm, there is much room for improving multi-label manifold learning algorithms for high-dimensional data. Thus far, researchers have experimented with mapping relationships from the feature space to the traditional logical label space (using neighbors in the label space, for example, to predict logical label vectors from the feature space's manifold structure). Here we combine the feature manifold's and label space's local topological structures to reconstruct the label manifold. To achieve this, we use a nonlinear manifold learning algorithm to transform the local topological structure from the feature space to the label space. Our algorithm adopts a regularized least-squares kernel method to realize the reconstruction process, employing an optimization function to find the best solution. Extensive experiments show that our algorithm significantly improves multi-label manifold learning in terms of learning accuracy and time complexity.

In multi-label learning, research on label correlation provides an effective solution to compress the hypothesis space of classifiers. However, the current works focus on the label correlation adapted to overall data, while ignoring the locally targeted information presented by some instances. The lack exploration on the distribution of local label correlation in multi-label instance space undoubtedly limits the in-depth application of label correlation in multi-label learning. Based on fuzzy rough set theory, the instance-level label correlation distribution is first proposed in this paper and applied to design a novel multi-label learner. For each multi-label instance, the local importance of features to label is quantitatively analyzed, by considering the decisive influence of input information on decision-making. According to coincidence degree between local feature weight distribution for different labels, the instance-level label correlation is constructed. In order to reflect the internal relationship between label variables objectively, the instance-level label correlation distribution is integrated into the empirical label relevance. On the basis, the label relevance matrix is used to define the constraints of optimization function in a new form. The relative position of sub-separating hyperplanes in input space is quantitatively characterized to reduce the complexity of multi-label classifier and improve learning performance. The experiment results on eighteen multi-label datasets illustrate the effectiveness of our algorithm. The impact of core parameters on performance is also dissected.

In multi-label learning with missing labels, previous works usually only consider all features, or only consider label-specific features. It is obviously inappropriate to select a subset of features only considering which have a great discriminability for all labels or for a label. Besides, they are built on an assumption that if two labels are correlated, their regression coefficient vectors should be similar. However, it is hard to hold this assumption in real-world applications. Therefore, we propose a novel multi-label classification approach for missing labels via considering common and label-specific features with correlation information. More specifically, we firstly utilize <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$l_{1}$</tex> - norm regularizer over label correlation matrix, and a new sup-plementary label matrix is augmented from the incomplete label matrix by learning label correlations. Then, for a better label recovery, we introduce <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$l_{2,1}$</tex> -norm and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$l_{1}$</tex> -norm regularizers to learn common and label-specific features simultaneously. Thirdly, to make more rational use of label correlation to solve missing labels, we use a regularizer to constrain label correlations on outputs of labels instead of regression coefficient matrix. Finally, an objective function is designed in terms of the above processing. Extensive experiments conducted on eight data sets demonstrate the effectiveness of the proposed approach.