- Research Article
5
- 10.1016/j.neucom.2013.10.033
Semi-supervised learning via sparse model
- Dec 01, 2013
- Neurocomputing
- Yu Wang + 4 more +4
Semi-supervised learning via sparse model
Automatic image annotation by semi-supervised manifold kernel density estimation
Semi-supervised learning via sparse model
Semi-supervised learning via sparse model
Graph-Based Semi-Supervised Learning: A Comprehensive Review.
Semi-supervised learning (SSL) has tremendous value in practice due to the utilization of both labeled and unlabelled data. An essential class of SSL methods, referred to as graph-based semi-supervised learning (GSSL) methods in the literature, is to first represent each sample as a node in an affinity graph, and then, the label information of unlabeled samples can be inferred based on the structure of the constructed graph. GSSL methods have demonstrated their advantages in various domains due to their uniqueness of structure, the universality of applications, and their scalability to large-scale data. Focusing on GSSL methods only, this work aims to provide both researchers and practitioners with a solid and systematic understanding of relevant advances as well as the underlying connections among them. The concentration on one class of SSL makes this article distinct from recent surveys that cover a more general and broader picture of SSL methods yet often neglect the fundamental understanding of GSSL methods. In particular, a significant contribution of this article lies in a newly generalized taxonomy for GSSL under the unified framework, with the most up-to-date references and valuable resources such as codes, datasets, and applications. Furthermore, we present several potential research directions as future work with our insights into this rapidly growing field.
Read moreGraph-based semi-supervised learning: A review
Graph-based semi-supervised learning: A review
Semi-Supervised Learning with Density-Sensitive Manifold graph
The key problem of Graph-Based Semi-Supervised Learning (GBSSL) methods is how to construct the graph structure under some assumptions. While distance information among graph nodes is investigated well for graph construction, the density information is not given enough attention. In this paper, we propose a novel GBSSL method, named Density-Sensitive Manifold Learning (DSML), which introduces density distribution into graph construction by calculating a new propagation coefficient matrix. The experimental results show that DSML scheme performs better than traditional GBSSL methods. More importantly, the new propagation coefficient matrix can be easily introduced into traditional GBSSL methods to improve their performance, which is also validated in the experiments.
Read moreRobust deep semi-supervised learning with label propagation and differential privacy
Semi-supervised learning (SSL) methods provide a powerful tool for utilizing abundant unlabeled data to strengthen standard supervised learning. Traditional graph-based SSL methods prevail in classical SSL problems for their intuitional implementation and effective performance. However, they encounter troubles when applying to image classification followed by modern deep learning, since the diffusion algorithms face the curse of dimensionality. In this study, we propose a simple and efficient SSL method, combining a graph-based SSL paradigm with differential privacy. We aim at developing coherent latent feature space of deep neural networks so that the diffusion algorithm in the latent space can give more precise predictions for unlabeled data. Our approach achieves state-of-the-art performance on the Cifar10, Cifar100, and Mini-imagenet benchmark datasets and obtains an error rate of 18.56% on Cifar10 using only 1% of all labels. Furthermore, our approach inherits the benefits of graph-based SSL methods with a simple training process and can be easily combined with any network architecture.
Read moreAutomatic image annotation via compact graph based semi-supervised learning
Automatic image annotation via compact graph based semi-supervised learning
Instance selection method for improving graph-based semi-supervised learning
Graph-based semi-supervised learning is an important semi-supervised learning paradigm. Although graph-based semi-supervised learning methods have been shown to be helpful in various situations, they may adversely affect performance when using unlabeled data. In this paper, we propose a new graph-based semi-supervised learning method based on instance selection in order to reduce the chances of performance degeneration. Our basic idea is that given a set of unlabeled instances, it is not the best approach to exploit all the unlabeled instances; instead, we should exploit the unlabeled instances that are highly likely to help improve the performance, while not taking into account the ones with high risk. We develop both transductive and inductive variants of our method. Experiments on a broad range of data sets show that the chances of performance degeneration of our proposed method are much smaller than those of many state-of-the-art graph-based semi-supervised learning methods.
Read moreGraph-based semi-supervised multi-label learning method
The problem of multi-label classification has attracted great interest in the last decade. However, most multi-label learning methods only focus on supervised settings, and can not effectively make use of relatively inexpensive and easily obtained large number of unlabeled samples. To solve this problem, we put forward a novel graph-based semi-supervised multi-label learning method, called GSMM. GSMM characterize the inherent correlations among multiple labels by Hilbert-Schmidt independence criterion. It's expected to derive the optimal assignment of class membership to unlabeled samples by maximizing the consistency of class label correlations and simultaneously as smooth as possible on sample feature graph. The experiments comparing GSMM to the state-of-the-art multi-label learning approaches on several real-world datasets show GSMM can effectively learn from the labeled and unlabeled samples. Especially when the labeled is relatively rare, it can improve the performance greatly.
Read moreImage/Video Semantic Analysis by Semi-Supervised Learning
The insufficiency of labeled training samples is a major obstacle in automatic semantic analysis of large scale image/video database. Semi-supervised learning, which attempts to learn from both labeled and unlabeled data, is a promising approach to tackle this problem. As a major family of semi-supervised learning, graph-based methods have attracted more and more recent research. In this chapter, a brief introduction is given on popular semi-supervised learning methods, especially the graph-based methods, as well as their applications in the area of image annotation, video annotation, and image retrieval. It is well known that the pair-wise similarity is an essential factor in graph propagation based semisupervised learning methods. A novel graph-based semi-supervised learning method, named Structure- Sensitive Anisotropic Manifold Ranking (SSAniMR), is derived from a PDE based anisotropic diffusion framework. Instead of using Euclidean distance only, SSAniMR further takes local structural difference into account to more accurately measure pair-wise similarity. Finally some future directions of using semi-supervised learning to analyze the multimedia content are discussed.
Read moreALG: Adaptive low-rank graph regularization for scalable semi-supervised and unsupervised learning
ALG: Adaptive low-rank graph regularization for scalable semi-supervised and unsupervised learning
Video semantic analysis based on structure-sensitive anisotropic manifold ranking
Video semantic analysis based on structure-sensitive anisotropic manifold ranking
Semi-supervised Learning via Adaptive Low-Rank Graph
Graph-based semi-supervised learning (SSL) is one of the most popular topics in the past decades. Most conventional graph-based SSL methods utilize two stage-approach to infer the class labels of the unlabeled data, i.e. it firstly constructs a graph for capturing the geometry of data manifold and then perform SSL for prediction. However, it suffers from three drawbacks: (1) the graph construction and SSL stages are separate. They do not share common information to enhance the performance of classification; (2) the graph construction and SSL should be scalable. However, most methods mainly focus on the improvement of classification accuracy but neglect the computational cost; (3) the graph should also be adaptive and robust to the parameters and datasets. However, this will usually increase computational cost making the efficiency cannot be guaranteed simultaneously. In this paper, we aim to handle the above issues. To achieve adaptiveness of SSL, we adopt a bilinear low-rank model for graph construction, where the coefficient matrix of the low-rank model is calculated through an adaptive and efficient procedure the corresponding constructed graph can capture the global structure of data manifold. Meriting from such a graph, we then propose a unified framework for scalable SSL, where we have involved the graph construction and SSL into a unified optimization problem. As a result, the discriminative information learned by SSL can be provided to improve the discriminative ability of graph construction, while the updated graph can further enhance the classification results of SSL. Simulation indicates that the proposed method can achieve better classification and clustering performance compared with other state-of-the-art graph-based SSL methods.
Read moreCapped l_2,1-Norm Regularized Dictionary Coding for Scalable Semi-Supervised Learning
During the past decade, graph-based semi-supervised learning has become one of the most important research areas in machine learning and artificial intelligence community. In this paper, we propose a Capped l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -Norm Regularized Dictionary Learning to construct the graph for semi-supervised learning (SSL). The new sparse coding is robust to the outliers and insensitive to noise. Then, the developed graph construction can better characterize the geometrical structure of data mainfold and be utilized for efficient graph based SSL. the proposed SSL methods can also be easily extended to deal with out-of-sample data. Simulation results show that the proposed method can achieve better performance compared with other state-of-the-art graph based SSL methods.
Read moreA splitting method for the locality regularized semi-supervised subspace clustering
Graph-based semi-supervised learning (G-SSL) methods play an increasingly important role in machine learning systems. Recently, latent low-rank representation (LatLRR) graph has gained great success in subspace clustering. However, LatLRR only considers the global structure, while the local geometric information, which is often important to many real applications, is ignored. In this paper, we propose a locality regularized LatLRR model (LR-LatLRR) for semi-supervised subspace clustering problems. This model incorporates two regularization terms into LatLRR by taking the local structure of data into account. Then, we develop an efficient splitting algorithm for solving LR-LatLRR. In addition, we also prove the global convergence of the proposed algorithm. Furthermore, we extend the LR-LatLRR model to a case of including the non-negative constraint. Finally, we conduct experiments on a synthetic data and several real data sets for the semi-supervised clustering problems. Experimental results show that our method can obtain high classification accuracy and outperforms several state-of-the-art G-SSL methods.
Read moreParticle Competition and Cooperation to Prevent Error Propagation from Mislabeled Data in Semi-supervised Learning
Semi-supervised learning is applied to classification problems where only a small portion of the data items is labeled. In these cases, the reliability of the labels is a crucial factor, because mislabeled items may propagate wrong labels to a large portion or even the entire data set. This paper aims to address this problem by presenting a graph-based (network-based) semi-supervised learning method, specifically designed to handle data sets with mislabeled samples. The method uses teams of walking particles, with competitive and cooperative behavior, for label propagation in the network constructed from the input data set. The proposed model is nature-inspired and it incorporates some features to make it robust to a considerable amount of mislabeled data items. Computer simulations show the performance of the method in the presence of different percentage of mislabeled data, in networks of different sizes and average node degree. Importantly, these simulations reveals the existence of the critical points of the mislabeled subset size, below which the network is free of wrong label contamination, but above which the mislabeled samples start to propagate their labels to the rest of the network. Moreover, numerical comparisons have been made among the proposed method and other representative graph-based semi-supervised learning methods using both artificial and real-world data sets. Interestingly, the proposed method has increasing better performance than the others as the percentage of mislabeled samples is getting larger.
Read more