Graph-based semi-supervised learning with manifold preprocessing for image classification

Graph-based semi-supervised learning (SSL) algorithms have been widely studied in the last few years. Most of these algorithms were designed from unconstrained optimization problems using a Laplacian regularizer term as smoothness functional in an attempt to reflect the intrinsic geometric structure of the data's marginal distribution. Although a number of recent research papers are still focusing on unconstrained methods for graph-based SSL, a recent statistical analysis showed that many of these algorithms may be unstable on transductive regression. Therefore, we focus on providing new constrained methods for graph-based SSL. We begin by analyzing the regularization framework of existing unconstrained methods. Then, we incorporate two normalization constraints into the optimization problem of three of these methods. We show that the proposed optimization problems have closed-form solution. By generalizing one of these constraints to any distribution, we provide generalized methods for constrained graph-based SSL. The proposed methods have a more flexible regularization framework than the corresponding unconstrained methods. More precisely, our methods can deal with any graph Laplacian and use higher order regularization, which is effective on general SSL taks. In order to show the effectiveness of the proposed methods, we provide comprehensive experimental analyses. Specifically, our experiments are subdivided into two parts. In the first part, we evaluate existing graph-based SSL algorithms on time series data to find their weaknesses. In the second part, we evaluate the proposed constrained methods against six state-of-the-art graph-based SSL algorithms on benchmark data sets. Since the widely used best case analysis may hide useful information concerning the SSL algorithms' performance with respect to parameter selection, we used recently proposed empirical evaluation models to evaluate our results. Our results show that our methods outperforms the competing methods on most parameter settings and graph construction methods. However, we found a few experimental settings in which our methods showed poor performance. In order to facilitate the reproduction of our results, the source codes, data sets, and experimental results are freely available.

In this article we present a semi‐supervised active learning algorithm for pattern discovery in information extraction from textual data. The patterns are reduced regular expressions composed of various characteristics of features useful in information extraction. Our major contribution is a semi‐supervised learning algorithm that extracts information from a set of examples labeled as relevant or irrelevant to a given attribute. The approach is semi‐supervised because it does not require precise labeling of the exact location of features in the training data. This significantly reduces the effort needed to develop a training set. An active learning algorithm is used to assist the semi‐supervised learning algorithm to further reduce the training set development effort. The active learning algorithm is seeded with a single positive example of a given attribute. The context of the seed is used to automatically identify candidates for additional positive examples of the given attribute. Candidate examples are manually pruned during the active learning phase, and our semi‐supervised learning algorithm automatically discovers reduced regular expressions for each attribute. We have successfully applied this learning technique in the extraction of textual features from police incident reports, university crime reports, and patents. The performance of our algorithm compares favorably with competitive extraction systems being used in criminal justice information systems.

Towards Dynamic Self-Training for Scalable Semi-Supervised Learning on Graphs

Hardware of Distributed WSNs is limited, which turns into a big challenge. In recent years, there are some pattern-based query strategies have been proposed. But compared to the methods based on machine learning, these methods need high energy consumption in the calculation of distance. In this paper, we propose a pattern-based query strategy based on semi-supervised machine learning in wireless sensor network. The strategy uses historical time series data streams for o†ine machine learning. First of all, we flnd out the flxed length subsequences and screen out the suitable data sequences as training dataset and sample dataset. And then, because labeled data is less in practical application, we do label propagation with graph-based semi-supervised machine learning algorithm, Local and Global Consistency (LGC), to increase the number of labeled data sequences. Next, we use the supervised learning algorithm Support Vector Machine (SVM) to train a classifler. Finally, for a new data sequence, we can directly get its category through the classifler. Experimental results show that the performance of our strategy is almost better than other pattern-based query strategies on classiflcation accuracy and energy consumption.

Local learning integrating global structure for large scale semi-supervised classification

While recent studies on semi-supervised learning have shown remarkable progress in leveraging both labeled and unlabeled data, most of them presume a basic setting of the model is randomly initialized. In this work, we consider semi-supervised learning and transfer learning jointly, leading to a more practical and competitive paradigm that can utilize both powerful pre-trained models from source domain as well as labeled/unlabeled data in the target domain. To better exploit the value of both pre-trained weights and unlabeled target examples, we introduce adaptive consistency regularization that consists of two complementary components: Adaptive Knowledge Consistency (AKC) on the examples between the source and target model, and Adaptive Representation Consistency (ARC) on the target model between labeled and unlabeled examples. Examples involved in the consistency regularization are adaptively selected according to their potential contributions to the target task. We conduct extensive experiments on popular benchmarks including CIFAR-10, CUB-200, and MURA, by fine-tuning the ImageNet pre-trained ResNet-50 model. Results show that our proposed adaptive consistency regularization outperforms state-of-the-art semi-supervised learning techniques such as Pseudo Label, Mean Teacher, and FixMatch. Moreover, our algorithm is orthogonal to existing methods and thus able to gain additional improvements on top of MixMatch and FixMatch. Our code is available at https://github.com/Walleclipse/Semi-Supervised-Transfer-Learning-Paddle.

Symmetric low-rank representation with adaptive distance penalty for semi-supervised learning

Noise-robust semi-supervised learning via fast sparse coding

In many practical data mining applications, such as web categorization, key gene selection, etc., generally, unlabeled training examples are readily available, but labeled ones are fairly expensive to obtain. Therefore, in recent years, semi-supervised learning algorithms such as graph-based methods have attracted much attention. However, most of these traditional methods adopted a Gaussian function to calculate the edge weights of the graph. In this paper, a novel weight for semi-supervised graph-based methods is proposed. The new method adds the label information from problem into the target function, and adopts the geodesic distance rather than Euclidean distance as the measure of the difference between two data points when conducting the calculation. In addition, we also add class prior knowledge from problem into semi-supervised learning algorithm. Here we address the problem of learning with local and global consistency (LGC). It was found that the effect of class prior knowledge was probably different between under high label rate and low label rate. Furthermore, we integrate sparse representation (SR) in LGC algorithm. Experiments on a UCI data set show that our proposed method outperforms the original algorithms.KeywordsSemi-supervised learningClassification errorLabel informationGeodesic distanceClass prior knowledgesparse representation

Semi-Supervised Learning

In this paper, we propose a novel graph-based semisupervised learning framework, called joint sparse representation and embedding propagation learning (JSREPL). The idea of JSREPL is to join EPL with sparse representation to perform label propagation. Like most of graph-based semisupervised propagation learning algorithms, JSREPL also constructs weights graph matrix from given data. Different from classical approaches which build weights graph matrix and estimate the labels of unlabeled data in sequence, JSREPL simultaneously builds weights graph matrix and estimates the labels of unlabeled data. We also propose an efficient algorithm to solve the proposed problem. The proposed method is applied to the problem of semisupervised image clustering using the ORL, Yale, PIE, and YaleB data sets. Our experiments demonstrate the effectiveness of our proposed algorithm.

The key problem of Graph-Based Semi-Supervised Learning (GBSSL) methods is how to construct the graph structure under some assumptions. While distance information among graph nodes is investigated well for graph construction, the density information is not given enough attention. In this paper, we propose a novel GBSSL method, named Density-Sensitive Manifold Learning (DSML), which introduces density distribution into graph construction by calculating a new propagation coefficient matrix. The experimental results show that DSML scheme performs better than traditional GBSSL methods. More importantly, the new propagation coefficient matrix can be easily introduced into traditional GBSSL methods to improve their performance, which is also validated in the experiments.

Aiming at the problem of object-based image retrieval, a novel semi-supervised multi-instance learning (MIL) algorithm based on RS (rough set) attribute reduction and Transductive support vector machine (TSVM) has been presented---RSTSVM- MIL algorithm. This algorithm regards the whole image as a bag, and the low-level visual feature of the segmented regions as instances, in order to transform every bag to a single sample, RSTSVM-MIL first generates a collection of visual-word to construct projection space by K-means clustering method in the instance feature space, then a nonlinear mapping is defined using these visual-word and maps every bag to a point in the projection space, extracting all bags' projection feature, which converts MIL problem to a standard supervised learning problem. Finally, in order to improve the efficiency of classifier training and to use a large amount of unlabeled image together with the labeled image to build better classifiers, we use RS (rough set) method to reduce the redundant information in projection feature firstly, then semi-supervised learning algorithm TSVM be used to train classifier. Experimental results on the SIVAL image set show that solving MIL problems using semi-supervised learning is feasible, and the RSTSVM-MIL is competitive with other state-of-the-art MIL algorithms in object- based image retrieval problem.

Semi-supervised matrixized least squares support vector machine

Employing unlabeled data to improve the classification performance of SVM, and its application in audio event classification