Local learning integrating global structure for large scale semi-supervised classification

Graph-based semi-supervised learning (SSL) algorithms have been widely studied in the last few years. Most of these algorithms were designed from unconstrained optimization problems using a Laplacian regularizer term as smoothness functional in an attempt to reflect the intrinsic geometric structure of the data's marginal distribution. Although a number of recent research papers are still focusing on unconstrained methods for graph-based SSL, a recent statistical analysis showed that many of these algorithms may be unstable on transductive regression. Therefore, we focus on providing new constrained methods for graph-based SSL. We begin by analyzing the regularization framework of existing unconstrained methods. Then, we incorporate two normalization constraints into the optimization problem of three of these methods. We show that the proposed optimization problems have closed-form solution. By generalizing one of these constraints to any distribution, we provide generalized methods for constrained graph-based SSL. The proposed methods have a more flexible regularization framework than the corresponding unconstrained methods. More precisely, our methods can deal with any graph Laplacian and use higher order regularization, which is effective on general SSL taks. In order to show the effectiveness of the proposed methods, we provide comprehensive experimental analyses. Specifically, our experiments are subdivided into two parts. In the first part, we evaluate existing graph-based SSL algorithms on time series data to find their weaknesses. In the second part, we evaluate the proposed constrained methods against six state-of-the-art graph-based SSL algorithms on benchmark data sets. Since the widely used best case analysis may hide useful information concerning the SSL algorithms' performance with respect to parameter selection, we used recently proposed empirical evaluation models to evaluate our results. Our results show that our methods outperforms the competing methods on most parameter settings and graph construction methods. However, we found a few experimental settings in which our methods showed poor performance. In order to facilitate the reproduction of our results, the source codes, data sets, and experimental results are freely available.

Employing unlabeled data to improve the classification performance of SVM, and its application in audio event classification

Towards Dynamic Self-Training for Scalable Semi-Supervised Learning on Graphs

In this article we present a semi‐supervised active learning algorithm for pattern discovery in information extraction from textual data. The patterns are reduced regular expressions composed of various characteristics of features useful in information extraction. Our major contribution is a semi‐supervised learning algorithm that extracts information from a set of examples labeled as relevant or irrelevant to a given attribute. The approach is semi‐supervised because it does not require precise labeling of the exact location of features in the training data. This significantly reduces the effort needed to develop a training set. An active learning algorithm is used to assist the semi‐supervised learning algorithm to further reduce the training set development effort. The active learning algorithm is seeded with a single positive example of a given attribute. The context of the seed is used to automatically identify candidates for additional positive examples of the given attribute. Candidate examples are manually pruned during the active learning phase, and our semi‐supervised learning algorithm automatically discovers reduced regular expressions for each attribute. We have successfully applied this learning technique in the extraction of textual features from police incident reports, university crime reports, and patents. The performance of our algorithm compares favorably with competitive extraction systems being used in criminal justice information systems.

Semi-Supervised Learning

A semi-supervised deep learning approach for cropped image detection

The key points of the semi-supervised learning problem are the label smoothness and cluster assumptions. In graph-based semi-supervised learning, graph representations of the data are so important that different graph representations can affect the classification results heavily. We present a novel method to produce a graph called smooth Markov random walk graph which takes into account the two assumptions employed by semi-supervised learning. The new graph is achieved by modifying the eigenspectrum of the transition matrix of Markov random walk graph and is sufficiently smooth with respect to the intrinsic structure of labeled and unlabeled points.We believe the smoother graph will benefit semi-supervised learning. Experiments on artificial and real world dataset indicate that our method provides superior classification accuracy over several state-of-the-art methods.

In real worlds applications, some former research papers have shown that manifold learning tries to discover the non-linear low-dimensional data manifold from a high-dimensional space. Many natural images and the face images are believed to be sampled from a manifold. In this paper, we try to investigate whether discovering such manifold can aid the semi-supervised learning algorithms. We propose a novel graph-based learning algorithm locality preserving graph-based semi-supervised method (LLGSM), which firstly use both labeled and unlabeled examples as unlabeled to discover the manifolds of the data samples and then use the projected labeled examples together with projected unlabeled ones to do classification. Experiments performed on some public image data sets have demonstrated the effectiveness of our algorithm.

Multitask learning in distributed environments is an important problem in the field of federated learning. In many practical problems, human or machine experts try to solve different, but highly correlated tasks, thus their performance can be improved through collaboration, but their private datasets and model parameters cannot be shared with other participants because of business and privacy issues. Semi-supervised learning is also actively researched nowadays, which is motivated by the utilization of the usually free, large set of unlabeled data in concept and representation learning. These unlabeled samples can be utilized in the knowledge sharing of the participants during their collaboration. We propose a method, called FedLinked, which combines the capabilities of these area (federated, multitask and semi-supervised learning) related algorithms in the form of a privacy preserving complex solution-block targeting semi-supervised multitask cross-silo problems. FedLinked is a collaborative learning method, which is implemented through the regularization of representation learning based on the cross-utility of participating clients. We evaluate FedLinked on image classification problems and compare its performance to FedAvg based and non-cooperating clients based solutions in multitask scenarios. Besides, a detailed analysis of our proposed method is provided.

We propose a semisupervised sparse graph learning (SSGL) method for feature extraction of hyperspectral remote sensing imagery in this paper. The proposed SSGL method aims to build a semisupervised sparse graph that can maximize the class discrimination and preserve the local neighborhood information by combining labeled and unlabeled samples. In our semisupervised sparse graph, we connect labeled samples according to their label information and sparse representation, connect unlabeled samples by sparse representation which set unlabeled training samples set as dictionary. Moreover, by setting sparse connections between labeled sample and unlabeled samples, the label information can be well propagated from labeled samples to unlabeled samples. In this way, the similarity of data points can be well modelled in the spectral feature space, and the data distribution characteristics can be better expressed. Experimental results on real hyperspectral images (HSIs) demonstrate the advantages of our method compared to some related feature extraction methods.

Noise-robust semi-supervised learning via fast sparse coding

Traditional graph-based semi-supervised learning (GBSSL) algorithms usually scale badly due to the expensive computational burden. The main bottleneck is that they need to compute the inversion of a huge matrix. In order to alleviate this problem, this paper proposes Neumann series approximation (NSA) to explicitly approximate the inversion process required by conventional GBSSL methodologies, which makes them computationally tractable for relatively large datasets. It is proved that the deviation between the approximation and direct inversion is bounded. Using real-world datasets related to handwritten digit recognition, speech recognition and text classification, the experimental results reveal that NSA accelerates the speed significantly without decreasing too much precision. We also empirically show that NSA outperforms other scalable approaches such as Nystrom method, Takahashi equation, Lanczos process based SVD and AnchorGraph regularization, in terms of both efficiency and accuracy.

In this paper, we propose a novel graph-based semisupervised learning framework, called joint sparse representation and embedding propagation learning (JSREPL). The idea of JSREPL is to join EPL with sparse representation to perform label propagation. Like most of graph-based semisupervised propagation learning algorithms, JSREPL also constructs weights graph matrix from given data. Different from classical approaches which build weights graph matrix and estimate the labels of unlabeled data in sequence, JSREPL simultaneously builds weights graph matrix and estimates the labels of unlabeled data. We also propose an efficient algorithm to solve the proposed problem. The proposed method is applied to the problem of semisupervised image clustering using the ORL, Yale, PIE, and YaleB data sets. Our experiments demonstrate the effectiveness of our proposed algorithm.

Hardware of Distributed WSNs is limited, which turns into a big challenge. In recent years, there are some pattern-based query strategies have been proposed. But compared to the methods based on machine learning, these methods need high energy consumption in the calculation of distance. In this paper, we propose a pattern-based query strategy based on semi-supervised machine learning in wireless sensor network. The strategy uses historical time series data streams for o†ine machine learning. First of all, we flnd out the flxed length subsequences and screen out the suitable data sequences as training dataset and sample dataset. And then, because labeled data is less in practical application, we do label propagation with graph-based semi-supervised machine learning algorithm, Local and Global Consistency (LGC), to increase the number of labeled data sequences. Next, we use the supervised learning algorithm Support Vector Machine (SVM) to train a classifler. Finally, for a new data sequence, we can directly get its category through the classifler. Experimental results show that the performance of our strategy is almost better than other pattern-based query strategies on classiflcation accuracy and energy consumption.

Following the success of supervised learning, semi-supervised learning (SSL) is now becoming increasingly popular. SSL is a family of methods, which in addition to a labeled training set, also use a sizable collection of unlabeled data for fitting a model. Most of the recent successful SSL methods are based on pseudo-labeling approaches: letting confident model predictions act as training labels. While these methods have shown impressive results on many benchmark datasets, a drawback of this approach is that not all unlabeled data are used during training. We propose a new SSL algorithm, DoubleMatch, which combines the pseudo-labeling technique with a self-supervised loss, enabling the model to utilize all unlabeled data in the training process. We show that this method achieves state-of-the-art accuracies on multiple benchmark datasets while also reducing training times compared to existing SSL methods. Code is available at https://github.com/walline/doublematch.