Active learning for hyperspectral image classification with a stacked autoencoders based neural network

Active learning can effectively reduce labelling effort for remote sensing image classification. In this paper, we propose a new active learning method for hyperspectral image classification. We consider batch mode active learning and relatively large amount of data which can be a problem when using current state of the art algorithm based on kernel machines. The active learning procedure is based on the uncertainty sampling strategy and a deep neural network. Stacked autoencoders trained on redundant spatial and spectral features and a few labeled training samples are used to initialize a deep neural network. Uncertainty for a given sample is measured by the difference between the largest two class outputs of the neural network. The less difference there is, the more uncertainty the sample has. Batch of samples with most uncertainty will be selected after label query and added into the training set. Then the neural network is retrained. And such active batch selection will iterate until the budget (the upper limit of label queries) is reached. Experimental results on Pavia university dataset showed that our method outperforms the current support vector machines (SVMs) based multiclass/level uncertainty (MCLU) method both in classification accuracy and generalization capability.

Active Learning is a machine learning and data mining technique that selects the most informative samples for labeling and uses them as training data; it is especially useful when there are large amount of unlabeled data and labeling them is expensive. Recently, batch-mode active learning, where a set of samples are selected concurrently for labeling, based on their collective merit, has attracted a lot of attention. The objective of batch-mode active learning is to select a set of informative samples so that a classifier learned on these samples has good generalization performance on the unlabeled data. Most of the existing batch-mode active learning methodologies try to achieve this by selecting samples based on certain criteria. In this article we propose a novel criterion which achieves good generalization performance of a classifier by specifically selecting a set of query samples that minimize the difference in distribution between the labeled and the unlabeled data, after annotation. We explicitly measure this difference based on all candidate subsets of the unlabeled data and select the best subset. The proposed objective is an NP-hard integer programming optimization problem. We provide two optimization techniques to solve this problem. In the first one, the problem is transformed into a convex quadratic programming problem and in the second method the problem is transformed into a linear programming problem. Our empirical studies using publicly available UCI datasets and two biomedical image databases demonstrate the effectiveness of the proposed approach in comparison with the state-of-the-art batch-mode active learning methods. We also present two extensions of the proposed approach, which incorporate uncertainty of the predicted labels of the unlabeled data and transfer learning in the proposed formulation. In addition, we present a joint optimization framework for performing both transfer and active learning simultaneously unlike the existing approaches of learning in two separate stages, that is, typically, transfer learning followed by active learning. We specifically minimize a common objective of reducing distribution difference between the domain adapted source, the queried and labeled samples and the rest of the unlabeled target domain data. Our empirical studies on two biomedical image databases and on a publicly available 20 Newsgroups dataset show that incorporation of uncertainty information and transfer learning further improves the performance of the proposed active learning based classifier. Our empirical studies also show that the proposed transfer-active method based on the joint optimization framework performs significantly better than a framework which implements transfer and active learning in two separate stages.

Support vector machine (SVM) active learning is one popular and successful technique for relevance feedback in content-based image retrieval (CBIR). Despite the success, conventional SVM active learning has two main drawbacks. First, the performance of SVM is usually limited by the number of labeled examples. It often suffers a poor performance for the small-sized labeled examples, which is the case in relevance feedback. Second, conventional approaches do not take into account the redundancy among examples, and could select multiple examples that are similar (or even identical). In this work, we propose a novel scheme for explicitly addressing the drawbacks. It first learns a kernel function from a mixture of labeled and unlabeled data, and therefore alleviates the problem of small-sized training data. The kernel will then be used for a batch mode active learning method to identify the most informative and diverse examples via a min-max framework. Two novel algorithms are proposed to solve the related combinatorial optimization: the first approach approximates the problem into a quadratic program, and the second solves the combinatorial optimization approximately by a greedy algorithm that exploits the merits of submodular functions. Extensive experiments with image retrieval using both natural photo images and medical images show that the proposed algorithms are significantly more effective than the state-of-the-art approaches. A demo is available at http://msm.cais.ntu.edu.sg/LSCBIR/.

The Sixth International Symposium on Neural Networks (ISNN 2009)

Deep neural network has been extensively applied to hyperspectral image (HSI) classification recently. However, its success is greatly attributed to numerous labeled samples, whose acquisition costs a large amount of time and money. In order to improve the classification performance while reducing the labeling cost, this article presents an active deep learning approach for HSI classification, which integrates both active learning and deep learning into a unified framework. First, we train a convolutional neural network (CNN) with a limited number of labeled pixels. Next, we actively select the most informative pixels from the candidate pool for labeling. Then, the CNN is fine-tuned with the new training set constructed by incorporating the newly labeled pixels. This step together with the previous step is iteratively conducted. Finally, Markov random field (MRF) is utilized to enforce class label smoothness to further boost the classification performance. Compared with the other state-of-the-art traditional and deep learning-based HSI classification methods, our proposed approach achieves better performance on three benchmark HSI data sets with significantly fewer labeled samples.

In content-base image retrieval, relevance feedback (RF) schemes based on support vector machine (SVM) have been widely used to narrow the semantic gap between low-level visual features and high-level human perception. However, the performance of image retrieval with SVM active learning is known to be poor when the training data is insufficient. In this paper, the problem is solved by incorporating the unlabelled images into the learning process. We proposed a semi-supervised active learning algorithm which uses not only labeled training samples but also unlabeled ones to build better models. In relevance feedback, active learning algorithm is often used to reduce the cost of labeling by selecting only the most informative data. In addition, we introduced a semi-supervised approach which employed Nearest-Neighbor technique to label the unlabeled sample with a certain degree of uncertainty in its class information. Using these samples, Fuzzy support vector machine (FSVM) which takes into account the fuzzy nature of some training samples during its training is trained. We compared our method with standard active SVM on a database of 10,000 images, the experiment results show that the efficiency of SVM active learning can be improved by incorporating unlabelled images, and thus improve the overall retrieval performance. DOI: http://dx.doi.org/10.11591/telkomnika.v11i7.2807

The high-dimensional data space generated by hyperspectral sensors introduces challenges for the conventional data analysis techniques. Popular dimensionality reduction techniques usually assume a Gaussian distribution, which may not be in accordance with real life. Metric learning methods, which explore the global data structure of the labeled training samples, have proved to be very efficient in hyperspectral fields. However, we can go further by utilizing locally adaptive decision constraints for the labeled training samples per class to obtain an even better performance. In this paper, we present the locally adaptive dimensionality reduction metric learning (LADRml) method for hyperspectral image classification. The aims of the presented method are: 1) first, to utilize the limited training samples to reduce the dimensionality of data without a certain distribution hypothesis; and 2) second, to better handle data with complex distributions by the use of locally adaptive decision constraints, which can assess the similarity between a pair of samples based on the distance changes before and after metric learning. The experimental results obtained with a number of challenging hyperspectral image datasets demonstrate that the proposed LADRml algorithm outperforms the state-of-the-art dimensionality reduction and metric learning methods.

Gapped k-mers frequency vectors (gkm-fv) has been presented for extracting sequence features. Coupled with support vector machine (gkm-SVM), gkm-fvs have been used to achieve effective sequence-based predictions. However, the huge computation of a large kernel matrix prevents it from using large amount of data. It is unclear how to combine gkm-fvs with other data sources in the context of string kernel. On the other hand, the high dimensionality, colinearity, and sparsity of gkm-fvs hinder the use of many traditional machine learning methods without a kernel trick. Therefore, we proposed a flexible and scalable framework gkm-DNN to achieve feature representation from high-dimensional gkm-fvs using deep neural networks (DNN). We first proposed a more concise version of gkm-fvs, which significantly reduce the dimension of gkm-fvs. Then, we implemented an efficient method to calculate the gkm-fv of a given sequence at the first time. Finally, we adopted a DNN model with gkm-fvs as inputs to achieve efficient feature representation and a prediction task. Here, we took the transcription factor binding site prediction as an illustrative application and applied gkm-DNN onto 467 small and 69 big human ENCODE ChIP-seq datasets to demonstrate its performance and compared it with the state-of-the-art method gkm-SVM.

Current deep learning methods are regarded as favorable if they empirically perform well on dedicated test sets. This mentality is seamlessly reflected in the resurfacing area of continual learning, where consecutively arriving data is investigated. The core challenge is framed as protecting previously acquired representations from being catastrophically forgotten. However, comparison of individual methods is nevertheless performed in isolation from the real world by monitoring accumulated benchmark test set performance. The closed world assumption remains predominant, i.e. models are evaluated on data that is guaranteed to originate from the same distribution as used for training. This poses a massive challenge as neural networks are well known to provide overconfident false predictions on unknown and corrupted instances. In this work we critically survey the literature and argue that notable lessons from open set recognition, identifying unknown examples outside of the observed set, and the adjacent field of active learning, querying data to maximize the expected performance gain, are frequently overlooked in the deep learning era. Hence, we propose a consolidated view to bridge continual learning, active learning and open set recognition in deep neural networks. Finally, the established synergies are supported empirically, showing joint improvement in alleviating catastrophic forgetting, querying data, selecting task orders, while exhibiting robust open world application.

In machine learning,active learningrefers to algorithms that autonomously select the data points from which they will learn. There are many data mining applications in which large amounts of unlabeled data are readily available, but labels (e.g., human annotations or results coming from complex experiments) are costly to obtain. In such scenarios, an active learning algorithm aims at identifying data points that, if labeled and used for training, would most improve the learned model. Labels are then obtained only for the most promising data points. This speeds up learning and reduces labeling costs. Support vector machine (SVM) classifiers are particularly well‐suited for active learning due to their convenient mathematical properties. They perform linear classification, typically in a kernel‐induced feature space, which makes expressing the distance of a data point from the decision boundary straightforward. Furthermore, heuristics can efficiently help estimate how strongly learning from a data point influences the current model. This information can be used to actively select training samples. After a brief introduction to the active learning problem, we discuss different query strategies for selecting informative data points and review how these strategies give rise to different variants of active learning withSVMs.This article is categorized under:Technologies > Machine Learning

Asymmetric propagation based batch mode active learning for image retrieval

Semantic segmentation is a task that heavily relies on the annotated data. The image annotation cost is very expensive. While active learning aims to select the most valuable samples by an iterative procedure. It can reduce the annotation cost and improve the performance of classification. In semantic segmentation, its more common to select a batch of instances instead of a single instance at each iteration. In this paper, we propose a novel batch mode active learning algorithm for semantic segmentation. Different from the previous active learning algorithms for the image classification, we first introduce a new selecting criterion : the image prior of nonlocal self-similarity. It can measure the interaction between the pixels of the image. We combine the informativeness and representativeness with the criterion of nonlocal self-similarity to complete the selection of images at each iteration. In addition, we also use the model uncertainty to measure the information of samples. The model uncertainty is captured by using Monte Carlo Dropout in the semantic segmentation model. In this work, we evaluate our method on the CamVid and PASCAL VOC 2012 datasets. The importance of the nonlocal self-similarity is also assessed. The experiments demonstrate that our algorithm outperforms current state-of-the-art active learning methods over the segmentation performance.

Due to the fact that it is expensive and time-consuming to annotate a large amount of data, the available labeled data to train a deep neural network is usually scarce, resulting in the poor performance of the deep classification network in most cases. In this paper, we propose a novel active learning method to train a competitive deep classification neural network by labeling a limited amount of diverse images. The proposed method advances most of the active learning methods in two aspects. First, active learning is enhanced with a co-auxiliary learning strategy. We use an auxiliary network to provide diverse pseudo-labels for the primary network. The auxiliary network is also adopted to remove a part of redundancy information from the candidate pool of active learning when image predictions of the primary network and the auxiliary network are the same. Meanwhile, the primary network can also provide pseudo-labels to improve the performance of the auxiliary network. Second, we further remove the redundancy within the query samples with multi-level diversity selection. The multi-level diversity strategy not only considers the feature-level diversity but also the class-level diversity with the predictions of the primary network, and it can select both representative and uncertain samples effectively and efficiently from the large-scale data. The extensive experiments on four large-scale datasets show that the proposed method outperforms the state-of-the-art methods.

Adaptive batch mode active learning with deep similarity

In this paper, a region-partitioning active learning (AL) technique is proposed for classification of remote sensing (RS) images based on the support vector machines (SVM) classifier. In the batch-mode AL process, diversity information is required to select a batch of informative samples. A new AL technique that aims to introduce diversity information is proposed based on relative positions of candidate samples in the feature space. The proposed technique selects informative samples according to an uncertainty criterion at each iteration. These samples are selected with an extra constraint to guarantee that they are not located in the same region of the feature space. The proposed technique is compared with state-of-the-art methods adopted in the RS community. Experimental tests were performed on three data sets, including one very high spatial resolution multispectral data set and two hyperspectral data sets. The proposed algorithm displays a classification performance that is similar to or even better than the state-of-the-art methods. In addition, the proposed algorithm performs efficiently in terms of computational time.