- Research Article
106
- 10.1016/j.patcog.2018.11.025
Semi-supervised person re-identification using multi-view clustering
- Nov 19, 2018
- Pattern Recognition
- Xiaomeng Xin + 5 more +5
Semi-supervised person re-identification using multi-view clustering
Semi-supervised person re-identification (Re-ID) is an extension of the existing popular Re-ID research, which only uses a small portion of labeled data, while the majority of the training samples are unlabeled. This paper approaches the problem by constructing a set of heterogeneous convolutional neural networks (CNNs) fine-tuned by utilizing the labeled training samples, and then propagating the labels to the unlabeled portion for further fine-tuning the overall system in a self-paced manner. In this work, a novel self-paced multi-view clustering is presented to generate pseudo labels for unlabeled training samples, which combines multiple heterogeneous CNNs features to cluster. In our clustering method, we introduce a self-paced regularizer to select reliable samples for fine-tuning each CNNs by minimizing ranking loss and identification loss. Specifically, we select a small portion of unlabeled training data when multiple CNNs are weak. With CNNs become stronger, more and more unlabeled samples are selected. Pseudo label estimation and CNNs training are improved simultaneously, which optimize alternatively until all the unlabeled training samples are selected. In our framework, both the optimization of multiple CNNs training and multi-view clustering on unlabeled training samples are self-paced optimizing procedure. Extensive experiments have been conducted on two large-scale Re-ID datasets to demonstrate the superiority of the proposed method.
Semi-supervised person re-identification using multi-view clustering
Semi-supervised person re-identification using multi-view clustering
Sparse Representation-Based Semi-Supervised Regression for People Counting
Label imbalance and the insufficiency of labeled training samples are major obstacles in most methods for counting people in images or videos. In this work, a sparse representation-based semi-supervised regression method is proposed to count people in images with limited data. The basic idea is to predict the unlabeled training data, select reliable samples to expand the labeled training set, and retrain the regression model. In the algorithm, the initial regression model, which is learned from the labeled training data, is used to predict the number of people in the unlabeled training dataset. Then, the unlabeled training samples are regarded as an over-complete dictionary. Each feature of the labeled training data can be expressed as a sparse linear approximation of the unlabeled data. In turn, the labels of the labeled training data can be estimated based on a sparse reconstruction in feature space. The label confidence in labeling an unlabeled sample is estimated by calculating the reconstruction error. The training set is updated by selecting unlabeled samples with minimal reconstruction errors, and the regression model is retrained on the new training set. A co-training style method is applied during the training process. The experimental results demonstrate that the proposed method has a low mean square error and mean absolute error compared with those of state-of-the-art people-counting benchmarks.
Read moreAutolabeling-Enhanced Active Learning for Cost-Efficient Surface Defect Visual Classification
Active learning can reduce the human effort required for labeling training samples while preserving the performance of visual classifiers. However, existing active learning frameworks cannot be used to perform visual classification of industrial product surface defects because they still require intensive manual annotation efforts. In this study, we propose a cost-efficient autolabeling-enhanced active learning (ALEAL) framework to reduce the human annotation effort required for surface defect visual classification. The proposed ALEAL framework employs a deep convolutional neural network (CNN) as a visual classifier trained from an initial set of human-labeled training samples. Then, the collected unlabeled training samples are input into the classifier for category confidence estimation. Next, a novel diverse cost-effective query strategy (DCEQS) is proposed to select some high-confidence samples for autolabeling and some informative samples for sample proposals that need labeling. Subsequently, to further reduce the human annotation effort, a novel autolabeling module is proposed and introduced in ALEAL that can automatically label a portion of the informative unlabeled training samples selected by the DCEQS. In this study, a novel attention-based similarity measurement network (ASMN) is proposed as an implementation of this autolabeling module by measuring the similarity between unlabeled and labeled samples. Finally, the remaining unlabeled samples are annotated by human experts, and all the newly labeled samples are used to retrain the classifier. Through the autolabeling process from the DCEQS and ASMN, ALEAL can automatically label additional training samples and achieve a competitive performance while requiring few human-labeled training samples, which is highly important in industrial applications. Extensive experimental results show that, compared with popular active learning methods, ALEAL can dramatically reduce the effort involved in human annotation and achieve state-of-the-art cost efficiency for the visual classification of industrial product surface defects.
Read moreA new semi-supervised approach for hyperspectral image classification with different active learning strategies
Supervised hyperspectral image classification is a difficult task due to the unbalance between the high dimensionality of the data and the limited availability of labeled training samples in real analysis scenarios. While the collection of labeled samples is generally difficult, expensive and time-consuming, unlabeled samples can be generated in a much easier way. This observation has fostered the idea of adopting semi-supervised learning (SSL) techniques in hyperspectral image classification. The main assumption of such techniques is that the new (unlabeled) training samples can be obtained from a (limited) set of available labeled samples without significant effort/cost. In this paper, we develop a new framework for SSL which exploits active learning (AL) for unlabeled sample selection. Specifically, we use AL to select the most informative unlabeled training samples and further evaluate two different strategies for active sample selection. In this work, the proposed approach is illustrated with the sparse multinomial logistic regression (SMLR) classifier learned with the MLR via variable splitting and augmented Lagrangian (LORSAL) algorithm. Our experimental results with a real hyperspectral image collected by the NASA Jet Propulsion Laboratory's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) indicate that the use of AL for unlabeled sample selection represents an effective and promising strategy in the context of semi-supervised hyperspectral data classification.
Read moreA semi-supervised learning method for remote sensing data mining
New approaches are needed to extract useful patterns from increasingly large multi-spectral remote sensing image databases in order to understand global climatic changes, vegetation dynamics, ocean processes, etc. Supervised learning, which is often used in land cover (thematic) classification of remote sensing imagery, requires large amounts of accurate training data. However, in many situations it is very difficult to collect labels for all training samples. In this paper we explore methods that utilize unlabeled samples in supervised learning for thematic information extraction from remote sensing imagery. Our objectives are to understand the impact of parameter estimation with small learning samples on classification accuracy, and to augment the parameter estimation with unlabeled training samples to improve land cover predictions. We have developed a semi-supervised learning method based on the expectation-maximization (EM) algorithm, and maximum likelihood and maximum a posteriori classifiers. This scheme utilizes a small set of labeled and a large number of unlabeled training samples. We have conducted several experiments on multi-spectral images to understand the impact of unlabeled samples on the classification performance. Our study shows that though in general classification accuracy improves with the addition of unlabeled training samples, it is not guaranteed to get consistently higher accuracies unless sufficient care is exercised when designing a semi-supervised classifier
Read moreAspect-based Opinion Summarization with Convolutional Neural Networks
This paper studies Aspect-based Opinion Summarization (AOS) of reviews on particular products. In practice, an AOS system needs to address two core subtasks, aspect extraction and sentiment classification. Most existing approaches to aspect extraction, using linguistic analysis or topic modeling, are general across different products but not precise enough or suitable for particular products. Instead we take a less general but more precise scheme, which directly maps each review sentence into pre-defined aspects. To tackle aspect mapping and sentiment classification, we propose two Convolutional Neural Network (CNN) based methods, cascaded CNN and multitask CNN. Cascaded CNN contains two levels of convolutional networks. Multiple CNNs at level 1 deal with aspect mapping task, and a single CNN at level 2 deals with sentiment classification. Multitask CNN also contains multiple aspect CNNs and a sentiment CNN, but different networks share the same word embeddings. Experimental results show that both cascaded and multitask CNNs with pre-trained word embedding outperform linear classifiers, and multitask CNN generally performs better than cascaded CNN.
Read moreNew results in semi-supervised learning using adaptive classifier fusion
In typical classification problems the data used to train a model for each class is often correctly labeled, and so that fully supervised learning can be utilized. For example, many illustrative labeled data sets can be found at sources such as the UCI Repository for Machine Learning (http://archive.ics.uci.edu/ml/), or at the Keel Data Set Repository (http://www.keel.es). However, increasingly many real world classification problems involve data that contain both labeled and unlabeled samples. In the latter case, the data samples are assumed to be missing all class label information, and when used as training data these samples are considered to be of unknown origin (i.e., to the learning system, actual class membership is completely unknown). Typically, when presented with a classification problem containing both labeled and unlabeled training samples, a technique that is often used is to throw out the unlabeled data. In other words, the unlabeled data are not included with existing labeled data for learning, and which can result in a poorly trained classifier that does not reach its full performance potential. In most cases, the primary reason that unlabeled data are not often used for training is that, and depending on the classifier, the correct optimal model for semi-supervised classification (i.e., a classifier that learns class membership using both labeled and unlabeled samples) can be far too complicated to develop. In previous work, results were shown based on the fusion of binary classifiers to improve performance in multiclass classification problems. In this case, Bayesian methods were used to fuse binary classifier fusion outputs, while selecting the most relevant classifier pairs to improve the overall classifier decision space. Here, this work is extended by developing new algorithms for improving semi-supervised classification performance. Results are demonstrated with real data form the UCI and Keel Repositories.
Read moreAutomated Documentation of Multiple Colonoscopy Quality Measures in Real-Time with Convolutional Neural Networks
Introduction: Convolutional neural networks (CNNs), a subset of deep learning and artificial intelligence, are at the cusp of revolutionizing medical practice. Toward the aim of improving colonoscopy quality while reducing costs of mundane tasks and overhead, we developed several CNNs capable of accurately identifying and classifying polyps and recording procedure times, cecal intubation, prep quality, and tools used. Our goal was to implement these multiple CNNs during live colonoscopy using a workstation with off-the shelf components that could interface with any scope manufacturer's video signal. Methods: Previously and at this meeting, we reported details of the training, validation, speed and accuracies of CNNs we developed for polyp detection (99.5%, 0.995 AUC, 90 FPS), optical pathology (93.1%, 90 FPS), cecal recognition (94%, 0.92 AUC, 90 FPS), Boston Bowel Prep Score (BBPS) (96.4%, 0.96 AUC, 1100 FPS), devices used (90-99%, 0.90-9.99 AUC, 125 FPS) and times of scope In/Out (99%, 0.99 AUC, 90 FPS). To accomplish our goal, we focused on optimization of the size, speed and accuracy our CNN models while adjusting timing of CNN activations during colonoscopy, and utilizing proprietary methods to: 1) reduce CPU/GPU loads; and 2) overlay CNNs on live video using universal video signals with no perceptible frame drop. Results: We have succeeded is running 7 separate CNNs with live feedback during colonoscopy (video link:https://youtu.be/_5MPjY8uqYg). Parameters are shown and recorded during colonoscopy including time of insertion, time of cecal intubation, time of scope out (cecal intubation rate, insertion time and withdrawal time), polyp detection/localization, polyp optical pathology (real-time ADR, recommended surveillance interval, and potential for “resect and discard”), tool use and intervention (CPT codes), and BBPS (a key quality measure). Polyp size and morphology (Paris classification), missed mucosal visualization and Mayo Endoscopic Score will soon be added.2761 Figure 1. Representative screen shots of multiple CNNs running during colonoscopyConclusion: We have demonstrated the feasibility of running multiple CNNs simultaneously that provide live feedback during colonoscopy, enhance colonoscopy quality and record key quality measures for automated procedure documentation, billing, internal quality improvement, research, and reporting to CMS registries such as GIQuIC. These can be accomplished with workstation built with off-the-shelf components. Validation awaits randomized multicenter studies.
Read moreA Semi-Supervised Stacked Autoencoder Using the Pseudo Label for Classification Tasks.
The efficiency and cognitive limitations of manual sample labeling result in a large number of unlabeled training samples in practical applications. Making full use of both labeled and unlabeled samples is the key to solving the semi-supervised problem. However, as a supervised algorithm, the stacked autoencoder (SAE) only considers labeled samples and is difficult to apply to semi-supervised problems. Thus, by introducing the pseudo-labeling method into the SAE, a novel pseudo label-based semi-supervised stacked autoencoder (PL-SSAE) is proposed to address the semi-supervised classification tasks. The PL-SSAE first utilizes the unsupervised pre-training on all samples by the autoencoder (AE) to initialize the network parameters. Then, by the iterative fine-tuning of the network parameters based on the labeled samples, the unlabeled samples are identified, and their pseudo labels are generated. Finally, the pseudo-labeled samples are used to construct the regularization term and fine-tune the network parameters to complete the training of the PL-SSAE. Different from the traditional SAE, the PL-SSAE requires all samples in pre-training and the unlabeled samples with pseudo labels in fine-tuning to fully exploit the feature and category information of the unlabeled samples. Empirical evaluations on various benchmark datasets show that the semi-supervised performance of the PL-SSAE is more competitive than that of the SAE, sparse stacked autoencoder (SSAE), semi-supervised stacked autoencoder (Semi-SAE) and semi-supervised stacked autoencoder (Semi-SSAE).
Read moreAdvanced Feature Fusion Algorithm Based on Multiple Convolutional Neural Network for Scene Recognition
Scene recognition is a popular open problem in the computer vision field. Among lots of methods proposed in recent years, Convolutional Neural Network (CNN) based approaches achieve the best performance in scene recognition. We propose in this paper an advanced feature fusion algorithm using Multiple Convolutional Neural Network (MultiCNN) for scene recognition. Unlike existing works that usually use individual convolutional neural network, a fusion of multiple different convolutional neural networks is applied for scene recognition. Firstly, we split training images in two directions and apply to three deep CNN model, and then extract features from the last full-connected (FC) layer and probabilistic layer on each model. Finally, feature vectors are fused with different fusion strategies in groups forwarded into SoftMax classifier. Our proposed algorithm is evaluated on three scene datasets for scene recognition. The experimental results demonstrate the effectiveness of proposed algorithm compared with other state-of-art approaches.
Read moreFusing CNN Models for Improved Parkinson’s Disease Detection from Handwritten Features
Neurodegenerative Diseases (NGD) are a group of progressive neurological disorders, including cognitive decline as Alzheimer’s Disease (AD) and Parkinson’s Disease (PD) that result in the progressive loss of neuronal structure or function. This can lead to a variety of symptoms, in this study we are interested in PD one of the neurological diseases that causes the loss of dopamine-producing neurons in the brain, leading to movement disorders., movement disorders, and dementia. In this study, we are interested in PD, one of the neurological diseases that causes the loss of dopamine-producing neurons in the brain, leading to movement disorders. Early diagnosis of PD seems to be the best way to improve the quality of the patient's life by prescribing the appropriate treatment. The relevant observed symptoms are often subtle; these include slow movement, decreased performance in carrying out daily tasks, tremors, muscle stiffness, and various other psychological symptoms. Handwriting or drawing analysis is one of the dominant mechanisms supporting the early diagnosis and assessment of PD. Based on that, to improve the reliability of Parkinson's Disease (PD) detection, we implemented various data augmentation techniques to increase the size of the dataset. We then deployed and trained various architectures of deep Convolutional Neural Networks (CNNs), each capturing different salient features and aspects of the input data due to their unique layout and structure. We then carefully selected promising feature vectors and applied various early fusion strategies before the final classification step. Early fusion combines the feature vectors extracted by multiple CNNs at an early stage, allowing the classification model to learn and recognize different representations of the data provided by these CNNs. This technique is very beneficial as it improves the model's ability to capture a wide range of features and improves overall system performance. Our experimental results demonstrate that the fusion of frozen features from multiple deep CNN models yields a substantial improvement in accuracy, achieving an impressive exactness rate of 96.29%. This performance surpasses that of individual CNN models and even outperforms other state-of-the-art approaches, highlighting the effectiveness of our fusion-based strategy in enhancing PD detection accuracy.
Read moreMotor Imagery, Execution, and Observation Classification using Small Amount of EEG Data with Multiple Two-Class CNNs.
This study attempted to classify a small amount of electroencephalogram (EEG) data on five states: four tasks involving right index-finger flexion (kinesthetic motor imagery, visual motor imagery, motor execution, and motor observation) and resting with eyes open. We employed a convolutional neural network (CNN) as a classifier and compared the classification accuracies of two types of CNNs: 1) a "single five-class CNN," which classified the aforementioned states with a single CNN and 2) "multiple two-class CNNs," wherein ten CNNs that classify pairs of states were combined. In addition, the classification accuracies were compared between two scenarios: one wherein the EEGs from all 19 scalp probe electrodes (19-channel EEG) were adopted as input data for the CNN, and the other wherein the EEGs of four regions closely related to the motor execution and observation of the index finger (4-channel EEG) were adopted. The classification accuracies of the single five-class CNN with 19- and 4-channel EEGs were 48.2 ± 5.9% and 46.6 ± 6.9%, respectively, and those of the multiple two-class CNNs with 19- and 4-channel EEGs were 52.8 ± 9.7% and 47.5 ± 9.4%, respectively. These results indicate the effectiveness of multiple two-class CNNs that utilize the EEGs of all scalp electrodes as input data for classifying motor imagery, execution, and observation, even in the case of the marginal dataset.
Read moreTransductive Semisupervised Deep Hashing.
Deep hashing methods have shown their superiority to traditional ones. However, they usually require a large amount of labeled training data for achieving high retrieval accuracies. We propose a novel transductive semisupervised deep hashing (TSSDH) method which is effective to train deep convolutional neural network (DCNN) models with both labeled and unlabeled training samples. TSSDH method consists of the following four main ingredients. First, we extend the traditional transductive learning (TL) principle to make it applicable to DCNN-based deep hashing. Second, we introduce confidence levels for unlabeled samples to reduce adverse effects from uncertain samples. Third, we employ a Gaussian likelihood loss for hash code learning to sufficiently penalize large Hamming distances for similar sample pairs. Fourth, we design the large-margin feature (LMF) regularization to make the learned features satisfy that the distances of similar sample pairs are minimized and the distances of dissimilar sample pairs are larger than a predefined margin. Comprehensive experiments show that the TSSDH method can produce superior image retrieval accuracies compared to the representative semisupervised deep hashing methods under the same number of labeled training samples.
Read moreMixture of counting CNNs
This paper proposes a crowd counting method. Crowd counting is difficult because of significant appearance changes of a target which caused by density and scale changes. Conventional crowd counting methods commonly utilize one predictor (e.g., regression and multi-class classifier). However, such only one predictor can not count targets with significant appearance changes well. In this paper, we propose to predict the number of targets using multiple convolutional neural networks (CNNs) specialized to a specific appearance, and those CNNs are adaptively selected according to the appearance of a test image. By integrating the selected CNNs, the proposed method has the robustness to large appearance changes. In experiments, we confirm that the proposed method can count crowd with lower counting error than VGGNet, integration of CNNs with fixed weights and conventional counting methods. Moreover, we confirm that each CNN automatically specialized to a specific appearance (e.g., dense region and sparse region) of crowd through training of CNNs.
Read moreMultiscale CNNs Ensemble Based Self-Learning for Hyperspectral Image Classification
Fully supervised methods for hyperspectral image (HSI) classification usually require a considerable number of training samples to obtain high classification accuracy. However, it is time-consuming and difficult to collect the training samples. Under this context, semisupervised learning, which can effectively augment the number of training samples and extract the underlying information among the unlabeled samples, gained much attention. In this letter, we propose a Multiscale convolutional neural networks (CNNs) Ensemble Based Self-Learning (MCE-SL) method for semisupervised HSI classification. Generally, the proposed MCE-SL method consists of the following two stages. In the first stage, the spatial information of different scales from limited labeled training samples are extracted to train several CNN models. In the second stage, the trained multiscale CNNs are used to classify the unlabeled samples. After error correction, the problem of label partially incorrect is alleviated, and unlabeled samples with high confidence will be added to the original training data set for the next training iteration. We conduct comprehensive experiments on two real HSI data sets, and the experimental results show that the proposed MCE-SL can obtain better classification performance compared with several traditional semisupervised methods in few iterations.
Read more