A Semisupervised Cascade Classification Algorithm

A new graph-based semi-supervised method for surface defect classification

Deep learning has been widely used in various tasks such as computer vision, natural language processing, predictive analysis, and recommendation systems in the past decade. However, practical scenarios often lack labeled data, posing challenges for traditional supervised methods. Semi-supervised classification methods address this by leveraging both labeled and unlabeled data to enhance model performance, but they face challenges in effectively utilizing unlabeled data and distinguishing reliable information from unreliable sources. This paper introduced ReliaMatch, a semi-supervised classification method that addresses these challenges by using a confidence threshold. It incorporates a curriculum learning stage, feature filtering, and pseudo-label filtering to improve classification accuracy and reliability. The feature filtering module eliminates ambiguous semantic features by comparing labeled and unlabeled data in the feature space. The pseudo-label filtering module removes unreliable pseudo-labels with low confidence, enhancing algorithm reliability. ReliaMatch employs a curriculum learning training mode, gradually increasing training dataset difficulty by combining selected samples and pseudo-labels with labeled data. This supervised approach enhances classification performance. Experimental results show that ReliaMatch effectively overcomes challenges associated with the underutilization of unlabeled data and the introduction of error information, outperforming the pseudo-label strategy in semi-supervised classification.

A simple and efficient semi-supervised classification method is presented. An unsupervised spectral mapping method is extended to a semi-supervised situation with multiplicative modulation of similarities between data. Our proposed algorithm is derived by linearization of this nonlinear semi-supervised mapping method. Experiments using the proposed method for some public benchmark data and color image data reveal that our method outperforms a supervised algorithm using the linear discriminant analysis and a previous semi-supervised classification method.

Machine learning, especially deep learning, has made great achievements in application domains such as computer vision, speech recognition, machine translation, etc. This great success not only depends on the progress of model design but also on the emergence of large-scale annotated benchmark datasets. However, in some application domains, it is difficult to apply machine learning methods to solve the new arising problems due to the lack of standardized and annotated datasets. People have to find their way from the very beginning of the data collecting, cleaning, and labeling. Unfortunately, there is not a guiding framework to deal with such a situation. This paper pays attention to it and proposes a general machine learning framework to solve the pratical problems from the scratch. It contains two main stages implemented by an unsupervised clustering method and a semi-supervised learning method representatively. In the first stage, an unsupervised clustering method is utilized to find representative data samples which are more important and can be used to do data cleaning and manual labeling. In the second stage, a semi-supervised method is adopted to predict the labels for the rest data samples and to construct a larger annotated dataset. A case study in the texture perception field has been done to confirm the effectiveness and efficiency of the proposed application framework.

Sample-weighted fused graph-based semi-supervised learning on multi-view data

Semi-supervised classification on data streams with recurring concept drift and concept evolution

Many learning techniques have been applied to identify disease-associated genes. At the early, they usually approached this problem as a binary classification, where training set is comprised of positive and negative samples. In which, positive samples are constructed from known disease genes, whereas negative samples are the remaining which are not known to be associated with diseases. This is the limitation of the binary classification-based solutions, since the negative training set should be actual non-disease genes; however, construction of this set is nearly impossible in biomedical researches. Therefore, to reduce this uncertainty, more realistic classification-based methods have been proposed. For instance, unary classification technique based on one-class SVM method was proposed by learning from only positive samples. In addition, the remaining set may contain unknown disease genes; therefore, semi-supervised methods such as binary semi-supervised and positive and unlabeled (PU) learning classifications have been proposed. In particular, PU learning methods, which learn from both known disease genes and the remaining genes, were shown to outperform others. In these studies, data sources are usually represented by vectorial form for binary classifiers, while they are in kernel matrices for unary and PU learning ones. The kernel-based data fusion is only suitable for data with different types and it seems unfair for the comparison based on different data representations. Therefore, in this study, we compared different classification techniques for the disease gene prediction based on vectorial representation of samples. The simulation result showed that the unary classification technique, which combines both density and class probability estimation strategies, achieved the best performance, whereas it is worst for the one-class SVM-based method. Interestingly, performance of the best binary classification technique is comparable with that of biased SVM-based PU learning and binary semi-supervised classification methods. And, they are all better than the multi-level SVM-based one.

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2008 SIAM International Conference on Data Mining (SDM)Semi-Supervised Classification with UniversumDan Zhang, Jingdong Wang, Fei Wang, and Changshui ZhangDan Zhang, Jingdong Wang, Fei Wang, and Changshui Zhangpp.323 - 333Chapter DOI:https://doi.org/10.1137/1.9781611972788.29PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract The Universum data, defined as a collection of “non-examples” that do not belong to any class of interest, have been shown to encode some prior knowledge by representing meaningful concepts in the same domain as the problem at hand. In this paper, we address a novel semi-supervised classification problem, called semi-supervised Universum, that can simultaneously utilize the labeled data, unlabeled data and the Universum data to improve the classification performance. We propose a graph based method to make use of the Universum data to help depict the prior information for possible classifiers. Like conventional graph based semi-supervised methods, the graph regularization is also utilized to favor the consistency between the labels. Furthermore, since the proposed method is a graph based one, it can be easily extended to the multiclass case. The empirical experiments on the USPS and MNIST datasets are presented to show that the proposed method can obtain superior performances over conventional supervised and semi-supervised methods. Previous chapter Next chapter RelatedDetails Published:2008ISBN:978-0-89871-654-2eISBN:978-1-61197-278-8 https://doi.org/10.1137/1.9781611972788Book Series Name:ProceedingsBook Code:PR130Book Pages:1-869

We present here a data mining approach for part-of-speech (POS) tagging, an important Natural language processing (NLP) classification task. We propose a semi-supervised associative classification method for POS tagging. Existing methods for building POS taggers require extensive domain and linguistic knowledge and resources. Our method uses a combination of a small POS tagged corpus and untagged text data as training data to build the classifier model using association rules. Our tagger works well with very little training data also. The use of semi-supervised learning provides the advantage of not requiring a large high quality tagged corpus. These properties make it especially suitable for resource poor languages. Our experiments on various resource-rich, resource-moderate and resource-poor languages show good performance without using any language specific linguistic information. We note that inclusion of such features in our method may further improve the performance. Results also show that for smaller training data sizes our tagger performs better than state-of-the-art CRF tagger using same features as our tagger.

In machine learning, whether one can build a more accurate classifier by using unlabeled data (semi-supervised learning) is an important issue. Although a number of semi-supervised methods have been proposed, their effectiveness on NLP tasks is not always clear. This paper presents a novel semi-supervised method that employs a learning paradigm which we call structural learning. The idea is to find what good classifiers are like by learning from thousands of automatically generated auxiliary classification problems on unlabeled data. By doing so, the common predictive structure shared by the multiple classification problems can be discovered, which can then be used to improve performance on the target problem. The method produces performance higher than the previous best results on CoNLL'00 syntactic chunking and CoNLL'03 named entity chunking (English and German).

SSC-EKE: Semi-supervised classification with extensive knowledge exploitation

New semi-supervised classification using a multi-modal feature joint [formula omitted]-norm based sparse representation

A practical problem in supervised deep learning for medical image segmentation is the lack of labeled data which is expensive and time-consuming to acquire. In contrast, there is a considerable amount of unlabeled data available in the clinic. To make better use of the unlabeled data and improve the generalization on limited labeled data, in this paper, a novel semi-supervised segmentation method via multi-task curriculum learning is presented. Here, curriculum learning means that when training the network, simpler knowledge is preferentially learned to assist the learning of more difficult knowledge. Concretely, our framework consists of a main segmentation task and two auxiliary tasks, i.e. the feature regression task and target detection task. The two auxiliary tasks predict some relatively simpler image-level attributes and bounding boxes as the pseudo labels for the main segmentation task, enforcing the pixel-level segmentation result to match the distribution of these pseudo labels. In addition, to solve the problem of class imbalance in the images, a bounding-box-based attention (BBA) module is embedded, enabling the segmentation network to concern more about the target region rather than the background. Furthermore, to alleviate the adverse effects caused by the possible deviation of pseudo labels, error tolerance mechanisms are also adopted in the auxiliary tasks, including inequality constraint and bounding-box amplification. Our method is validated on ACDC2017 and PROMISE12 datasets. Experimental results demonstrate that compared with the full supervision method and state-of-the-art semi-supervised methods, our method yields a much better segmentation performance on a small labeled dataset. Code is available at https://github.com/DeepMedLab/MTCL.

Wheat breeding heavily relies on the observation of various traits during the wheat growth process. Among all traits, wheat head density stands out as a particularly crucial characteristic. Despite the realization of high-throughput phenotypic data collection for wheat, the development of efficient and robust models for extracting traits from raw data remains a significant challenge. Numerous fully supervised target detection algorithms have been employed to address the wheat head detection problem. However, constrained by the exorbitant cost of dataset creation, especially the manual annotation cost, fully supervised target detection algorithms struggle to unleash their full potential. Semi-supervised training methods can leverage unlabeled data to enhance model performance, addressing the issue of insufficient labeled data. This paper introduces a one-stage anchor-based semi-supervised wheat head detector, named “Wheat Teacher”, which combines two semi-supervised methods, pseudo-labeling, and consistency regularization. Furthermore, two novel dynamic threshold components, Pseudo-label Dynamic Allocator and Loss Dynamic Threshold, are designed specifically for wheat head detection scenarios to allocate pseudo-labels and filter losses. We conducted detailed experiments on the largest wheat head public dataset, GWHD2021. Compared with various types of detectors, Wheat Teacher achieved a mAP0.5 of 92.8% with only 20% labeled data. This result surpassed the test outcomes of two fully supervised object detection models trained with 100% labeled data, and the difference with the other two fully supervised models trained with 100% labeled data was within 1%. Moreover, Wheat Teacher exhibits improvements of 2.1%, 3.6%, 5.1%, 37.7%, and 25.8% in mAP0.5 under different labeled data usage ratios of 20%, 10%, 5%, 2%, and 1%, respectively, validating the effectiveness of our semi-supervised approach. These experiments demonstrate the significant potential of Wheat Teacher in wheat head detection.

The privileged information that is available only for the training examples and not available for test examples, is a new concept proposed by Vapnik and Vashist (Neural Netw 22(5–6):544–557, 2009). With the help of the privileged information, learning using privileged information (LUPI) (Neural Netw 22(5–6):544–557, 2009) can significantly accelerate the speed of learning. However, LUPI is a standard supervised learning method. In fact, in many real-world problems, there are also a lot of unlabeled data. This drives us to solve problems under a semi-supervised learning framework. In this paper, we propose a semi-supervised learning using privileged information (called Semi-LUPI), which can exploit both the distribution information in unlabeled data and privileged information to improve the efficiency of the learning. Furthermore, we also compare the relative importance of both types of information for the learning model. All experiments verify the effectiveness of the proposed method, and simultaneously show that Semi-LUPI can obtain superior performances over traditional supervised and semi-supervised methods.