Structured Semi-supervised Discriminant Analysis

An improved manifold learning method, called enhanced semi-supervised local fisher discriminant analysis (ESELF), for gene expression data classification is proposed. Motivated by the fact that semi-supervised and parameter-free are two desirable and promising characteristics for dimension reduction, a new difference-based optimization objective function with unlabeled samples has been designed. The proposed method preserves the global structure of unlabeled samples in addition to separating labeled samples in different classes from each other. The semi-supervised method has an analytic form of the globally optimal solution and it can be computed based on eigen decompositions. The experimental results and comparisons on synthetic data and two DNA micro array datasets demonstrate the effectiveness of the proposed method.

The traditional classification method based on supervised learning classifies remote sensing (RS) images by using sufficient labelled samples. However, the number of labelled samples is limited due to the expensive and time-consuming collection. To effectively utilize the information of unlabelled samples in the learning process, this paper proposes a novel semi-supervised classification method based on class certainty of samples (CCS). First, the class certainty of unlabelled samples obtained based on multi-class SVM is smoothed for robustness. Then, a new semi-supervised linear discriminant analysis (LDA) is presented based on class certainty, which improves the separability of samples in the projection subspace. Finally, the nearest neighbor classifier is adopted to classify the images. The experimental results demonstrate that the proposed method can effectively exploit the information of unlabelled samples and greatly improve the classification effect compared with other state-of-the-art approaches.

This letter presents a novel semi-supervised method based on hypergraph learning for polarimetric synthetic aperture radar (PolSAR) image classification. Compared with the classic support vector machine, simple-graph learning, k-nearest neighbour (k-NN) and semi-supervised discriminant analysis (SDA) classifiers, the proposed method achieves better performance with fewer labelled points for PolSAR imagery. A hyperspectral image is used for comparison with use of PolSAR imagery, and the proposed method is found to be inferior to k-NN and SDA for the hyperspectral image. The performance of our method is evaluated in single, dual and full-polarization cases, respectively. The results demonstrate that the performance of our method in the full-polarization case is superior to that in either single or dual-polarization case.

Tracking is recently formulated as a problem of discriminating the object from its nearby background, where the classifier is updated by new samples successively arriving during tracking. Depending on whether labeling the samples or not, the tracker can be designed in a supervised or semi-supervised manner. This paper proposes a novel semi-supervised algorithm for tracking by combining Semi-supervised Discriminant Analysis (SDA) with an online boosting framework. Using the local geometric structure information from the samples, the SDA-based weak classifier is made more robust to outliers. Meanwhile, we design an incremental updating mechanism for SDA so that it can adapt to appearance changes. We further propose an Extended SDA (ESDA) algorithm, which gives better discrimination ability. Results on several challenging video sequences demonstrate the effectiveness of the method.

A novel Bayesian logistic discriminant model: An application to face recognition

This paper investigates the use of semi-supervised learning (SSL) for predicting Alzheimers Disease (AD) conversion in Mild Cognitive Impairment (MCI) patients based on Magnetic Resonance Imaging (MRI). SSL methods differ from standard supervised learning methods in that they make use of unlabeled data - in this case data from MCI subjects whose final diagnosis is not yet known. We compare two widely used semi-supervised methods (low density separation (LDS) and semi-supervised discriminant analysis (SDA)) to the corresponding supervised methods using real and synthetic MRI data of MCI subjects. With simulated data, using SSL instead of supervised learning led to higher classification performance in certain cases, however, the applicability of semi-supervised methods depended strongly on the data distributions. With real MRI data, the SSL methods achieved significantly better classification performances over supervised methods. Moreover, even using a small number of unlabeled samples improved the AD conversion predictions.

In the paper, we present an improved approach based on Semisupervised Discriminant Analysis (SDA), called semi-supervised local discriminant embedding (SLDE), for reducing the dimensionality of the feature space. We take the manifold structure into account and try to learn a subspace in which the Euclidean distances can better reflect class structure of the images. The weight matrix and the scatter matrices in SDA are improved to make efficient use of both labeled and unlabeled images. After being embedded into a low-dimensional subspace, the similar images maintain their intrinsic neighbor relations, whereas the dissimilarity neighboring images no longer stick to one another. Experiments have been carried out to validate our approach.

Semisupervised Discriminant Analysis (SDA) aims at dimensionality reduction with both limited labeled data and copious unlabeled data, but it may fail to discover the intrinsic geometry structure when the data manifold is highly nonlinear. The kernel trick is widely used to map the original nonlinearly separable problem to an intrinsically larger dimensionality space where the classes are linearly separable. Inspired by low-rank representation (LLR), we proposed a novel kernel SDA method called low-rank kernel-based SDA (LRKSDA) algorithm where the LRR is used as the kernel representation. Since LRR can capture the global data structures and get the lowest rank representation in a parameter-free way, the low-rank kernel method is extremely effective and robust for kinds of data. Extensive experiments on public databases show that the proposed LRKSDA dimensionality reduction algorithm can achieve better performance than other related kernel SDA methods.

Research was conducted to investigate the use of optical, air-born acoustic and plasma charge signals for detecting defects in laser welds. Full penetration, partial penetration and gapped-full penetration butt-joint welds were made in 1.4mm cold-rolled steel sheet. A 3 kW CO2 laser was used to produce these three classes of welds while laser power, travel speed and joint fitup were varied to produce the various conditions. Visible light, audible sound and electrical charge near the plume were measured simultaneously in real-time using a silicon photodiode, a condensor microphone and a plasma charge sensing gas nozzle, respectively. The spectra of the sensor signals were analyzed by a statistical technique known as Linear Discriminant Analysis (LDA). Stepwise LDA was used to identify the most important frequency bands and canonical discriminant analysis was employed to develop discriminant functions. Reliability analysis of the training samples revealed that this LDA method classified laser welds with reliability ranging from 85% to 96%, depending on the welds selected for inclusion in the training set. When the linear discriminant function was validated using new (untrained) samples, it distinguished the three different classes of laser welds with approximately 87% accuracy. Among the three signals, sound produced better separation between the classes of welds, although a discriminator based on all three signals combined and one that used only plasma charge and acoustic signals were of almost equal reliability.Research was conducted to investigate the use of optical, air-born acoustic and plasma charge signals for detecting defects in laser welds. Full penetration, partial penetration and gapped-full penetration butt-joint welds were made in 1.4mm cold-rolled steel sheet. A 3 kW CO2 laser was used to produce these three classes of welds while laser power, travel speed and joint fitup were varied to produce the various conditions. Visible light, audible sound and electrical charge near the plume were measured simultaneously in real-time using a silicon photodiode, a condensor microphone and a plasma charge sensing gas nozzle, respectively. The spectra of the sensor signals were analyzed by a statistical technique known as Linear Discriminant Analysis (LDA). Stepwise LDA was used to identify the most important frequency bands and canonical discriminant analysis was employed to develop discriminant functions. Reliability analysis of the training samples revealed that this LDA method classified laser welds with rel...

Recently, deep learning techniques achieve remarkable classification performance on histopathology images. How-ever, they usually require a large amount of labeled training images to obtain satisfactory accuracy, and manual labeling is labor expensive and time consuming. To address this issue, in this paper, we propose a novel semi-supervised deep learning framework, namely semi-supervised deep linear discriminant analysis, by taking advantage of the deep neural network (DNN) and the graph to simultaneously exploit the semantic information of labeled and unlabeled images for classification. Specifically, we first replace the loss function of the DNN with the objective function of linear discriminant analysis to produce features minimizing the intra-class distance yet maximizing the inter-class distance, in order to construct a robust and effective graph Laplacian. Afterwards, we design a new objective function via employing the graph constructed by features of labeled and un-labeled images, and then adopt this objective as the loss function of the DNN to produce features for classification. Experiments on skeletal muscle and lung cancer images demonstrate the proposed framework outperforms several recent state of the arts.

Fisher discriminant analysis (FDA) is a classic supervised dimensionality reduction method in statistical pattern recognition. FDA can maximize the scatter between different classes, while minimizing the scatter within each class. As it only utilizes the labeled data and ignores the unlabeled data in the analysis process of FDA, it cannot be used to solve the unsupervised learning problems. Its performance is also very poor in dealing with semi-supervised learning problems in some cases. Recently, several semi-supervised learning methods as an extension of FDA have proposed. Most of these methods solve the semi-supervised problem by using a tradeoff parameter that evaluates the ratio of the supervised and unsupervised methods. In this paper, we propose a general semi-supervised dimensionality learning idea for the partially labeled data, namely the reconstruction probability class of labeled and unlabeled data. Based on the probability class optimizes Fisher criterion function, we propose a novel Semi-Supervised Local Fisher Discriminant Analysis (S2LFDA) method. Experimental results on real-world datasets demonstrate its effectiveness compared to the existing similar correlation methods.

A computer vision system for automatic recognition and classification of five varieties of plant leaves under controlled laboratory imaging conditions, comprising: 1–Cydonia oblonga (quince), 2–Eucalyptus camaldulensis dehn (river red gum), 3–Malus pumila (apple), 4–Pistacia atlantica (mt. Atlas mastic tree) and 5–Prunus armeniaca (apricot), is proposed. 516 tree leaves images were taken and 285 features computed from each object including shape features, color features, texture features based on the gray level co-occurrence matrix, texture descriptors based on histogram and moment invariants. Seven discriminant features were selected and input for classification purposes using three classifiers: hybrid artificial neural network–ant bee colony (ANN–ABC), hybrid artificial neural network–biogeography based optimization (ANN–BBO) and Fisher linear discriminant analysis (LDA). Mean correct classification rates (CCR), resulted in 94.04%, 89.23%, and 93.99%, for hybrid ANN–ABC; hybrid ANN–BBO; and LDA classifiers, respectively. Best classifier mean area under curve (AUC), mean sensitivity, and mean specificity, were computed for the five tree varieties under study, resulting in: 1–Cydonia oblonga (quince) 0.991 (ANN–ABC), 95.89% (ANN–ABC), 95.91% (ANN–ABC); 2–Eucalyptus camaldulensis dehn (river red gum) 1.00 (LDA), 100% (LDA), 100% (LDA); 3–Malus pumila (apple) 0.996 (LDA), 96.63% (LDA), 94.99% (LDA); 4–Pistacia atlantica (mt. Atlas mastic tree) 0.979 (LDA), 91.71% (LDA), 82.57% (LDA); and 5–Prunus armeniaca (apricot) 0.994 (LDA), 88.67% (LDA), 94.65% (LDA), respectively.

In this paper, a robust face recognition algorithm is proposed, which is based on the elastic graph matching (EGM) and discriminative feature analysis algorithm. We introduce a cost function for the EGM taking account of variations in face pose and facial expressions, and propose its optimization procedure. Our proposed cost function uses a set of Gabor-wavelet-based features, called robust jet, which are robust against the variations. The robust jet is defined in terms of discrete Fourier transform coefficients of Gabor coefficients. To cope with the difference between face poses of test face and reference faces, 2 x 2 warping matrix is incorporated in the proposed cost function. For the discriminative feature analysis, linear projection discriminant analysis and kernel-based projection discriminant analysis are introduced. These methods are motivated to solve the small-size problem of training samples. The basic idea of PDA is that a class is represented by a subspace spanned by some training samples of the class instead of using sample mean vector, that the distance from a pattern to a class is defined by using the error vector between the pattern and its projection to the subspace representing the class, and that an optimum feature selection rule is developed using the distance concept in a similar way as in the conventional linear discriminant analysis. In order to evaluate the performance of our face recognition algorithm, we carried out some experiments using the well-known FERET face database, and compared the performance with recently developed approaches. We observed that our algorithm outperformed the compared approaches.

Dimensionality reduction is a very important part in the field of face recognition. In view of the problem of the traditional dimensionality reduction methods are inconvenient to select neighbor parameter K and the “dense” characteristic of the low-rank representation coefficient matrix. We presented a method that semi-supervised discriminant analysis via weighted low-rank representation and adaptive neighbor selection (ANSWLR-SDA). First, we uses all the within-class samples to construct the within-class graph which can describe the within-class compactness, and then adaptively chooses the between-class samples to construct the between-class graph which can describe the between-class respectively. On this basis, we use a regularization term by weighted low-rank represented to maintain the global similarity structure of samples. Finally, we carry out the experiments on FERET and yale_faceb databases, and compare this method with the traditional dimensionality reduction methods and the results demonstrate that ANSWLR-SDA method is effectiveness and robust to different types of noise than other state-of-art face recognition method.

Linear Discriminant Analysis (LDA) is a well-known technique for feature extraction and dimension reduction. The performance of classical LDA however, significantly degrades on the High Dimension Low Sample Size (HDLSS) data for the ill-posed inverse problem. Existing approaches for HDLSS data classification typically assume the data in question are with Gaussian distribution and deal the HDLSS classification problem with regularization. However, these assumptions are too strict to hold in many emerging real-life applications, such as enabling personalized predictive analysis using Electronic Health Records (EHRs) data collected from an extremely limited number of patients who have been diagnosed with or without the target disease for prediction. In this paper, we revised the problem of predictive analysis of disease using personal EHR data and LDA classifier. To fill the gap, in this paper, we first studied an analytical model that understands the accuracy of LDA for classifying data with arbitrary distribution. The model gives a theoretical upper bound of LDA error rate that is controlled by two factors: (1) the statistical convergence rate of (inverse) covariance matrix estimators and (2) the divergence of the training/testing datasets to fitted distributions. To this end, we could lower the error rate by balancing the two factors for better classification performance. Hereby, we further proposed a novel LDA classifier De-Sparse that leverages De-sparsified Graphical Lasso to improve the estimation of LDA, which outperforms state-of-the-art LDA approaches developed for HDLSS data. Such advances and effectiveness are further demonstrated by both theoretical analysis and extensive experiments on EHR datasets https://www.overleaf.com/project/5d2728c718f6ff3b2bcf5991 .