Adaptive Maximum Entropy Graph-Guided Fast Locality Discriminant Analysis.

Classical Linear Discriminant Analysis (LDA) is usually suffers from the small sample size (SSS) problem when dealing with the high dimensional face data. Many methods have been proposed for solving this problem such as Fisherface and Null Space LDA (N-LDA), but these methods are overfitted to the training set and inevitably lose some useful discriminative information in many cases. To effectively utilize nearly all useful discriminative information, a not completely random sampling framework for the integration of multiple features is developed. However, this method has the following main disadvantage: By directly employing feature extraction, the newly constructed variables may contain lots of information originated from those redundant features in the original space. So, in this paper, we introduce a new random sampling LDA by incorporating feature selection for face recognition, that is, some redundant features are removed using the given feature selection methods at first, and then PCA is employed, finally we use random sampling to generate multiple feature subsets. Along this, corresponding weak LDA classifiers are naturally generated and an integrated classifier is developed using a fusion rule. Experiments on 4 face datasets (AR, ORL, Yale, YaleB) show the effectiveness of our algorithm.

Linear discriminant analysis (LDA), as an effective feature extraction method, has been widely applied in high-dimensional data analysis. However, its discriminative performance is still severely limited by the following factors. First, the restriction on the total number of features available from LDA has seriously limited its application to problems where the feature dimension is much larger than the number of classes. Second, LDA cannot deal with data containing multiple clusters (or subclasses) within a class because it cannot correctly depict the local structure of the data. To alleviate this issue, we propose a locality-preserving multiprojection discriminant analysis (LPMDA) model to extract more discriminative features preserving local structure. Specifically, LPMDA rephrases the objective function of LDA as a convex discriminant analysis framework from the perspective of metric learning, allowing for extracting more features than the number of classes. Furthermore, an auto-optimized graph technique is also integrated into the discriminant analysis framework to explore the local structure of the data. An efficient iterative optimization algorithm is presented to solve LPMDA. Extensive experiments on several benchmark datasets confirm the effectiveness of the proposed method.

Local structured feature learning with dynamic maximum entropy graph

A major and demanding issue in the telecommunications industry is the prediction of churn customers. Churn describes the customer who is attrite from one Telecom service provider to competitors searching for better services offers. Companies from the Telco sector frequently have customer relationship management offices it is the main objective in how to win back defecting clients because preserve long-term customers can be much more beneficial to a company than gain newly recruited customers. Researchers and practitioners are paying great attention and investing more in developing a robust customer churn prediction model, especially in the telecommunication business by proposed numerous machine learning approaches. Many approaches of Classification are established, but the most effective in recent times is a tree-based method. The main contribution of this research is to predict churners/non-churners in the Telecom sector based on project pursuit Random Forest (PPForest) that uses discriminant feature analysis as a novelty extension of the conventional Random Forest approach for learning oblique Project Pursuit tree (PPtree). The proposed methodology leverages the advantage of two discriminant analysis methods to calculate the project index used in the construction of PPtree. The first method used Support Vector Machines (SVM) as a classifier in the construction of PPForest to differentiate between churners and non-churners customers. The second method is a Linear Discriminant Analysis (LDA) to achieve linear splitting of variables node during oblique PPtree construction to produce individual classifiers that are robust and more diverse than classical Random Forest. It found that the proposed methods enjoy the best performance measurements e.g. Accuracy, hit rate, ROC curve, Gini coefficient, Kolmogorov-Smirnov statistic and lift coefficient, H-measure, AUC. Moreover, PPForest based on direct applied of LDA on the raw data delivers an effective evaluator for the customer churn prediction model.

Dimension reduction of multimodal data by auto-weighted local discriminant analysis

The performance of supervised learning-based seganalysis depends on the choice of both classifier and features which represent the image. Features extracted from images may contain irrelevant and redundant features which makes them inefficient for machine learning. Relevant features not only decrease the processing time to train a classifier but also provide better generalisation. Linear discriminant classifier which is commonly used for classification may not be able to classify in better way non-linearly separable data. Recently, exponential discriminant analysis, a variant of linear discriminant analysis (LDA), is proposed which transforms the scatter matrices to a new space by distance diffusion mapping. This provides exponential discriminant analysis (EDA) much more discriminant power to classify non-linearly separable data and helps in improving classification accuracy in comparison to LDA. In this paper, the performance of EDA in conjunction with feature selection methods has been investigated. For feature selection, Kullback divergence, Chernoff distance measures and linear regression measures are used to determine relevant features from higher-order statistics of images. The performance is evaluated in terms classification error and computation time. Experimental results show that exponential discriminate analysis in conjunction with linear regression significantly performs better in terms of both classification error and compilation time of training classifier. Defence Science Journal, 2012, 62(1), pp.19 -24 , DOI:http://dx.doi.org/10.14429/dsj. 62.1437

Comparison of the performance of multiclass classifiers in chemical data: Addressing the problem of overfitting with the permutation test

To retrospectively develop and evaluate computer-aided diagnosis (CAD) models that include both mammographic and sonographic descriptors. Institutional review board approval was obtained for this HIPAA-compliant study. A waiver of informed consent was obtained. Mammographic and sonographic examinations were performed in 737 patients (age range, 17-87 years), which yielded 803 breast mass lesions (296 malignant, 507 benign). Radiologist-interpreted features from mammograms and sonograms were used as input features for linear discriminant analysis (LDA) and artificial neural network (ANN) models to differentiate benign from malignant lesions. An LDA with all the features was compared with an LDA with only stepwise-selected features. Classification performances were quantified by using receiver operating characteristic (ROC) analysis and were evaluated in a train, validate, and retest scheme. On the retest set, both LDAs were compared with radiologist assessment score of malignancy. Both the LDA and ANN achieved high classification performance with cross validation (area under the ROC curve [A(z)] = 0.92 +/- 0.01 [standard deviation] and (0.90)A(z) = 0.54 +/- 0.08 for LDA, A(z) = 0.92 +/- 0.01 and (0.90)A(z) = 0.55 +/- 0.08 for ANN). Results of both models generalized well to the retest set, with no significant performance differences between the validate and retest sets (P > .1). On the retest set, there were no significant performance differences between LDA with all features and LDA with only the stepwise-selected features (P > .3) and between either LDA and radiologist assessment score (P > .2). Results showed that combining mammographic and sonographic descriptors in a CAD model can result in high classification and generalization performance. On the retest set, LDA performance matched radiologist classification performance.

Research was conducted to investigate the use of optical, air-born acoustic and plasma charge signals for detecting defects in laser welds. Full penetration, partial penetration and gapped-full penetration butt-joint welds were made in 1.4mm cold-rolled steel sheet. A 3 kW CO2 laser was used to produce these three classes of welds while laser power, travel speed and joint fitup were varied to produce the various conditions. Visible light, audible sound and electrical charge near the plume were measured simultaneously in real-time using a silicon photodiode, a condensor microphone and a plasma charge sensing gas nozzle, respectively. The spectra of the sensor signals were analyzed by a statistical technique known as Linear Discriminant Analysis (LDA). Stepwise LDA was used to identify the most important frequency bands and canonical discriminant analysis was employed to develop discriminant functions. Reliability analysis of the training samples revealed that this LDA method classified laser welds with reliability ranging from 85% to 96%, depending on the welds selected for inclusion in the training set. When the linear discriminant function was validated using new (untrained) samples, it distinguished the three different classes of laser welds with approximately 87% accuracy. Among the three signals, sound produced better separation between the classes of welds, although a discriminator based on all three signals combined and one that used only plasma charge and acoustic signals were of almost equal reliability.Research was conducted to investigate the use of optical, air-born acoustic and plasma charge signals for detecting defects in laser welds. Full penetration, partial penetration and gapped-full penetration butt-joint welds were made in 1.4mm cold-rolled steel sheet. A 3 kW CO2 laser was used to produce these three classes of welds while laser power, travel speed and joint fitup were varied to produce the various conditions. Visible light, audible sound and electrical charge near the plume were measured simultaneously in real-time using a silicon photodiode, a condensor microphone and a plasma charge sensing gas nozzle, respectively. The spectra of the sensor signals were analyzed by a statistical technique known as Linear Discriminant Analysis (LDA). Stepwise LDA was used to identify the most important frequency bands and canonical discriminant analysis was employed to develop discriminant functions. Reliability analysis of the training samples revealed that this LDA method classified laser welds with rel...

Linear Discriminant Analysis (LDA) and Multilinear Principal Component Analysis (MPCA) are leading subspace methods for achieving dimension reduction based on supervised learning. Both LDA and MPCA use class labels of data samples to calculate subspaces onto which these samples are projected. Furthermore, both methods have been successfully applied to face recognition. Although LDA and MPCA share common goals and methodologies, in previous research they have been applied separately and independently. In this paper, we propose an extension of LDA to multiple factor frameworks. Our proposed method, Multifactor Discriminant Analysis, aims to obtain multilinear projections that maximize the between-class scatter while minimizing the withinclass scatter, which is the same core fundamental objective of LDA. Moreover, Multifactor Discriminant Analysis (MDA), like MPCA, uses multifactor analysis and calculates subject parameters that represent the characteristics of subjects and are invariant to other changes, such as viewpoints or lighting conditions. In this way, our proposed MDA combines the best virtues of both LDA and MPCA for face recognition.

Analytical capabilities of chemiresistive microsensor arrays in a simulated Martian atmosphere

Orthogonal Linear Discriminant Analysis (OLDA) has been widely studied for dimensionality reduction, as the orthogonality constraints on its projection matrix enable more effective elimination of redundant features compared to conventional Linear Discriminant Analysis (LDA). However, most existing methods for solving OLDA rely on iterative optimization to sequentially construct orthogonal components, incurring massive repeated high-cost matrix operations and thus leading to substantial computational inefficiency. To address this limitation, we propose QR decomposition-based Fast OLDA (QR-FOLDA), a method built upon a theoretical result (Theorem 1) established in this work: the optimal solutions of LDA remain valid under any full-rank linear transformation. By leveraging this property, QR-FOLDA applies QR decomposition directly to the optimal LDA solution, thereby enforcing orthogonality while avoiding the repeated high-cost matrix operations typically involved in iterative optimization procedures. Experimental evaluations conducted on nine real-world datasets across various domains show that QR-FOLDA not only achieves substantial improvements in computational efficiency compared to existing OLDA methods but also delivers superior classification performance. These findings position QR-FOLDA as a theoretically sound and practically efficient solution for orthogonal discriminant analysis.

Qinghai-Tibet Plateau (QTP) rape honey, recognized as a Protected Geographical Indication (PGI) product in China, has faced significant challenges due to fraudulent mislabeling of its origins in the market. To ensure the authenticity of PGI honey products and uphold market integrity, it is crucial to develop a rapid, precise, and efficient geographical traceability technology. A total of 208 honey samples were collected from QTP (n = 71) and 5 provinces in the southern region (SR, n = 137) of China. Stable isotope ratios (δ13C, δ15N, δ2H, and δ18O) of bulk honey, endogenous proteins, and saccharides (glucose, fructose, and sucrose) were measured. One-way analysis of variance (ANOVA) was employed to analyze regional differences among the variables. Partial least squares discriminant analysis (PLS-DA) and linear discriminant analysis (LDA) models were constructed based on stable isotopic data to discriminate honey sample origins. ANOVA indicated the geospatial differences (P < 0.05) in δ2H and δ18O of bulk honey, as well as all four ratios of honey protein, are significant between QTP and SR. LDA exhibited superior discrimination performance, with leave-one-out cross-validation accuracies of 87.3% for QTP, 89.1% for SR. An integrated strategy combining stable isotope ratios analysis with multivariate modeling provides an accurate and effective verification method for geographical origin traceability of high-value honey from QTP. This approach provides a reliable tool to address the issue of fraudulent mislabeling of PGI rape honey. Stable isotopic signatures of Qinghai-Tibet Plateau rape honey were discussed. Bulk and component-specific isotopic ratios were informative geospatial indicators. Machine learning algorithms significantly enhanced honey origin discrimination. LDA accuracy for Qinghai-Tibet Plateau honey samples reached up to 87.3%. This strategy was developed to combat origin mislabeling and ensure food integrity.

The growth of data today poses a challenge in management and inference. While feature extraction methods are capable of reducing the size of the data for inference, they do not help in minimizing the cost of data storage. On the other hand, feature selection helps to remove the redundant features and therefore is helpful not only in inference but also in reducing management costs. This work presents a novel parallel feature selection approach for classification, namely Parallel Feature Selection using Trace criterion (PFST), which scales up to very large datasets. Our method uses trace criterion, a measure of class separability used in Fisher's Discriminant Analysis, to evaluate feature usefulness. We analyzed the criterion's desirable properties theoretically. Based on the criterion, PFST rapidly finds important features out of a set of features for big datasets by first making a forward selection with early removal of seemingly redundant features parallelly. After the most important features are included in the model, we check back their contribution for possible interaction that may improve the fit. Lastly, we make a backward selection to check back possible redundant added by the forward steps. We evaluate our methods via various experiments using Linear Discriminant Analysis as the classifier on selected features. The experiments show that our method can produce a small set of features in a fraction of the amount of time by the other methods under comparison. In addition, the classifier trained on the features selected by PFST not only achieves better accuracy than the ones chosen by other approaches, but can also achieve better accuracy than the classification on all available features.

A novel Bayesian logistic discriminant model: An application to face recognition