Recognizing MNIST Handwritten Data Set Using PCA and LDA

Improved PCA + LDA Applies to Gastric Cancer Image Classification Process

We investigated three classifiers for the task of distinguishing between benign and malignant breast lesions. Classification performance was measured in terms of area under the ROC curve (AUC value). We compared linear discriminant analysis (LDA), quadratic discriminant analysis (QDA) and a Bayesian neural net (BNN) with 5 hidden units. For each lesion, 46 image features were extracted and principal component analysis (PCA) of these features was used as classifier input. For each classifier, the optimal number of principal components was determined by performing PCA within each step of a leave-one-case-out protocol for the training dataset (1125 lesions, 14% cancer prevalence) and determining which number of components maximized the AUC value. Subsequently, each classifier was trained on the training dataset and applied ‘cold turkey’ to an independent test set from a different population (341 lesions, 30% cancer prevalence). The optimal number of principal components for LDA was 24, accounting for 97% of the variance in the image features. For QDA and BNN, these numbers were 5 (70%) and 15 (93%), respectively. The LDA, QDA and BNN obtained AUC values of 0.88, 0.85, and 0.91, respectively, in the leave-one-case-out analysis. In the independent test – with AUCs of 0.88, 0.76, and 0.82 – only LDA achieved performance identical to that for the training set (lower bound of 95% non-inferiority interval -.0067), while the others performed significantly worse (p-values << 0.05). While the more complex BNN classifier outperformed the others in leave-one-case-out of a large dataset, LDA was the robust best-performer in an independent test.

Feature extraction for machine learning-based intrusion detection in IoT networks

A feature extraction and fusion algorithm was constructed by combining principal component analysis (PCA) and linear discriminant analysis (LDA) to detect a fault state of the induction motor. After yielding a feature vector with PCA and LDA from current signal that was measured by an experiment, the reference data were used to produce matching values. In a diagnostic step, two matching values that were obtained by PCA and LDA, respectively, were combined by probability model, and a faulted signal was finally diagnosed. As the proposed diagnosis algorithm brings only merits of PCA and LDA into relief, it shows excellent performance under the noisy environment. The simulation was executed under various noisy conditions in order to demonstrate the suitability of the proposed algorithm and showed more excellent performance than the case just using conventional PCA or LDA.

Remote sensing Hyperspectral Image (HSI) comprises significant information about the earth’s surface which is actually acquired by hundred of narrow and adjacent spectral bands. The intended performance of classification accuracy does not attain due to the volume of the original HSI dataset and the enormous quantity of spectral bands. As such, dimensionality reduction approaches using feature extraction and selection are typically adopted to enhance classification performance. The unsupervised Principal Component Analysis (PCA), as well as the supervised Linear Discriminant Analysis (LDA), are commonly used as linear feature extraction methods for feature subspace detection. However, due to considering the effects of global variation, both PCA and LDA fail to extract local characteristics of HSI. In this paper, we propose a segmented LDA-based (SLDA) feature extraction where we apply the LDA in a segmented way to extract better local characteristics as well as global characteristics from the HSI. Per-pixel classification using a Support Vector Machine (SVM) is applied to our proposed SLDA method, PCA, Segmented-PCA (SPCA), and LDA on the Indian Pines agricultural HSI dataset. The experimental results show that the overall classification performance of SLDA (90.60%) remarkably outperforms all the other investigated methods: PCA (85.55%), SPCA (86.96%), LDA (86.45%), and the complete original dataset without employing any feature reduction method (83.10%). The proposed SLDA also requires the least amount of space complexity in different implementation scenarios.

Problem statement: In facial biometrics, face features are used as the required human traits for automatic recognition. Feature extracted from face images are significant for face biometrics system performance. Approach: In this thesis, a framework of facial biometric was designed based on two subspace methods i.e., Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). First, PCA is used for dimension reduction, where original face images are projected into lower-dimensional face representations. Second, LDA was proposed to provide a solution of better discriminant. Both PCA and LDA features were presented to Euclidean distance measurement which is conveniently used as a benchmark. The algorithms were evaluated in face identification and verification using a standard face database-AT and T and a locally collected database-CBE. Each database consists of 400 images and 320 images respectively. Results: LDA-based methods outperform PCA for both face identification and verification. For face identification, PCA achieves accuracy of 91.9% (AT and T) and 76.7% (CBE) while LDA 94.2% (AT and T) and 83.1% (CBE). For face verification, PCA achieves Equal Error Rate (EER) of 1.15% (AT and T), 7.3% (CBE) while LDA 0.78% (AT and T) and 5.81% (CBE). Conclusion/Recommendations: This study had proved that, when given sufficient training samples, LDA is able to provide better discriminant ability in feature extraction for face biometrics.

Machine learning-enhanced Attenuated Total Reflectance Fourier Transform Infrared (ATR-FTIR) spectroscopy for discrimination of green Arabica and Robusta coffee beans

Principal component analysis (PCA) is linear method for feature extraction that is known as Karhonen Loove method. PCA was first proposed to recognize face by Turk and Pentland, and was also known as eigenface in 1991 [Turk, 1991]. However, PCA has some weaknesses. The first, it cannot capture the simplest invariance of the face image [Arif et al., 2008b] , when this information is not provided in the training data. The last, the result of feature extraction is global structure [Arif, 2008]. The PCA is very simple, has overcome curse of dimensionality problem, this method have been known and expanded by some researchers to recognize face such as Linear Discriminant Analysis (LDA)[Yambor, 2000; A.M. Martinez, 2003; J.H.P.N. Belhumeur 1998], Linear Preserving Projection that known Lapalacianfaces [Cai, 2005; Cai et al, 2006; Kokiopoulou, 2004; X. He et al., 2005], Independent Component Analysis, Kernel Principal Component Analysis [Scholkopf et al., 1998; Sch olkopf 1999], Kernel Linear Discriminant Analysis (KLDA) [Mika, 1999] and maximum feature value selection of nonlinear function based on Kernel PCA [Arif et al., 2008b]. As we know, PCA is dimensionality reduction method based on object appearance by projecting an original ndimensional (row*column) image into k eigenface where k<<n. Although PCA have been developed into some methods, but in some cases, PCA can outperform LDA, LPP and ICA when it uses small sample size.

The management and disposal of waste is a severe social issue and an essential part of ecological sustainability. As an important component of the green, low-carbon, and recycling economic system, the identification and classification of recyclable waste is the premise of its reuse and energy conservation. The main issues at hand are to improve the classification accuracy and reliability of recyclable waste and to achieve automatic classification. The methods based on physical characteristics and image-based methods are inaccurate and unreliable. The current spectroscopy methods need to process the detected samples in advance, unsuitable for automatic detection. Based on material composition properties, the Laser-Induced Breakdown Spectroscopy (LIBS) technology is here proposed to accurately and reliably identify and classify recyclable waste into six categories at the level of consumer, such as paper, plastic, glass, metal, textile, and wood. The method is also used to subclassify the same category of waste for reuse at the level of a recycling factory. We subclassified metals into iron, stainless steel, copper, and aluminum and plastics into polyvinylchloride, polyoxymethylene, acrylonitrile-butadiene-styrene, polyamide, polyethylene, and polytetrafluoroethylene. The drop-dimension methods of LIBS spectra of waste were researched to eliminate noise and redundant information by principal component analysis (PCA) and linear discriminant analysis (LDA), respectively. Their clustering effects were analyzed to choose a suitable dimension. Combining the random forest (RF), back propagation neural network (BPNN), and convolutional neural network (CNN), we established and compared five classification models, PCA + RF, PCA + BPNN, LDA + RF, LDA + BPNN, and 1D-CNN. For the classification of six categories, the accuracies of proposed classification models are all more than 96%, and LDA(5D) + RF has 100% accuracy and optimal classification performance indices. For the subclassification of metals and plastics, PCA(8D) + RF has the highest classification accuracy of 98.77% and 99.52%, respectively.

An electronic nose (E-nose) was developed to distinguish different stored eggs by using principal component analysis (PCA),linear discriminant analysis (LDA),BP neural network (BPNN) and the combination of a genetic algorithm and BP neural network (GANN). Results showed that the E-nose can distinguish eggs of different storage time under cool and room-temperature storage by LDA and PCA. LDA preceded PCA. Better prediction values were obtained by GANN than by BPNN. Relationships were established between the E-nose signal and egg quality indices by quadratic polynomial step regression (QPSR). The prediction models indicated good prediction performance with correlation coefficient between predicted and measured values greater than 0.90.

Near infrared reflectance spectroscopy was used to discriminate different species of freshwater fish samples. Samples from seven freshwater fish species of the family Cyprinidae (black carp ( Mylopharyngodon piceus), grass carp ( Ctenopharyngodon idellus), silver carp ( Hypophthalmichthys molitrix), bighead carp ( Aristichthys nobilis), common carp ( Cyprinus carpio), crucian ( Carassius auratus), and bream ( Parabramis pekinensis)) were scanned by near infrared reflectance spectroscopy from 1000 nm to 1799 nm. Linear discriminant analysis models were built for the classification of species. We inspected the effect of partial least square, principal component analysis, competitive adaptive reweighted sampling, and fast Fourier transform on linear discriminant analysis. The results showed that the dimension reduction methods worked very well for this example. Linear discriminant analysis models which were combined with principal component analysis and fast Fourier transform could classify accurately all the samples under multiplicative scatter correction pre-processing. According to the loadings in principal component analysis, spectra wavelengths 1000, 1001, 1154, 1208, 1284, 1288, 1497, 1665, and 1770 nm were selected as effective wavelengths in linear discriminant analysis. The discriminant analysis was simplified by using effective wavelengths as independent variables in a linear discriminant analysis model. This study indicated that linear discriminant analysis combined with near infrared reflectance spectroscopy could be an effective strategy for the classification of freshwater fish species.

Despite the fact that technology is progressing, people continue to consume deadly wild mushrooms due to their inability in classifying the different mushroom categories. Currently, no specific traits are defined to accurately predict the edibility of mushrooms based on their attributes. To overcome this challenge, Machine Learning [ML] can be used for identifying the poisonous mushroom based on the feature appearance. By considering the above, the Mushroom dataset extracted from UCI data warehouse are used for predicting the mushroom edibility level. The division of mushroom edible classes are achieved in four different ways. Firstly, the dataset is preprocessed with feature scaling and missing values. Secondly, raw data set is fitted to all the classifier with and without the presence of feature scaling. Thirdly, raw data is applied with the principal component analysis with 8, 10 and 12 components and PCA reduced dataset is fitted to all the classifier with and without the presence of feature scaling. Fourth, the raw data is applied with the Linear discriminant analysis and LDA reduced dataset is fitted to all the classifier with and without the presence of feature scaling. Fifth, raw data is applied with the Factor analysis with 8, 10 and 12 components and FA reduced dataset is fitted to all the classifier with and without the presence of feature scaling. Sixth, the performance of raw data set, PCA reduced data set, LDA reduced dataset and FA reduced dataset are compared by analyzing the performance metrics like Precision, Recall, Accuracy and F-score. The implementation is done by using python language under Spyder platform with Anaconda Navigator. Experimental results show that, the kernel SVM, KNN and Adaboost classifier for the FA reduced dataset tends to retain the accuracy with 100% before and after feature scaling. The KNN classifier with LDA reduced dataset tends to retain the 96% accuracy before and after feature scaling. Kernel SVM, KNN and Adaboost classifier for the FA reduced dataset tends to retain the accuracy with 99% before and after feature scaling. From the above analysis, KNN classifier ismore efficient based on its accuracy with all PCA, LDA and FA reduced dataset.KeywordsMachine learningClassificationPrecisionAccuracyPCALDA and factor analysis

Towards robustness and sensitivity of rapid Baijiu (Chinese liquor) discrimination using Raman spectroscopy and chemometrics: Dimension reduction, machine learning, and auxiliary sample

Drug resistance plays a vital role in both cancer treatment and prognosis. Especially, early insights into such drug‐induced resistance in acute myeloid leukemia (AML) can help to improve treatment plans, reduce costs, and bring overall positive outcomes for patients. Raman spectroscopy provides precise biomolecular information and can provide all these necessities effectively. In this study, we employed machine learning (ML) discrimination of Raman micro‐spectroscopic data of myelocytic leukemia cell line HL‐60 from its drug‐resistant counterpart HL‐60/MX2. Principal component analysis (PCA), linear discriminant analysis (LDA), and logistic regression (LR) methods were evaluated for their ability to identify and discriminate drug resistance in AML cells. Our study demonstrates the power of ML to classify drug‐induced resistance in AML cells utilizing subtle variations in biomolecular information contained in molecular spectroscopic data by obtaining 94.11% and 97.05% classification accuracies by LDA and LR models, respectively. We also showed that the ML methods are discernable. Our findings depict the importance of automation and its optimal usage in cancer study and diagnosis. The results of our study are expected to take ML‐assisted Raman spectroscopy one step closer to making it a generalized tool in medical diagnosis in the future.

To evaluate the usefulness of differentially expressed proteins from colorectal cancer (CRC) tissues for differentiating cancer and normal tissues. A Proteomic approach was used to identify the differentially expressed proteins between CRC and normal tissues. The proteins were extracted using Tris buffer and thiourea lysis buffer (TLB) for extraction of aqueous soluble and membrane-associated proteins, respectively. Chemometrics, namely principal component analysis (PCA) and linear discriminant analysis (LDA), were used to assess the usefulness of these proteins for identifying the cancerous state of tissues. Differentially expressed proteins identified were 37 aqueous soluble proteins in Tris extracts and 24 membrane-associated proteins in TLB extracts. Based on the protein spots intensity on 2D-gel images, PCA by applying an eigenvalue > 1 was successfully used to reduce the number of principal components (PCs) into 12 and seven PCs for Tris and TLB extracts, respectively, and subsequently six PCs, respectively from both the extracts were used for LDA. The LDA classification for Tris extract showed 82.7% of original samples were correctly classified, whereas 82.7% were correctly classified for the cross-validated samples. The LDA for TLB extract showed that 78.8% of original samples and 71.2% of the cross-validated samples were correctly classified. The classification of CRC tissues by PCA and LDA provided a promising distinction between normal and cancer types. These methods can possibly be used for identification of potential biomarkers among the differentially expressed proteins identified.