Recursive Bayesian Linear Discriminant for Classification

The enhanced capabilities of the remote sensing devices lead to capture more precise and accurate spatial and spectral information about surface materials. Increased spectral resolution results in more number of spectral bands and raises the challenge of data dimensionality. This high volume data holds plenty of redundant information. This redundancy affects both the time as well as space complexity of the system. To process and analyse the hyperspectral data with less computational cost with no information loss, data dimensionality needs to be reduced. The literature shows that the traditional image processing techniques with some modifications are applied for hyperspectral dimensionality reduction, but none of the methods give specific solution. This paper evaluates the performances and limitations of the state-of-the-art dimensionality reduction techniques. The algorithms studied and evaluated are Principal Components Analysis, Independent Component Analysis, Minimum Noise Fraction, Fisher Linear Discriminant Analysis, Factor Analysis and Linear Discriminant Analysis. The experiments are performed on the Indian Pines AVIRIS & Gulbarga Subset (AVIRIS-NG) hyperspectral datasets.

In this paper, we investigate the application of Fisher’s linear discriminant analysis (FLDA) to hyperspectral remote sensing image classification. The basic idea of FLDA is to design an optimal transform so that the classes can be well separated in the low-dimensional space. The practical difficulty of applying FLDA to hyperspectral images includes the unavailability of enough training samples and unknown information for all the classes present. So the original FLDA is modified to avoid the requirements of complete class knowledge, such as the number of actual classes present. We also investigate the performance of the class of principal component analysis (PCA) techniques prior to FLDA and find that the interference and noise adjusted PCA (INAPCA) can provide the improvement in the final classification.KeywordsFisher’s Linear Discriminant AnalysisDimensionality ReductionClassificationHyperspectral Imagery

“Small Sample Size” (SSS) problem would occur while using linear discriminant analysis (LDA) algorithm with traditional Fisher criterion if the within-class scatter matrix is singular. The combination of maximum scatter difference (MSD) criterion and LDA algorithm for solve SSS problem is described. It is employed to detect three kinds of Chinese herbal medicines from different growing areas by machine olfaction. Compared with PCA or PCA + LDA algorithm, the classification result was enhanced. It works out that only a few samples of Anhui Atractylodes are classified incorrectly, however, the classification rate reaches 97.8%. DOI: http://dx.doi.org/10.11591/telkomnika.v11i2.1986

Modelling performance during repetitive precision tasks using wearable sensors: a data-driven approach

The Fisher's linear discriminant analysis (FLDA) is a successful linear feature extraction method which aims to maximize the between-class separability and to minimize the within-class variability. In order to accomplish its goal, FLDA maximizes the Fisher's linear discriminant analysis criterion given in the paper. In this paper we first address the limitations of the classical FLDA criterion and then discuss new criterion functions which were introduced to overcome those limitations. Finally, we demonstrate the capability of the new criteria both theoretically and empirically.

A derivative of Fisher's Linear Discriminant Analysis (FLDA), named Fisherapples for the recognition of apple lesions which is not sensitive to large variations in illumination is proposed in this paper. We make use of the linear projection that is orthogonal to the within-class scatter of the apple images from a high-dimensional image space to a considerably low-dimensional image space. It separates the data-cases well, projecting away variations in lighting. Our approach maximizes the ratio of between-class scatter to that of within-class scatter of apple lesions, i.e., we can get maximal between-class distances and minimal within-class distances after projection. This implies that the gap between the classes becomes bigger and ensures optimal separability in the new space. Besides, we take advantage of Principal Component Analysis (PCA) to project the set of apple images to a lower dimensional space in order to overcome the complication of the singular within-class scatter matrix. After that, the resulting within-class scatter becomes nonsingular and subsequently we can use standard FLDA to reduce the dimension further. Consequently, it is effortless for the computer to calculate the result. Experimental results demonstrate that Fisherapples performs better in apple lesion recognition than PCA.

The nondestructive characterization of citrus varieties (Egyptian sweet orange, Lane Late navel orange, Australian orange, and Blood orange) was developed based on near-infrared diffuse reflectance spectroscopy (NIRDRS) together with principal component analysis (PCA) and Fisher linear discriminant analysis (FLDA). An experiment for the penetration of NIRDRS into the peel was designed and the effects of different spectral acquisition points were investigated. Pretreatments were used to eliminate the spectral interferences. As an unsupervised pattern recognition method, PCA was used to establish the characterization models. Furthermore, supervised pattern recognition based on PCA and FLDA was employed to enhance the accuracy. The results of the penetration experiments show that near-infrared light enters the citrus peel and is able to characterize the internal composition. Even with the optimized spectral pretreatment, accurate characterization of citrus varieties was not achieved by PCA. However, the accurate characterization of citrus varieties was provided by PCA-FLDA. The accuracies of four spectral acquisition points are 95%, while the characterization accuracies of six spectral acquisition points are 100% combined with optimized spectral pretreatment. Therefore, NIRDRS with PCA-FLDA is suitable for the rapid and nondestructive characterization of citrus varieties.

<p class="Abstract">Internet of things (IoT) systems have experienced significant growth in data traffic, resulting in security and real-time processing issues. Intrusion detection systems (IDS) are currently an indispensable tool for self-protection against various attacks. However, IoT systems face serious challenges due to the functional diversity of attacks, resulting in detection methods with machine learning (ML) and limited static models generated by the linear discriminant analysis (LDA) algorithm. The process entails adjusting the model parameters in real time as new data arrives. This paper proposes a new method of an IDS based on the LDA algorithm with the incremental model. The model framework is trained and tested on the IoT intrusion dataset (UNSW-NB15) using the streaming linear discriminant analysis (SLDA) ML algorithm. Our approach increased model accuracy after each training, resulting in continuous model improvement. The comparison reveals that our dynamic model becomes more accurate after each batch and can detect new types of attacks.</p>

Introduction: Glioblastoma and anaplastic astrocytoma (ANA) are two of the most common primary brain tumors in adults. The differential diagnosis is important for treatment recommendations and prognosis assessment. This study aimed to assess the discriminative ability of texture analysis using machine learning to distinguish glioblastoma from ANA.Methods: A total of 123 patients with glioblastoma (n = 76) or ANA (n = 47) were enrolled in this study. Texture features were extracted from contrast-enhanced Magnetic Resonance (MR) images using LifeX package. Three independent feature-selection methods were performed to select the most discriminating parameters:Distance Correlation, least absolute shrinkage and selection operator (LASSO), and gradient correlation decision tree (GBDT). These selected features (datasets) were then randomly split into the training and the validation group at the ratio of 4:1 and were fed into linear discriminant analysis (LDA), respectively, and independently. The standard sensitivity, specificity, the areas under receiver operating characteristic curve (AUC) and accuracy were calculated for both training and validation group.Results: All three models (Distance Correlation + LDA, LASSO + LDA and GBDT + LDA) showed promising ability to discriminate glioblastoma from ANA, with AUCs ≥0.95 for both the training and the validation group using LDA algorithm and no overfitting was observed. LASSO + LDA showed the best discriminative ability in horizontal comparison among three models.Conclusion: Our study shows that MRI texture analysis using LDA algorithm had promising ability to discriminate glioblastoma from ANA. Multi-center studies with greater number of patients are warranted in future studies to confirm the preliminary result.

A computer vision system for automatic recognition and classification of five varieties of plant leaves under controlled laboratory imaging conditions, comprising: 1–Cydonia oblonga (quince), 2–Eucalyptus camaldulensis dehn (river red gum), 3–Malus pumila (apple), 4–Pistacia atlantica (mt. Atlas mastic tree) and 5–Prunus armeniaca (apricot), is proposed. 516 tree leaves images were taken and 285 features computed from each object including shape features, color features, texture features based on the gray level co-occurrence matrix, texture descriptors based on histogram and moment invariants. Seven discriminant features were selected and input for classification purposes using three classifiers: hybrid artificial neural network–ant bee colony (ANN–ABC), hybrid artificial neural network–biogeography based optimization (ANN–BBO) and Fisher linear discriminant analysis (LDA). Mean correct classification rates (CCR), resulted in 94.04%, 89.23%, and 93.99%, for hybrid ANN–ABC; hybrid ANN–BBO; and LDA classifiers, respectively. Best classifier mean area under curve (AUC), mean sensitivity, and mean specificity, were computed for the five tree varieties under study, resulting in: 1–Cydonia oblonga (quince) 0.991 (ANN–ABC), 95.89% (ANN–ABC), 95.91% (ANN–ABC); 2–Eucalyptus camaldulensis dehn (river red gum) 1.00 (LDA), 100% (LDA), 100% (LDA); 3–Malus pumila (apple) 0.996 (LDA), 96.63% (LDA), 94.99% (LDA); 4–Pistacia atlantica (mt. Atlas mastic tree) 0.979 (LDA), 91.71% (LDA), 82.57% (LDA); and 5–Prunus armeniaca (apricot) 0.994 (LDA), 88.67% (LDA), 94.65% (LDA), respectively.

Principal component analysis (PCA) is linear method for feature extraction that is known as Karhonen Loove method. PCA was first proposed to recognize face by Turk and Pentland, and was also known as eigenface in 1991 [Turk, 1991]. However, PCA has some weaknesses. The first, it cannot capture the simplest invariance of the face image [Arif et al., 2008b] , when this information is not provided in the training data. The last, the result of feature extraction is global structure [Arif, 2008]. The PCA is very simple, has overcome curse of dimensionality problem, this method have been known and expanded by some researchers to recognize face such as Linear Discriminant Analysis (LDA)[Yambor, 2000; A.M. Martinez, 2003; J.H.P.N. Belhumeur 1998], Linear Preserving Projection that known Lapalacianfaces [Cai, 2005; Cai et al, 2006; Kokiopoulou, 2004; X. He et al., 2005], Independent Component Analysis, Kernel Principal Component Analysis [Scholkopf et al., 1998; Sch olkopf 1999], Kernel Linear Discriminant Analysis (KLDA) [Mika, 1999] and maximum feature value selection of nonlinear function based on Kernel PCA [Arif et al., 2008b]. As we know, PCA is dimensionality reduction method based on object appearance by projecting an original ndimensional (row*column) image into k eigenface where k<<n. Although PCA have been developed into some methods, but in some cases, PCA can outperform LDA, LPP and ICA when it uses small sample size.

Manifold learning approaches such as locally linear embedding algorithm (LLE) and isometric mapping (Isomap) algorithm are aimed to discover the intrinsical low dimensional variables from high-dimensional nonlinear data. While, in order to achieve effective recognition tasks based on manifold learning, many problems remain to be solved. In this paper, we propose unified algorithm based on LLE and linear discriminant analysis (ULLELDA) for those remained problems. First, training samples are mapped into low-dimensional embedding space and then LDA algorithm is used to project samples into discriminant space for enlarging between-class distances and decreasing within-class distance. Second, the unknown samples are directly mapped into discriminant space without the computation of the corresponding one in the low-dimensional embedding space. Experiments on several face databases show the advantages of the proposed algorithm.

Accurate quantification of brain tissues is a very challenging problem in neuroimaging, such as quantification of gray matter (GM), white matter (WM) and cerebrospinal fluid (CSF), and white matter lesions (WMLs). However, on many cases brain tissues and white matter lesions cannot be segmented and separated simultaneously by current techniques. Recently, a TRIO algorithm (TRIOA) is proposed to integrate three algorithms, Independent Component Analysis (ICA), Support Vector Machine (SVM) and Fisher's Linear Discriminant Analysis (FLDA) (ICA+SVM+IFLDA) which can effectively classify GM, WM and CSF by considering MR images as multispectral images in the native coordinate space. This paper further extends TRIOA in conjunction with Band Expansion Process (BEP), called Extended TRIOA (ETRIOA), to classify brain tissues as well as WMLs simultaneously. The accuracy assessment of the ETRIOA was evaluated by using the similarity index. The conducted experimental results demonstrate the clinical applicability of ETRIOA in simultaneous classification of GM, WM, CSF and WMLs.

In this study, terahertz time-domain spectroscopy (THz-TDS) was proposed to identify coffee of three different varieties and three different roasting degrees of one variety. Principal component analysis (PCA) was applied to extract features from frequency-domain spectral data, and the extracted features were used for classification prediction through linear discrimination (LD), support vector machine (SVM), naive Bayes (NB), and k-nearest neighbors (KNN). The classification effect and misclassification of the model were analyzed via confusion matrix. The coffee varieties, namely Catimor, Typica 1, and Typica 2, under the condition of shallow drying were used for comparative tests. The LD classification model combined with PCA had the best effect of dimension reduction classification, while the speed and accuracy reached 20 ms and 100%, respectively. The LD model was found with the highest speed (25 ms) and accuracy (100%) by comparing the classification results of Typica 1 for three different roasting degrees. The coffee bean quality detection method based on THz-TDS combined with a modeling analysis method had a higher accuracy, faster speed, and simpler operation, and it is expected to become an effective detection method in coffee identification.

Hyperspectral image based automatic target recognition (ATR) systems often project the high dimensional hyperspectral reflectance signatures onto a lower dimensional subspace using techniques such as principal components analysis (PCA), Fisher's linear discriminant analysis (LDA) and stepwise LDA. In a general classification framework, these projections are sub-optimal, and in the absence of sufficient training data, are likely to be ill conditioned. In recent work, the authors proposed a divide and conquer approach that partitions the hyperspectral space into contiguous subspaces followed by multi-classifiers and decision fusion. Although this technique alleviated the small sample size problem and provided a good recognition performance in light and moderate pixel mixing, the performance significantly decreased under severe mixing conditions, as does with conventional ATR techniques. In this work, the authors propose a kernel discriminant analysis based projection in each subspace of the partition, followed by the multi-classifier, decision fusion framework to ensure robust recognition even in severe pixel mixing conditions. The performance of the proposed system (as measured by overall target recognition accuracies) is greatly superior to conventional dimensionality reduction techniques, as well as the more recently proposed divide-and-conquer technique.