A unified framework for generalized Linear Discriminant Analysis

Chapter 9 - Effective dimensionality reduction model with machine learning classification for microarray gene expression data

The classification of tissue samples based on gene expression data is an important problem in medical diagnosis of diseases such as cancer. In gene expression data, the number of genes is usually very high (in the thousands) compared to the number of data samples (in the tens or low hundreds); that is, the data dimension is large compared to the number of data points (such data is said to be undersampled). To cope with performance and accuracy problems associated with high dimensionality, it is commonplace to apply a preprocessing step that transforms the data to a space of significantly lower dimension with limited loss of the information present in the original data. Linear Discriminant Analysis (LDA) is a well-known technique for dimension reduction and feature extraction, but it is not applicable for undersampled data due to singularity problems associated with the matrices in the underlying representation. This paper presents a dimension reduction and feature extraction scheme, called Uncorrelated Linear Discriminant Analysis (ULDA), for undersampled problems and illustrates its utility on gene expression data. ULDA employs the Generalized Singular Value Decomposition method to handle undersampled data and the features that it produces in the transformed space are uncorrelated, which makes it attractive for gene expression data. The properties of ULDA are established rigorously and extensive experimental results on gene expression data are presented to illustrate its effectiveness in classifying tissue samples. These results provide a comparative study of various state-of-the-art classification methods on well-known gene expression data sets.

Linear Discriminant Analysis (LDA) has been one of the popular subspace methods for face recognition. But this method suffers from the small sample size (SSS) problem, also known as 'curse of dimensionality'. Various techniques have been proposed in literature to overcome this limitation. But it is still unclear which method provides the best solution to SSS problem. In this paper, we have investigated the performance of some popular subspace methods such as principal component analysis (PCA), PCA + LDA, LDA via QR decomposition, Null-space LDA, Exponential Discriminant Analysis (EDA), PCA+EDA etc. Extensive experiments have been performed on five publically available face datasets viz. AR, CMU-PIE, PIX, Yale and YaleB. The performance is measured in terms of average classification accuracy. Experimental results show that the performance increases with the increase in the number of images per person in training set irrespective of the datasets. There is no clear winner among the subspace methods under investigation. But, the performance of PCA+LDA and PCA+EDA is consistent in tackling SSS problem irrespective of the dataset and can also handle the illumination variation in face recognition.

A topological approach for cancer subtyping from gene expression data.

The recent explosion in procurement and availability of high-dimensional gene- and protein-expression profile datasets for cancer diagnostics has necessitated the development of sophisticated machine learning tools with which to analyze them. A major limitation in the ability to accurate classify these high-dimensional datasets stems from the 'curse of dimensionality', occurring in situations where the number of genes or peptides significantly exceeds the total number of patient samples. Previous attempts at dealing with this issue have mostly centered on the use of a dimensionality reduction (DR) scheme, Principal Component Analysis (PCA), to obtain a low-dimensional projection of the high-dimensional data. However, linear PCA and other linear DR methods, which rely on Euclidean distances to estimate object similarity, do not account for the inherent underlying nonlinear structure associated with most biomedical data. The motivation behind this work is to identify the appropriate DR methods for analysis of high-dimensional gene- and protein-expression studies. Towards this end, we empirically and rigorously compare three nonlinear (Isomap, Locally Linear Embedding, Laplacian Eigenmaps) and three linear DR schemes (PCA, Linear Discriminant Analysis, Multidimensional Scaling) with the intent of determining a reduced subspace representation in which the individual object classes are more easily discriminable.

In order to understand even the simplest cellular processes, we need to integrate proteomic, gene expression and other biomolecular data. To date, most computational approaches aimed at integrating proteomics and gene expression data use direct gene/protein correlation measures. However, due to post-transcriptional and translational regulations, the correspondence between the expression of a gene and its protein is complicated. We apply a multivariate statistical method, co-inertia analysis (CIA), to visualise gene and proteomic expression data stemming from the same biological samples. Principal components analysis or correspondence analysis can be used for data exploration on single datasets. CIA is then used to explore the relationships between two or more datasets. We further explore the data by projecting gene ontology (GO) information onto these plots to describe the cellular processes in action. We apply these techniques to gene expression and protein abundance data from studies of the human malarial parasite life cycle and the NCI-60 cancer cell lines. In each case, we visualise gene expression, protein abundance and GO classes in the same low dimensional projections and identify GO classes that are likely to be of biological importance.

A reliable and precise classification of tumors is essential for successful diagnosis and treatment of cancer. cDNA microarrays and high-density oligonucleotide chips are novel biotechnologies increasingly used in cancer research. By allowing the monitoring of expression levels in cells for thousands of genes simultaneously, microarray experiments may lead to a more complete understanding of the molecular variations among tumors and hence to a finer and more informative classification. The ability to successfully distinguish between tumor classes (already known or yet to be discovered) using gene expression data is an important aspect of this novel approach to cancer classification. This article compares the performance of different discrimination methods for the classification of tumors based on gene expression data. The methods include nearest-neighbor classifiers, linear discriminant analysis, and classification trees. Recent machine learning approaches, such as bagging and boosting, are also considered. The discrimination methods are applied to datasets from three recently published cancer gene expression studies.

In this paper, a new Face image classification or Recognition method based on Two Dimensional Discrete Cosine Transform (2D - DCT) with Linear Discriminant Analysis (LDA) and Probabilistic Neural Network (PNN) is proposed. This method consists of three steps, i) Transformation of images from special to frequency domain using Two dimensional discrete cosine transform ii) Feature extraction using Linear Discriminant Analysis and iii) classification using Probabilistic Neural Network. Linear Discriminant Analysis searches the directions for maximum discrimination of classes in addition to dimensionality reduction. Combination of Two Dimensional Discrete Cosine transform and Linear Discriminant Analysis is used for improving the capability of Linear Discriminant Analysis when few samples of images are available. Probabilistic Neural network (PNN) is a promising tool and gives fast and accurate classification of face images. Evaluation was performed on two face data bases. First database of 400 face images from AT&T (ORL) face database, and the second database of thirteen students are taken. The proposed method gives fast and better recognition rate when compared to other classifiers. The main advantage of this method is its high speed processing capability and low computational requirements in terms of both speed and memory utilizations.

Multi-valued neurons (MVN) are the neural processing elements with complex-valued weights and high functionality. It is possible to implement an arbitrary mapping described by partial-defined multiple-valued function on the single MVN. The MVN-based neural networks are applied to temporal classification of images of gene expression patterns, obtained by confocal scanning microscopy. The classification results confirmed the efficiency of this method for image recognition. It was shown that frequency domain of the representation of images is highly effective for their description.

The classification of cancer subtypes, which is critical for successful treatment, has been studied extensively with the use of gene expression profiles from oligonucleotide chips or cDNA microarrays. Various pattern recognition methods have been successfully applied to gene expression data. However, these methods are not optimal, rather they are high-performance classifiers that emphasize only classification accuracy. In this paper, we propose an approach for the construction of the optimal linear classifier using gene expression data. Two linear classification methods, linear discriminant analysis (LDA) and discriminant partial least-squares (DPLS), are applied to distinguish acute leukemia subtypes. These methods are shown to give satisfactory accuracy. Moreover, we determined optimally the number of genes participating in the classification (a remarkably small number compared to previous results) on the basis of the statistical significance test. Thus, the proposed method constructs the optimal classifier that is composed of a small size predictor and provides high accuracy.

Problem statement: In facial biometrics, face features are used as the required human traits for automatic recognition. Feature extracted from face images are significant for face biometrics system performance. Approach: In this thesis, a framework of facial biometric was designed based on two subspace methods i.e., Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). First, PCA is used for dimension reduction, where original face images are projected into lower-dimensional face representations. Second, LDA was proposed to provide a solution of better discriminant. Both PCA and LDA features were presented to Euclidean distance measurement which is conveniently used as a benchmark. The algorithms were evaluated in face identification and verification using a standard face database-AT and T and a locally collected database-CBE. Each database consists of 400 images and 320 images respectively. Results: LDA-based methods outperform PCA for both face identification and verification. For face identification, PCA achieves accuracy of 91.9% (AT and T) and 76.7% (CBE) while LDA 94.2% (AT and T) and 83.1% (CBE). For face verification, PCA achieves Equal Error Rate (EER) of 1.15% (AT and T), 7.3% (CBE) while LDA 0.78% (AT and T) and 5.81% (CBE). Conclusion/Recommendations: This study had proved that, when given sufficient training samples, LDA is able to provide better discriminant ability in feature extraction for face biometrics.

Interpretability of selected variables and performance comparison of variable selection methods in a polyethylene and polypropylene NIR classification task

Fisher linear discriminant analysis (FLDA) is a classic linear feature extraction and dimensionality reduction approach for face recognition. It is known that geometric distribution weight information of image data plays an important role in machine learning approaches. However, FLDA does not employ the geometric distribution weight information of facial images in the training stage. Hence, its recognition accuracy will be affected. In order to enhance the classification power of FLDA method, this paper utilizes radial basis function (RBF) with fractional order to model the geometric distribution weight information of the training samples and proposes a novel geometric distribution weight information based Fisher discriminant criterion. Subsequently, a geometric distribution weight information based LDA (GLDA) algorithm is developed and successfully applied to face recognition. Two publicly available face databases, namely, ORL and FERET databases, are selected for evaluation. Compared with some LDA-based algorithms, experimental results exhibit that our GLDA approach gives superior performance.

To design a face gloss classification model and to provide an automatic and quantitative approach for the diagnosis of Chinese medicine (CM) based on the face images. To classify the face gloss images into two groups (gloss and non-gloss), feature extraction methods were applied to the original images. The original images were supposed to obtain a more ideal representation in which gloss information was better revealed in four color spaces [including red, green, blue (RGB), hue, saturation, value (HSV), Gray and Lab]. Principal component analysis (PCA), 2-dimensional PCA (2DPCA), 2-directional 2-dimensional PCA [(2D)2PCA], linear discriminant analysis (LDA), 2-dimensional LDA (2DLDA), and partial least squares (PLS) were used as the feature extraction methods of face gloss. k nearest neighbor was used as the classifification method. All the six feature extraction methods were useful in extracting information of face gloss, especially LDA, which had the best prediction accuracy in the 4 color spaces. The average accuracy of LDA in the Lab was 7%-10% higher than that of PCA, 2DPCA, (2D)2PCA and 2DLDA P<0.05). The prediction accuracy of LDA reached 98% in the Lab color space and showed practical usage in clinical diagnosis. The consistent rate between the CM experts and the facial diagnosis system was 81%. A computer-assisted classifification model was designed to provide an automatic and quantitative approach for the gloss diagnosis of CM based on the face images.

BackgroundA complete understanding of the regulatory mechanisms of gene expression is the next important issue of genomics. Many bioinformaticians have developed methods and algorithms for predicting transcriptional regulatory mechanisms from sequence, gene expression, and binding data. However, most of these studies involved the use of yeast which has much simpler regulatory networks than human and has many genome wide binding data and gene expression data under diverse conditions. Studies of genome wide transcriptional networks of human genomes currently lag behind those of yeast.ResultsWe report herein a new method that combines gene expression data analysis with promoter analysis to infer transcriptional regulatory elements of human genes. The Z scores from the application of gene set analysis with gene sets of transcription factor binding sites (TFBSs) were successfully used to represent the activity of TFBSs in a given microarray data set. A significant correlation between the Z scores of gene sets of TFBSs and individual genes across multiple conditions permitted successful identification of many known human transcriptional regulatory elements of genes as well as the prediction of numerous putative TFBSs of many genes which will constitute a good starting point for further experiments. Using Z scores of gene sets of TFBSs produced better predictions than the use of mRNA levels of a transcription factor itself, suggesting that the Z scores of gene sets of TFBSs better represent diverse mechanisms for changing the activity of transcription factors in the cell. In addition, cis-regulatory modules, combinations of co-acting TFBSs, were readily identified by our analysis.ConclusionBy a strategic combination of gene set level analysis of gene expression data sets and promoter analysis, we were able to identify and predict many transcriptional regulatory elements of human genes. We conclude that this approach will aid in decoding some of the important transcriptional regulatory elements of human genes.