Multi-view Discriminative Fusion on Canonical Correlation Analysis in Event Classification

Canonical correlation analysis using within-class coupling

The canonical correlation analysis (CCA) aims at measuring linear relationships between two sets of variables (views) that can be used for feature extraction in classification problems with multiview data. However, the correlated features extracted by the CCA may not be class discriminative, since CCA does not utilize the class labels in its traditional formulation. Although there is a method called discriminative CCA (DCCA) that aims to increase the discriminative ability of CCA inspired from the linear discriminant analysis (LDA), it has been shown that the extracted features with this method are identical to those by the LDA with respect to an orthogonal transformation. Therefore, DCCA is simply equivalent to applying single-view (regular) LDA to each one of the views separately. Besides, DCCA and the other similar DCCA approaches have generalization problems due to the sample covariance matrices used in their computation, which are sensitive to outliers and noisy samples. In this paper, we propose a method, called discriminative alternating regression (D-AR), to explore correlated and also discriminative features. D-AR utilizes two (alternating) multilayer perceptrons, each with a linear hidden layer, learning to predict both the class labels and the outputs of each other. We show that the features found by D-AR on training sets significantly accomplish higher classification accuracies on test sets of facial expression recognition, object recognition, and image retrieval experimental data sets.

Steady State Visual Evoked Potential (SSVEP) has been commonly adopted in Brain Computer Interface (BCI) applications. For wearable BCI applications, several aspects of SSVEP-based BCI systems, such as speed, subject variability, and accurate target detection, are under ongoing research investigations. Up to date, Canonical Correlation Analysis (CCA) has been considered the state-of-the-art feature extraction method for SSVEP-based BCI systems. Nevertheless, although CCA outperforms traditional SSVEP detection methods, such as Power Spectral Density Analysis (PSDA), achieving high accuracies when detecting target frequencies is still a challenging task due to user variation and physiological changes in the human body. In this paper, we investigate an SSVEP-based BCI application using wireless EEG recording and an Android tablet-based user interface. We propose a fusion of CCA and PSDA solutions at the score level by dividing their score space into multiple partitions, and extract and combine their complementary discriminative information to minimize the detection error in a linear fashion. We investigated transforming the fusion score space to lower dimensions with the purpose of alleviating redundancy. As such, we employed Principal Component Analysis (PCA), and Linear Discriminant Analysis (LDA). Our experimental results demonstrated that our proposed score fusion method is effective in reducing the effect of noise and non-stationary elements in EEG dynamics. Average detection accuracies improved from 63% for CCA to 72% for fusion+PCA and further improved to 98% for fusion+LDA.

The application of three multivariate analysis techniques (canonical discriminant analysis (CDA), principal component analysis (PCA), and canonical correlation analysis (CCA)) for evaluation of pasture botanical composition data is illustrated and discussed. CDA and PCA were used to describe differences in pasture botanical composition for different microsites within a pasture near Palmerston North, New Zealand. CCA could not be validly applied to this data set because a sampling strategy inappropriate for CCA had been used to collect the data. However, CCA is conceptually ideal for determining association between two groups of variables and CCA was used for a second data set from the Hawkes Bay region to establish association between differences in pasture botanical composition and differences in environmental variables. CCA identified a transition from white clover (Trifolium repens L.) to subterranean clover {Trifolium subterraneum L.) presence associated with decreasing rainfall, and a similar transition from ryegrass presence to browntop presence associated with altitude. There are practical difficulties in obtaining a suitable data set for canonical correlation, but with attention to sampling strategy a more precise definition of effect of environmental factors on pasture botanical composition would be expected.

Dimensionality reduction of image features using the canonical contextual correlation projection

We address the problem of comparing sets of images for object recognition, where the sets may represent variations in an object's appearance due to changing camera pose and lighting conditions. Canonical Correlations (also known as principal or canonical angles), which can be thought of as the angles between two d-dimensional subspaces, have recently attracted attention for image set matching. Canonical correlations offer many benefits in accuracy, efficiency, and robustness compared to the two main classical methods: parametric distribution-based and nonparametric sample-based matching of sets. Here, this is first demonstrated experimentally for reasonably sized data sets using existing methods exploiting canonical correlations. Motivated by their proven effectiveness, a novel discriminative learning method over sets is proposed for set classification. Specifically, inspired by classical Linear Discriminant Analysis (LDA), we develop a linear discriminant function that maximizes the canonical correlations of within-class sets and minimizes the canonical correlations of between-class sets. Image sets transformed by the discriminant function are then compared by the canonical correlations. Classical orthogonal subspace method (OSM) is also investigated for the similar purpose and compared with the proposed method. The proposed method is evaluated on various object recognition problems using face image sets with arbitrary motion captured under different illuminations and image sets of 500 general objects taken at different views. The method is also applied to object category recognition using ETH-80 database. The proposed method is shown to outperform the state-of-the-art methods in terms of accuracy and efficiency.

Chapter 11 - Canonical analysis

Hotelling’s Canonical Correlation Analysis (CCA) works with two sets of related variables, also called views, and its goal is to find their linear projections with maximal mutual correlation. CCA is most suitable for unsupervised feature extraction when given two views but it has been also long known that in supervised learning when there is only a single view of data given, the supervision signal (class-labels) can be given to CCA as the second view and CCA simply reduces to Fisher’s Linear Discriminant Analysis (LDA). However, it is unclear how to use this equivalence for extracting features from multiview data in semisupervised setting (i.e. what modification to the CCA mechanism could incorporate the class-labels along with the two views of the data when labels of some samples are unknown). In this paper, a CCA-based method supplemented by the essence of LDA is proposed for semi-supervised feature extraction from multiview data.KeywordsSemisupervised LearningFeature ExtractionMultiview LearningLDACCA

An emerging trend in video event classification is to learn an event from a bank of concept detector scores. Different from existing work, which simply relies on a bank containing all available detectors, we propose in this paper an algorithm that learns from examples what concepts in a bank are most informative per event, which we call the conceptlet. We model finding the conceptlet out of a large set of concept detectors as an importance sampling problem. Our proposed approximate algorithm finds the optimal conceptlet using a cross-entropy optimization. We study the behavior of video event classification based on conceptlets by performing four experiments on challenging internet video from the 2010 and 2012 TRECVID multimedia event detection tasks and Columbia's consumer video dataset. Starting from a concept bank of more than thousand precomputed detectors, our experiments establish there are (sets of) individual concept detectors that are more discriminative and appear to be more descriptive for a particular event than others, event classification using an automatically obtained conceptlet is more robust than using all available concepts, and conceptlets obtained with our cross-entropy algorithm are better than conceptlets from state-of-the-art feature selection algorithms. What is more, the conceptlets make sense for the events of interest, without being programmed to do so.

Research was conducted to investigate the use of optical, air-born acoustic and plasma charge signals for detecting defects in laser welds. Full penetration, partial penetration and gapped-full penetration butt-joint welds were made in 1.4mm cold-rolled steel sheet. A 3 kW CO2 laser was used to produce these three classes of welds while laser power, travel speed and joint fitup were varied to produce the various conditions. Visible light, audible sound and electrical charge near the plume were measured simultaneously in real-time using a silicon photodiode, a condensor microphone and a plasma charge sensing gas nozzle, respectively. The spectra of the sensor signals were analyzed by a statistical technique known as Linear Discriminant Analysis (LDA). Stepwise LDA was used to identify the most important frequency bands and canonical discriminant analysis was employed to develop discriminant functions. Reliability analysis of the training samples revealed that this LDA method classified laser welds with reliability ranging from 85% to 96%, depending on the welds selected for inclusion in the training set. When the linear discriminant function was validated using new (untrained) samples, it distinguished the three different classes of laser welds with approximately 87% accuracy. Among the three signals, sound produced better separation between the classes of welds, although a discriminator based on all three signals combined and one that used only plasma charge and acoustic signals were of almost equal reliability.Research was conducted to investigate the use of optical, air-born acoustic and plasma charge signals for detecting defects in laser welds. Full penetration, partial penetration and gapped-full penetration butt-joint welds were made in 1.4mm cold-rolled steel sheet. A 3 kW CO2 laser was used to produce these three classes of welds while laser power, travel speed and joint fitup were varied to produce the various conditions. Visible light, audible sound and electrical charge near the plume were measured simultaneously in real-time using a silicon photodiode, a condensor microphone and a plasma charge sensing gas nozzle, respectively. The spectra of the sensor signals were analyzed by a statistical technique known as Linear Discriminant Analysis (LDA). Stepwise LDA was used to identify the most important frequency bands and canonical discriminant analysis was employed to develop discriminant functions. Reliability analysis of the training samples revealed that this LDA method classified laser welds with rel...

Vegetation indices derived from remote sensing measurements are commonly used to describe and monitor vegetation. However, the same plant community can have a different NDVI (normalized difference vegetation index) depending on weather conditions, and this complicates classification of plant communities. The present study develops methods of classifying the types of plant communities based on long-term NDVI data (MODIS/Aqua). The number of variables is reduced by introducing two integrated parameters of the NDVI seasonal series, facilitating classification of the meadow, steppe, and forest plant communities in Siberia using linear discriminant analysis. The quality of classification conducted by using the markers characterizing NDVI dynamics during 2003–2017 varies between 94% (forest and steppe) and 68% (meadow and forest). In addition to determining phenological markers, canonical correlations have been calculated between the time series of the proposed markers and the time series of monthly average air temperatures. Based on this, each pixel with a definite plant composition can be characterized by only four values of canonical correlation coefficients over the entire period analyzed. By using canonical correlations between NDVI and weather parameters and employing linear discriminant analysis, one can obtain a highly accurate classification of the study plant communities.

Canonical correlation analysis (CCA) has recently attracted great attention and many experimental results have illustrated its effectiveness. In this paper, we study the relationship between CCA classifier and minimum squared error (MSE) classifier. It helps us look into the nature of CCA classifier. In traditional CCA method, the class-membership matrix is deliberately coded in full rank. Under this case, we will prove CCA is equivalent to MSE classifier. It is also shown that even the class-membership matrix is centered and thus not in full rank, CCA is equivalent to Fisher linear discriminant analysis (FDA). Some experiments are presented to verify the results.

Classification accuracy and correlation: LDA in failure prediction

This study explores various options available for choosing the number of principal coordinates m in the canonical analysis of principal coordinates ‘CAP’, a useful procedure that has wide-ranging application wherever multivariate data sets are collected or generated. Choosing too few coordinates (small m) in this constrained (i.e. hypothesis-based) ordination procedure may lead to inadequate separation of the groups (when used as a canonical discriminant analysis) or to inadequate correlation between explanatory and response variables (when used as a canonical correlations analysis), whereas choosing too many (large m) may lead to overparameterization, resulting in overfitting of the data and spurious relationships. It is shown here that the optimum number of principal coordinates is simply the one that results in the smallest P value in the canonical analysis carried out using permutations. For data in which more than one m value results in the same minimum P value, m should be chosen from that set to be the number of principal coordinates that minimizes the leave-one-out residual sum of squares. This choice of m provides suitable solutions for each of the 17 case studies investigated here (which yielded 17 canonical discriminant analyses and 7 canonical correlation analyses).

Multi-omics analysis offers a promising avenue to a better understanding of complex biological phenomena. In particular, untangling the pathophysiology of multifactorial health conditions such as the inflammatory bowel disease (IBD) could benefit from simultaneous consideration of several omics levels. However, taking full advantage of multi-omics data requires the adoption of suitable new tools. Multi-view learning, a machine learning technique that natively joins together heterogeneous data, is a natural source for such methods. Here we present a new approach to variable selection in unsupervised multi-view learning by applying stability selection to canonical correlation analysis (CCA). We apply our method, StabilityCCA, to simulated and real multi-omics data, and demonstrate its ability to find relevant variables and improve the stability of variable selection. In a case study on an IBD microbiome data set, we link together metagenomics and metabolomics, revealing a connection between their joint structure and the disease, and identifying potential biomarkers. Our results showcase the usefulness of multi-view learning in multi-omics analysis and demonstrate StabilityCCA as a powerful tool for biomarker discovery.