Semi-supervised Feature Extraction Using Independent Factor Analysis

Graph embedding (GE) frameworks are used for extracting the discriminative features of hyperspectral images (HSIs). However, it is difficult to select a proper neighborhood size for graph construction. To overcome this difficulty, a semisupervised feature extraction (FE) method, called semisupervised collaborative representation graph embedding (SCRGE), is proposed. The proposed algorithm utilizes collaborative representation (CR) to obtain the collaborative coefficients of labeled and unlabeled samples. Then, a semisupervised graph is constructed using the collaborative coefficients of the labeled samples within the same class and the collaborative coefficients of the unlabeled samples, and an interclass graph is constructed using the collaborative coefficients of the labeled samples in different classes. Finally, a projection matrix for FE is obtained by embedding these graphs into a low-dimensional space. SCRGE not only inherits the merits of CR to reveal the collaborative reconstructive properties of data but also enhances intraclass compactness and interclass separability to improve the discriminating power for classification. Experimental results on three real HSIs datasets demonstrate that SCRGE outperforms other state-of-the-art FE methods in terms of classification accuracy.

Recently, the semi-supervised feature extraction, which uses both a few labeled samples and a large quantity of unlabeled samples to improve discrimination of extracted features, has attracted attention in PolSAR data processing. However, most of the feature extraction methods are based on matrix operations and need to re-arrange the multiple features into vectors. Such a vectorization way may break the spatial structure and correlation among the neighboring pixels, which leads to the degradation of classification performance. To address this issue and incorporate the superiority of semi-supervised learning, a novel semi-supervised tensorial locally linear embedding (STLLE) method is proposed for PolSAR data. Each pixel and its spatial neighbors in a local window are represented as a third-order tensor, whose first two orders represent the neighborhood pixels, and the third order denotes the multiple features. The feature extraction is converted to find an optimal projection that can transform the high-dimensional features to low-dimensional ones. This can be solved by designing an objective function, in which the tensorial discriminant item includes the between-class and within-class scatter matrices on each mode of the labeled tensors. A regularizer is based on tensorial locally linear embedding graph using labeled and unlabeled tensor samples. Moreover, the pairwise class constraints are also introduced. Finally, we can use all the extracted features with a classifier such as K-Nearest Neighbor. Experimental results on three real PolSAR datasets demonstrate that STLLE significantly improves classification accuracies compared with state-of-the-art methods, which clearly verifies that the extracted features preserve the data structures well and are more discriminative.

Independent factor discriminant analysis

A probabilistic model with spike-and-slab regularization for inferential fault detection and isolation of industrial processes

Face technologies which can be applied to access control and surveillance, are essential to intelligent vision-based human computer interaction. The research efforts in this field include face detecting, face recognition, face retrieval, etc. However, these tasks are challenging because of variability in view point, lighting, pose and expression of human faces. The ideal face representation should consider the variability so as to we can develop robust algorithms for our applications. Independent Component Analysis (ICA) as an unsupervised learning technique has been used to find such a representation and obtained good performances in some applications. In the first part of this paper, we depict the models of ICA and its extensions: Independent Subspace Analysis (ISA) and Topographic ICA (TICA).Then we summaraize the process in the applications of ICA and its extension in Face images. At last we propose a promising direction for future research.

How many channels are suitable for independent component analysis in motor imagery brain-computer interface

We present a new approach to independent component analysis (ICA) by extending the formulation of univariate source signals to multivariate source signals. The new approach is termed independent vector analysis (IVA). In the model, we assume that linear mixing model exists in each dimension separately, and the latent sources are independent of the others. In contrast to ICA, the sources are random vectors, not just single variables, which means the elements of a random vector are closely related to the others. Thus, we assume the dependency between the elements of a source vector. In this manner, we define dependence between vectors as Kullback-Leibler divergence between the total joint probability of vectors and the product of marginal probabilities of vectors. Then, the model allows independence between multivariate source signals represented as random vectors, and dependence between the source signals within the vector representation. The proposed vector density model can for example capture variance dependencies within a vector source signal. There are several applications of this new formulation. In the separation of acoustic sources, the algorithm mitigates the permutation problem, i.e. the usual ICA algorithms applied in to the frequency domain mixture data suffer from the unknown permutation of the output signals. Although there are several engineering solutions to fix this problem after the ICA stage, the proposed method provides a natural solution to the problem by capturing the inherent dependencies of the acoustic signals. It therefore avoids the permutation problem and allows the separation of sources in very challenging environments for many sound sources.

Principal Component Analysis (PCA), also known as Proper Orthogonal Decomposition (POD), is one of the prevailing Reduced Order Modeling (ROM) techniques for aerodynamic data analysis. Along with the PCA, its variant for missing data, gappy POD, has recently found its application by transforming problems into missing data problems. In this paper, an alternative based on probability theory, i.e., Probabilistic Principal Component Analysis (PPCA), is employed for aerodynamic data reconstruction problems in lieu of the PCA and the gappy POD. Unlike the PCA, the PPCA generates spanning vectors of low-dimensional space iteratively with an Expectation Maximization (EM) algorithm. The combined algorithm, termed as Expectation-Maximization Probabilistic Principal Component Analysis (EM-PCA), can deal with not only complete data but also incomplete data sets, since the EM algorithm naturally incorporates missing data in its formulation. Therefore, the EM-PCA has potential to be a substitute for both PCA and gappy POD. For illustration, the EM-PCA is applied to complete as well as incomplete snapshot ensembles compiled from aerodynamic simulations, and acquired modes and restored data are verified with those obtained by the PCA and the gappy POD.

In order to identify time-varying transient modal parameters only from nonstationary vibration response measurement signals for slow linear time-varying (SLTV) structures which are weakly damped, a moving window differential evolution (DE) independent component analysis- (ICA-) based operational modal analysis (OMA) method is proposed in this paper. Firstly, in order to overcome the problems in traditional ICA-based OMA, such as easy to go into local optima and difficult-to-identify high-order modal parameters, we combine DE with ICA and propose a differential evolution independent component analysis- (DEICA-) based OMA method for linear time invariant (LTI) structures. Secondly, we combine the moving widow technique with DEICA and propose a moving window differential evolution independent component analysis- (MWDEICA-) based OMA method for SLTV structures. The MWDEICA-based OMA method has high global searching ability, robustness, and complexity of time and space. The modal identification results in a three-degree-of-freedom structure with slow time-varying mass show that this MWDEICA-based OMA method can identify transient time-varying modal parameters effectively only from nonstationary vibration response measurement signals and has better performances than moving window traditional ICA-based OMA.

Dimensionality of semantic meaning and evoked potential fields

Inferring resting-state connectivity patterns from functional magnetic resonance imaging (fMRI) data is a challenging task for any analytical technique. In this paper, we review a probabilistic independent component analysis (PICA) approach, optimized for the analysis of fMRI data, and discuss the role which this exploratory technique can take in scientific investigations into the structure of these effects. We apply PICA to fMRI data acquired at rest, in order to characterize the spatio-temporal structure of such data, and demonstrate that this is an effective and robust tool for the identification of low-frequency resting-state patterns from data acquired at various different spatial and temporal resolutions. We show that these networks exhibit high spatial consistency across subjects and closely resemble discrete cortical functional networks such as visual cortical areas or sensory-motor cortex.

Robust probabilistic principal component analysis for process modeling subject to scaled mixture Gaussian noise

Electrocardiogram (ECG) signals are normally contaminated by various physiological and nonphysiological artifacts. Among these artifacts baseline wandering, electrode movement and muscle artifacts are particularly difficult to remove. Independent component analysis (ICA) is a well-known technique of blind source separation (BSS) and is extensively used in literature for ECG artifact elimination. In this article, the independent vector analysis (IVA) is used for artifact removal in the ECG data. This technique takes advantage of both the canonical correlation analysis (CCA) and the ICA due to the utilization of second-order and high order statistics for un-mixing of the recorded mixed data. The utilization of recorded signals along with their delayed versions makes the IVA-based technique more practical. The proposed technique is evaluated on real and simulated ECG signals and it shows that the proposed technique outperforms the CCA and ICA because it removes the artifacts while altering the ECG signals minimally.

Independent vector analysis (IVA) is a special form of independent component analysis (ICA), which has demonstrated its prominent performance in solving convolutive blind source separation (BSS) problems in the frequency domain. Most IVA algorithms are based on optimizing certain contrast functions, where the main difficulty of these approaches lies in finding a reliable and fast estimation of the unknown distribution of sources. Despite the rich availability of efficient tensorial approaches to the standard ICA problem, these methods have not been explored considerably for IVA. In this article, we propose a matrix joint diagonalization approach to solve the complex IVA problem. The new factorization neither relies on a whitening process, nor does it require an estimate of the joint probability distribution of the dependent signal groups. The latter is in contrast to most IVA approaches up to date. The underlying geometry of the problem is investigated together with a critical point analysis of the resulting cost function. A conjugate gradient algorithm on the appropriate manifold setting is developed.

Independent vector analysis followed by HMM-based feature enhancement for robust speech recognition