Independent Vector Analysis: Definition and Algorithms

Independent vector analysis followed by HMM-based feature enhancement for robust speech recognition

Independent vector analysis (IVA) is a special form of independent component analysis (ICA), which has demonstrated its prominent performance in solving convolutive blind source separation (BSS) problems in the frequency domain. Most IVA algorithms are based on optimizing certain contrast functions, where the main difficulty of these approaches lies in finding a reliable and fast estimation of the unknown distribution of sources. Despite the rich availability of efficient tensorial approaches to the standard ICA problem, these methods have not been explored considerably for IVA. In this article, we propose a matrix joint diagonalization approach to solve the complex IVA problem. The new factorization neither relies on a whitening process, nor does it require an estimate of the joint probability distribution of the dependent signal groups. The latter is in contrast to most IVA approaches up to date. The underlying geometry of the problem is investigated together with a critical point analysis of the resulting cost function. A conjugate gradient algorithm on the appropriate manifold setting is developed.

Independent vector analysis (IVA) minimizes an objective function to estimate separation filters that separate mixture signals into individual source signals. Unfortunately, IVA often suffers from the well-known block permutation problem. To mitigate that problem, the use of geometry knowledge has been studied, but two crucial issues remain: the necessity of non-differential processes outside the minimization and of high-level geometrical clues such as the directions of arrival (DOAs) of the source signals. This paper thus presents a novel IVA method whose objective function has a differentiable max-directivity beamforming normalization (MDBN) term. This term uses geometry knowledge from only a low-level geometrical clue (the positions of the microphone array) via the traditional beamforming (BF) concept that each separation filter should have a maximum gain for each specific DOA across all frequency bins. Thus, our overall objective function can be minimized by gradient descent, and the MDBN term encourages the separation filters to focus on specific directions, which implicitly estimates the most reasonable DOAs of the source signals at each iteration. Therefore, our method uses geometry knowledge while avoiding the above two issues and estimates good separation filters mitigating the permutation problem. Several experiments show that our method outperforms the conventional BF and IVA methods.

Electrocardiogram (ECG) signals are normally contaminated by various physiological and nonphysiological artifacts. Among these artifacts baseline wandering, electrode movement and muscle artifacts are particularly difficult to remove. Independent component analysis (ICA) is a well-known technique of blind source separation (BSS) and is extensively used in literature for ECG artifact elimination. In this article, the independent vector analysis (IVA) is used for artifact removal in the ECG data. This technique takes advantage of both the canonical correlation analysis (CCA) and the ICA due to the utilization of second-order and high order statistics for un-mixing of the recorded mixed data. The utilization of recorded signals along with their delayed versions makes the IVA-based technique more practical. The proposed technique is evaluated on real and simulated ECG signals and it shows that the proposed technique outperforms the CCA and ICA because it removes the artifacts while altering the ECG signals minimally.

A new independent component analysis (ICA) formulation called independent vector analysis (IVA) was proposed in order to solve the permutation problem in convolutive blind source separation (BSS). Instead of running ICA in each frequency bin separately and correcting the disorder with an additional algorithmic scheme afterwards, IVA exploited the dependency among the frequency components of a source and dealt with them as a multivariate source by modeling it with sparse and spherically, or radially, symmetric joint probability density functions (pdfs). In this paper, we compare the speech separation performances of IVA by using a group of l <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sup> -norm-invariant sparse pdfs where the value of and the sparseness can be controlled. Also, we derive an IVA algorithm from a nonparametric perspective with the constraint of spherical symmetry and high dimensionality. Simulation results confirm the efficiency of assuming sparseness and spherical symmetry for the speech model in the frequency domain.

Unsupervised latent variable models-blind source separation (BSS) especially-enjoy a strong reputation for their interpretability. But they seldom combine the rich diversity of information available in multiple datasets, even though multidatasets yield insightful joint solutions otherwise unavailable in isolation. We present a direct, principled approach to multidataset combination that takes advantage of multidimensional subspace structures. In turn, we extend BSS models to capture the underlying modes of shared and unique variability across and within datasets. Our approach leverages joint information from heterogeneous datasets in a flexible and synergistic fashion. We call this method multidataset independent subspace analysis (MISA). Methodological innovations exploiting the Kotz distribution for subspace modeling, in conjunction with a novel combinatorial optimization for evasion of local minima, enable MISA to produce a robust generalization of independent component analysis (ICA), independent vector analysis (IVA), and independent subspace analysis (ISA) in a single unified model. We highlight the utility of MISA for multimodal information fusion, including sample-poor regimes ( N = 600 ) and low signal-to-noise ratio, promoting novel applications in both unimodal and multimodal brain imaging data.

The independent vector analysis (IVA) algorithm employs a multivariate source prior to retain the dependency between different frequency bins of each source and thereby avoids the permutation problem that is inherent to blind source separation (BSS). In this paper, a multivariate Student's t distribution is adopted as the source prior, which because of its heavy tail nature can better model the large amplitude information in the frequency bins. Therefore it can improve the separation performance and the convergence speed of the IVA and fast version of the IVA (FastIVA) algorithms as compared with the IVA algorithm based on another multivariate super Gaussian source prior. Separation performance with real binaural room impulse responses (BRIRs) is evaluated by detailed simulation studies when using the different source priors, and the experimental results confirm that the IVA and the FastIVA with the proposed multivariate Student's t source prior can consistently achieve improved and faster separation performance.

Electrocardiogram (ECG) signal processing aims basically 1) at artifact reduction to make the ECG signals cleaner and better interpretable by human or machine observers, 2) at revealing aspects not immediately observable in plain measured ECG signals even after artifact reduction, or 3) at diagnostics decision support and automated ECG signal interpretation, including classification of ECG signals into different classes associated with normal or pathological heart function. Thus, sports related applications aside, body surface ECG signal processing aims at enhancing ECG based diagnostics. In this Chapter, we review and demonstrate a statistical signal processing approach, independent component analysis (ICA), which is inherently very suitable for ECG signal processing regarding the aims 1) and 2) above, and also equally applicable as a component in systems aimed at accomplishing the aim 3). For more on general ECG signal processing, the reader is directed to the textbook written by Sornmo & Laguna (2005), and for a thorough treatment of ICA to the Hyvarinen’s book (Hyvarinen et al., 2001). A concise review on ICA in ECG signal processing has been presented by Castells et al. (2007a). In this Chapter, we describe and illustrate several widely adopted applications of ICA in ECG signal processing, and discuss associated practical aspects, some of which are not generally found in the literature. The treatment of the matter is aimed at conceptual and practical understanding, leaving the mathematical derivations and proofs far mostly for the interested reader to find in the references. ICA (Castells et al., 2007a; Comon, 1994; Hyvarinen et al., 2001; Hyvarinen & Oja, 2000; Naik & Kumar, 2011) is a statistical signal processing method for decomposing a set of signals into a set of mutually independent component signals. In general, in the applications of ICA, including in ECG signal processing, the objective is that the resulting independent component signals are the original source signals. Since ICA operates purely based on the input signals and a few assumptions, ICA belongs to the class of methods called blind source separation methods. For ICA, a source signal is called an independent component (IC). The terms ‘IC’ and ‘source signal’ are here used interchangeably. For ECG, the source signals are the bioelectrical signals generated by the heart, and all the possible artifact signals. Generally, ICA input signals are the observed signals, which may be measurement time series, such as sampled voltage values in time as in the case of ECGs, image pixel values, or basically any sets of values fulfilling the assumptions of ICA. In the sequel, the term

Analysis of time-evolving data is crucial to understand the functioning of dynamic systems such as the brain. For instance, analysis of functional magnetic resonance imaging (fMRI) data collected during a task may reveal spatial regions of interest, and how they evolve during the task. However, capturing underlying spatial patterns as well as their change in time is challenging. The traditional approach in fMRI data analysis is to assume that underlying spatial regions of interest are static. In this article, using fractional amplitude of low-frequency fluctuations (fALFF) as an effective way to summarize the variability in fMRI data collected during a task, we arrange time-evolving fMRI data as a subjects by voxels by time windows tensor, and analyze the tensor using a tensor factorization-based approach called a PARAFAC2 model to reveal spatial dynamics. The PARAFAC2 model jointly analyzes data from multiple time windows revealing subject-mode patterns, evolving spatial regions (also referred to as networks) and temporal patterns. We compare the PARAFAC2 model with matrix factorization-based approaches relying on independent components, namely, joint independent component analysis (ICA) and independent vector analysis (IVA), commonly used in neuroimaging data analysis. We assess the performance of the methods in terms of capturing evolving networks through extensive numerical experiments demonstrating their modeling assumptions. In particular, we show that (i) PARAFAC2 provides a compact representation in all modes, i.e., subjects, time, and voxels, revealing temporal patterns as well as evolving spatial networks, (ii) joint ICA is as effective as PARAFAC2 in terms of revealing evolving networks but does not reveal temporal patterns, (iii) IVA's performance depends on sample size, data distribution and covariance structure of underlying networks. When these assumptions are satisfied, IVA is as accurate as the other methods, (iv) when subject-mode patterns differ from one time window to another, IVA is the most accurate. Furthermore, we analyze real fMRI data collected during a sensory motor task, and demonstrate that a component indicating statistically significant group difference between patients with schizophrenia and healthy controls is captured, which includes primary and secondary motor regions, cerebellum, and temporal lobe, revealing a meaningful spatial map and its temporal change.

Fast fixed-point independent vector analysis (FastIVA) is an improved independent vector analysis (IVA) method, which can achieve faster and better separation performance than original IVA. As an example IVA method, it is designed to solve the permutation problem in frequency domain independent component analysis by retaining the higher order statistical dependency between frequencies during learning. However, the performance of all IVA methods is limited due to the dimensionality of the parameter space commonly encountered in practical frequency-domain source separation problems and the spherical symmetry assumed with the source model. In this article, a particular permutation problem encountered in using the FastIVA algorithm is highlighted, namely the block permutation problem. Therefore a new audio video based fast fixed-point independent vector analysis algorithm is proposed, which uses video information to provide a smart initialization for the optimization problem. The method cannot only avoid the ill convergence resulting from the block permutation problem but also improve the separation performance even in noisy and high reverberant environments. Different multisource datasets including the real audio video corpus AV16.3 are used to verify the proposed method. For the evaluation of the separation performance on real room recordings, a new pitch based evaluation criterion is also proposed.

In this paper, we present a quasi-Newton (QN) algorithm for joint independent subspace analysis (JISA). JISA is a recently proposed generalization of independent vector analysis (IVA). JISA extends classical blind source separation (BSS) to jointly resolve several BSS problems by exploiting statistical dependence between latent sources across mixtures, as well as relaxing the assumption of statistical independence within each mixture. Algebraically, JISA based on second-order statistics amounts to coupled block diagonalization of a set of covariance and cross-covariance matrices, as well as block diagonalization of a single permuted covariance matrix. The proposed QN algorithm achieves asymptotically the minimal mean square error (MMSE) in the separation of multidimensional Gaussian components. Numerical experiments demonstrate convergence and source separation properties of the proposed algorithm.KeywordsBlind source separationIndependent vector analysisIndependent subspace analysisJoint block diagonalization

When applied to functional magnetic resonance imaging (fMRI) data, independent vector analysis (IVA) provides superior performance in capturing subject variability within one group, as compared to the widely used group independent component analysis (ICA) approach. However, the effectiveness of IVA algorithms in preserving variability between different groups of subjects has not been studied yet, although it is of great interest in most fMRI studies, especially for identifying biomarkers for diagnosis of mental disorders. In this paper, we introduce a methodology that uses graph-theoretical analysis and statistical analysis for assessing the ability of IVA algorithms to capture group variability. We generate multi-subject fMRI-like datasets with increasing spatial variability for a selected component between two groups and compare a robust IVA algorithm to group ICA approach. Our experimental results show that IVA can successfully preserve group variability, indicating its potential in extracting biomarkers across groups of subjects in fMRI analysis.

Independent component analysis (ICA) is a multivariate data analysis tool. The basic principle of ICA is the assumption of independency of the source data. On the separation of the data source, ICA algorithm searches for a demixing matrix that will maximize the independency. This searching process is mostly done in iterative way and involving high order statistics. This process is time consuming. For a certain application, such as speech, where the source signal has its power at the lower frequency, we can reduce the data length by removing the high frequency component. Wavelet decomposition is a popular method for this purpose. In this paper, we propose the data reduction using wavelet as a preprocessing of ICA to speed up the ICA computation. We investigate Haar, Daubechies 2, Daubechies 3, and Daubechies 4 wavelet as the wavelet analysis. We further investigate the computation time as the function of level of decomposition of the wavelet. In this study, we found that Haar wavelet at third level of decomposition gave the biggest advantage of computation speed, which is about 40-50%.

Independent vector analysis (IVA) can be viewed as an extension of independent component analysis (ICA) to multiple datasets. It exploits the statistical dependency between different datasets through mutual information. In the context of motor imagery classification based on electroencephalogram (EEG) signals for the brain-computer interface (BCI), several methods have been proposed to extract features efficiently, mainly based on common spatial patterns, filter banks, and deep learning. However, most methods use only one dataset at a time, which may not be sufficient for dealing with a multi-source retrieving problem in certain scenarios. From this perspective, this paper proposes an original approach for feature extraction through multiple datasets based on IVA to improve the classification of EEG-based motor imagery movements. The IVA components were used as features to classify imagined movements using consolidated classifiers (support vector machines and K-nearest neighbors) and deep classifiers (EEGNet and EEGInception). The results show an interesting performance concerning the clustering of MI-based BCI patients, and the proposed method reached an average accuracy of 86.7%.

During the last decade, distributed microphone arrays have been proposed in order to increase accuracy and spatial coverage of speaker localization systems operating in large and reverberant rooms. In principle, the framework provided by a distributed microphone network can also be applied effectively when using Blind Source Separation (BSS). Separation is commonly performed by processing the signals sampled at closely spaced microphones in a single adaptation step, for example by means of Independent Component Analysis (ICA). When the microphone spacing or the distance between source and microphones increase, the separation performance reduces due to spatial aliasing effects and to a reduced spatial coherence at microphones. In this paper we propose a new method, here referred to as Cooperative Wiener ICA (CW-ICA), which is able to apply BSS to signals acquired by a network of distributed microphone arrays. Different ICA adaptations are applied to the signals recorded by each array and are interconnected in order to constrain each adaptation to converge to a solution related to the same physical interpretation. A preliminary analysis on a network of two arrays shows that the proposed method can be applied successfully to source separation and localization tasks.