Complex-valued independent vector analysis: Application to multivariate Gaussian model

Independent vector analysis (IVA) is a special form of independent component analysis (ICA), which has demonstrated its prominent performance in solving convolutive blind source separation (BSS) problems in the frequency domain. Most IVA algorithms are based on optimizing certain contrast functions, where the main difficulty of these approaches lies in finding a reliable and fast estimation of the unknown distribution of sources. Despite the rich availability of efficient tensorial approaches to the standard ICA problem, these methods have not been explored considerably for IVA. In this article, we propose a matrix joint diagonalization approach to solve the complex IVA problem. The new factorization neither relies on a whitening process, nor does it require an estimate of the joint probability distribution of the dependent signal groups. The latter is in contrast to most IVA approaches up to date. The underlying geometry of the problem is investigated together with a critical point analysis of the resulting cost function. A conjugate gradient algorithm on the appropriate manifold setting is developed.

Independent vector analysis (IVA) is an attractive solution to address the problem of joint blind source separation (JBSS), that is, the simultaneous extraction of latent sources from several datasets implicitly sharing some information. Among IVA approaches, we focus here on the celebrated IVA-G model, that describes observed data through the mixing of independent Gaussian source vectors across the datasets. IVA-G algorithms usually seek the values of demixing matrices that maximize the joint likelihood of the datasets, estimating the sources using these demixing matrices. Instead, we write the likelihood of the data with respect to both the demixing matrices and the precision matrices of the source estimates. This allows us to formulate a cost function whose mathematical properties enable the use of a proximal alternating algorithm based on closed form operators with provable convergence to a critical point. After establishing the convergence properties of the new algorithm, we illustrate its desirable performance in separating sources with covariance structures that represent varying degrees of difficulty for JBSS.

A new independent component analysis (ICA) formulation called independent vector analysis (IVA) was proposed in order to solve the permutation problem in convolutive blind source separation (BSS). Instead of running ICA in each frequency bin separately and correcting the disorder with an additional algorithmic scheme afterwards, IVA exploited the dependency among the frequency components of a source and dealt with them as a multivariate source by modeling it with sparse and spherically, or radially, symmetric joint probability density functions (pdfs). In this paper, we compare the speech separation performances of IVA by using a group of l <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sup> -norm-invariant sparse pdfs where the value of and the sparseness can be controlled. Also, we derive an IVA algorithm from a nonparametric perspective with the constraint of spherical symmetry and high dimensionality. Simulation results confirm the efficiency of assuming sparseness and spherical symmetry for the speech model in the frequency domain.

Fast fixed-point independent vector analysis algorithms for convolutive blind source separation

In this paper, we present a quasi-Newton (QN) algorithm for joint independent subspace analysis (JISA). JISA is a recently proposed generalization of independent vector analysis (IVA). JISA extends classical blind source separation (BSS) to jointly resolve several BSS problems by exploiting statistical dependence between latent sources across mixtures, as well as relaxing the assumption of statistical independence within each mixture. Algebraically, JISA based on second-order statistics amounts to coupled block diagonalization of a set of covariance and cross-covariance matrices, as well as block diagonalization of a single permuted covariance matrix. The proposed QN algorithm achieves asymptotically the minimal mean square error (MMSE) in the separation of multidimensional Gaussian components. Numerical experiments demonstrate convergence and source separation properties of the proposed algorithm.KeywordsBlind source separationIndependent vector analysisIndependent subspace analysisJoint block diagonalization

A novel method is proposed to improve the performance of independent vector analysis (IVA) for blind signal separation of acoustic mixtures. IVA is a frequency-domain approach that successfully resolves the well-known permutation problem by applying a spherical dependency model to all pairs of frequency bins. The dependency model of IVA is equivalent to a single clique in an undirected graph; a clique in graph theory is defined as a subset of vertices in which any pair of vertices is connected by an undirected edge. Therefore, IVA imposes the same amount of statistical dependency on every pair of frequency bins, which may not match the characteristics of real-world signals. The proposed method allows variable amounts of statistical dependencies according to the correlation coefficients observed in real acoustic signals and, hence, enables more accurate modeling of statistical dependencies. A number of cliques constitutes the new dependency graph so that neighboring frequency bins are assigned to the same clique, while distant bins are assigned to different cliques. The permutation ambiguity is resolved by overlapped frequency bins between neighboring cliques. For speech signals, we observed especially strong correlations across neighboring frequency bins and a decrease in these correlations with an increase in the distance between bins. The clique sizes are either fixed, or determined by the reciprocal of the mel-frequency scale to impose a wider dependency on low-frequency components. Experimental results showed improved performances over conventional IVA. The signal-to-interference ratio improved from 15.5 to 18.8 dB on average for seven different source locations. When we varied the clique sizes according to the observed correlations, the stability of the proposed method increased with a large number of cliques.

Speeches in real world environment are normally mixed together convolutedly. The speech mixtures are not instantaneous at all. Rather speeches are mixed component wise. Independent Vector Analysis (IVA) is an approach for separating convolutive mixture in frequency domain. Entropy estimation is an important part of IVA. In this paper, IVA is implemented by estimating entropy using recursive copula splitting. It measures entropy by decomposing probability density function (PDF) into a product of marginal (1D) densities and a copula. This entropy estimator improves IVA performance in different types of real world speech mixtures. We have proved that by estimating entropy using recursive copula splitting makes IVA algorithm simple and more efficient.'

The noncircular fast independent vector analysis (nc-FastIVA) algorithm is an effective approach to achieve the joint blind source separation (JBSS) problem. However, the original nc-FastIVA algorithm does not take noise into account. In this letter, a new fixed-point algorithm is proposed for more realistic cases when noise is present. Simulations show that the proposed method can have better performance than the original nc-FastIVA algorithm for solving the noisy JBSS problem.

The independent vector analysis (IVA) algorithm employs a multivariate source prior to retain the dependency between different frequency bins of each source and thereby avoids the permutation problem that is inherent to blind source separation (BSS). In this paper, a multivariate Student's t distribution is adopted as the source prior, which because of its heavy tail nature can better model the large amplitude information in the frequency bins. Therefore it can improve the separation performance and the convergence speed of the IVA and fast version of the IVA (FastIVA) algorithms as compared with the IVA algorithm based on another multivariate super Gaussian source prior. Separation performance with real binaural room impulse responses (BRIRs) is evaluated by detailed simulation studies when using the different source priors, and the experimental results confirm that the IVA and the FastIVA with the proposed multivariate Student's t source prior can consistently achieve improved and faster separation performance.

Independent vector analysis (IVA) minimizes an objective function to estimate separation filters that separate mixture signals into individual source signals. Unfortunately, IVA often suffers from the well-known block permutation problem. To mitigate that problem, the use of geometry knowledge has been studied, but two crucial issues remain: the necessity of non-differential processes outside the minimization and of high-level geometrical clues such as the directions of arrival (DOAs) of the source signals. This paper thus presents a novel IVA method whose objective function has a differentiable max-directivity beamforming normalization (MDBN) term. This term uses geometry knowledge from only a low-level geometrical clue (the positions of the microphone array) via the traditional beamforming (BF) concept that each separation filter should have a maximum gain for each specific DOA across all frequency bins. Thus, our overall objective function can be minimized by gradient descent, and the MDBN term encourages the separation filters to focus on specific directions, which implicitly estimates the most reasonable DOAs of the source signals at each iteration. Therefore, our method uses geometry knowledge while avoiding the above two issues and estimates good separation filters mitigating the permutation problem. Several experiments show that our method outperforms the conventional BF and IVA methods.

Analysis of time-evolving data is crucial to understand the functioning of dynamic systems such as the brain. For instance, analysis of functional magnetic resonance imaging (fMRI) data collected during a task may reveal spatial regions of interest, and how they evolve during the task. However, capturing underlying spatial patterns as well as their change in time is challenging. The traditional approach in fMRI data analysis is to assume that underlying spatial regions of interest are static. In this article, using fractional amplitude of low-frequency fluctuations (fALFF) as an effective way to summarize the variability in fMRI data collected during a task, we arrange time-evolving fMRI data as a subjects by voxels by time windows tensor, and analyze the tensor using a tensor factorization-based approach called a PARAFAC2 model to reveal spatial dynamics. The PARAFAC2 model jointly analyzes data from multiple time windows revealing subject-mode patterns, evolving spatial regions (also referred to as networks) and temporal patterns. We compare the PARAFAC2 model with matrix factorization-based approaches relying on independent components, namely, joint independent component analysis (ICA) and independent vector analysis (IVA), commonly used in neuroimaging data analysis. We assess the performance of the methods in terms of capturing evolving networks through extensive numerical experiments demonstrating their modeling assumptions. In particular, we show that (i) PARAFAC2 provides a compact representation in all modes, i.e., subjects, time, and voxels, revealing temporal patterns as well as evolving spatial networks, (ii) joint ICA is as effective as PARAFAC2 in terms of revealing evolving networks but does not reveal temporal patterns, (iii) IVA's performance depends on sample size, data distribution and covariance structure of underlying networks. When these assumptions are satisfied, IVA is as accurate as the other methods, (iv) when subject-mode patterns differ from one time window to another, IVA is the most accurate. Furthermore, we analyze real fMRI data collected during a sensory motor task, and demonstrate that a component indicating statistically significant group difference between patients with schizophrenia and healthy controls is captured, which includes primary and secondary motor regions, cerebellum, and temporal lobe, revealing a meaningful spatial map and its temporal change.

Independent vector analysis followed by HMM-based feature enhancement for robust speech recognition

When applied to functional magnetic resonance imaging (fMRI) data, independent vector analysis (IVA) provides superior performance in capturing subject variability within one group, as compared to the widely used group independent component analysis (ICA) approach. However, the effectiveness of IVA algorithms in preserving variability between different groups of subjects has not been studied yet, although it is of great interest in most fMRI studies, especially for identifying biomarkers for diagnosis of mental disorders. In this paper, we introduce a methodology that uses graph-theoretical analysis and statistical analysis for assessing the ability of IVA algorithms to capture group variability. We generate multi-subject fMRI-like datasets with increasing spatial variability for a selected component between two groups and compare a robust IVA algorithm to group ICA approach. Our experimental results show that IVA can successfully preserve group variability, indicating its potential in extracting biomarkers across groups of subjects in fMRI analysis.

Fast fixed-point independent vector analysis (FastIVA) is an improved independent vector analysis (IVA) method, which can achieve faster and better separation performance than original IVA. As an example IVA method, it is designed to solve the permutation problem in frequency domain independent component analysis by retaining the higher order statistical dependency between frequencies during learning. However, the performance of all IVA methods is limited due to the dimensionality of the parameter space commonly encountered in practical frequency-domain source separation problems and the spherical symmetry assumed with the source model. In this article, a particular permutation problem encountered in using the FastIVA algorithm is highlighted, namely the block permutation problem. Therefore a new audio video based fast fixed-point independent vector analysis algorithm is proposed, which uses video information to provide a smart initialization for the optimization problem. The method cannot only avoid the ill convergence resulting from the block permutation problem but also improve the separation performance even in noisy and high reverberant environments. Different multisource datasets including the real audio video corpus AV16.3 are used to verify the proposed method. For the evaluation of the separation performance on real room recordings, a new pitch based evaluation criterion is also proposed.

The issue of convolutive blind source separation (BSS) is addressed in this paper. Independent low-rank matrix analysis (ILRMA), unifying frequency-domain independent component analysis (FDICA) and nonnegative matrix factorization (NMF), is a method that has recently proposed to model low-rank structure of source spectra by using NMF in addition to independence between sources used in FDICA and independent vector analysis (IVA). Although ILRMA has been shown to provide better separation performance than FDICA and IVA, the frequency components of each source are assumed to be independent in ILRMA due to NMF modeling of source spectra, which may degrade its performance when the short-term Fourier transform (STFT) is unable to decorrelate the frequency components for each source. This paper therefore presents a new BSS method that unifies IVA and positive semidefinite tensor factorization (PSDTF). PSDTF models not only power spectra in the same way NMF does but also models the correlations between frequency bins in each source. The proposed method can be viewed as a multichannel extension of PSDTF and exploits both the independence between sources and the inter-frequency correlations as a clue for separating mixtures. Experimental results indicate the improved performance of our approach.