Student's t multichannel nonnegative matrix factorization for blind source separation

Joint-diagonalizability-constrained multichannel nonnegative matrix factorization based on time-variant multivariate complex sub-Gaussian distribution

This article describes a computationally-efficient blind source separation (BSS) method based on the independence, low-rankness, and directivity of the sources. A typical approach to BSS is unsupervised learning of a probabilistic model that consists of a source model representing the time-frequency structure of source images and a spatial model representing their inter-channel covariance structure. Building upon the low-rank source model based on nonnegative matrix factorization (NMF), which has been considered to be effective for inter-frequency source alignment, multichannel NMF (MNMF) assumes source images to follow multivariate complex Gaussian distributions with unconstrained full-rank spatial covariance matrices (SCMs). An effective way of reducing the computational cost and initialization sensitivity of MNMF is to restrict the degree of freedom of SCMs. While a variant of MNMF called independent low-rank matrix analysis (ILRMA) severely restricts SCMs to rank-1 matrices under an idealized condition that only directional and less-echoic sources exist, we restrict SCMs to jointly-diagonalizable yet full-rank matrices in a frequency-wise manner, resulting in FastMNMF1. To help inter-frequency source alignment, we then propose FastMNMF2 that shares the directional feature of each source over all frequency bins. To explicitly consider the directivity or diffuseness of each source, we also propose rank-constrained FastMNMF that enables us to individually specify the ranks of SCMs. Our experiments showed the superiority of FastMNMF over MNMF and ILRMA in speech separation and the effectiveness of the rank constraint in speech enhancement.

Spatial gradients in the primary geomagnetic fields directly contribute to both the amplitudes and phases of inter-station transfer functions (IS-TFs). This suggests that, for the analysis of subsurface resistivity structures, IS-TFs should be carefully treated by checking the establishment of the plane-wave assumption. Geomagnetic time-series data include various and complicated characteristics and accordingly, time–frequency domain analysis is suitable for the discussion of spatial gradients of time-varying geomagnetic fields. However, such evaluations are complicated by the huge amount of information contained in the spectrograms from several stations. Therefore, we propose a Multi-Channel Nonnegative Matrix Factorization (MC-NMF) method that can decompose raw spectrograms into several components, allowing the spatial gradient of each geomagnetic temporal variation to be identified. We confirm that such components actually affect the estimation of IS-TFs using data acquired at the Kakioka and Memambetsu magnetic observatories in Japan. We derive the year-to-year changes in IS-TFs from each set of paired stations among Kakioka, Kanoya, and Memambetsu observatories. Although the IS-TFs should exhibit opposite polarities (a negative correlation) when the input and output observatories are swapped; surprisingly, some of them have “identical” polarities. The application of MC-NMF shows that the analyzed geomagnetic data include several components that have various spatial gradients. Although IS-TFs sometimes fail to give the expected implication regarding the spatial gradients of geomagnetic temporal variations, MC-NMF can verify whether the IS-TFs exhibit any spatial gradients. Thus, the use of IS-TFs with MC-NMF can yield better implications regarding subsurface resistivity information.

This paper presents a robust variant of nonnegative matrix factorization (NMF) based on complex Student's t distributions (t-NMF) for source separation of single-channel audio signals. The Itakura-Saito divergence NMF (Gaussian NMF) is justified for this purpose under an assumption that the complex spectra of source signals and those of the mixture signal are complex Gaussian distributed (the additiv-ity of power spectra holds). In fact, however, the source spectra are often heavy-tailed distributed. When the source spectra are complex Cauchy distributed, for example, the mixture spectra are also complex Cauchy distributed (the additivity of amplitude spectra holds). Using the complex t distribution that includes the complex Gaussian and Cauchy distributions as its special cases, we propose t-NMF as a unified extension of Gaussian NMF and Cauchy NMF. Furthermore, we propose the corresponding variant of positive semidefinite tensor factorization based on multivariate complex t distributions (t-PSDTF). The experimental results showed that while t-NMF and t-PSDTF were comparative to Gaussian counterparts in terms of peak performance, they worked much better on average because they are insensitive to initialization and tend to avoid local optima.

This paper proposes visually-informed sound source separation for audio-visual understanding of indoor scenes captured by distributed microphone arrays and cameras. Our approach leverages the 3D information of sound-emitting objects, reconstructed via 3D Gaussian splatting (3DGS), to overcome a limitation of modern blind source separation methods like multichannel nonnegative matrix factorization (MNMF). While adaptable and potentially performant, the iterative optimization of MNMF often converges to poor local minima due to the highly-expressive full-rank spatial covariance matrices (SCMs) of sources. Our key idea is to treat the set of 3D Gaussians representing a sizable sound source object as a collection of sub-sources that share an audio signal but have unique emission weights, both of which are to be estimated jointly from an observed mixture. To enforce this structure, we guide MNMF by regularizing the SCM of each source object at each frequency. Specifically, we use a prior that centers the SCM estimate around a weighted sum of theoretical SCMs, which are analytically derived from the 3D Gaussian positions. Experiments with simulated data, featuring two 3D human models, demonstrated the effectiveness of the proposed method. To our knowledge, this is the first work to use 3D Gaussians as a common primitive for joint audio-visual analysis.

This paper describes an ultimate covariance-aware multichannel extension of nonnegative matrix factorization (NMF) for blind source separation (BSS). A typical approach to BSS is to integrate a low-rank source model with a full-rank spatial model as multichannel NMF (MNMF) based on full-rank spatial covariance matrices (CMs) or its efficient version named FastMNMF based on jointly-diagonalizable spatial CMs do. The NMF-based phase-unaware source model, however, can deal with only the positive cooccurrence relations between time-frequency bins. To overcome this limitation, we propose an efficient multichannel extension of correlated tensor factorization (CTF) named FastMCTF based on jointly-diagonalizable temporal, frequency, and spatial CMs. Integration of the jointly-diagonalizable full-rank source model proposed by FastCTF with the jointly-diagonalizable full-rank spatial model proposed by FastMNMF enables us to completely consider the positive and negative covariance relations between frequency bins, time frames, and channels. We derive a convergence-guaranteed parameter estimation algorithm based on the multiplicative update and iterative projection and experimentally show the potential of the proposed method.

To improve the performance of blind audio source separation of convolutive mixtures, the local Gaussian model (LGM) having full rank covariance matrices proposed by Duong et al. is extended. The previous model basically assumes that all sources contribute to each time-frequency slot, which may fail to capture the characteristic of signals with many intermittent silent periods. A constraint on source sets that contribute to each time-frequency slot is therefore explicitly introduced. This approach can be regarded as a relaxation of the sparsity constraint in the conventional time-frequency mask. The proposed model is jointly optimized among the original local Gaussian model parameters, the relaxed version of the time-frequency mask, and a permutation alignment, leading to a robust permutation-free algorithm. We also present a novel multi-channel Wiener filter weighted by a relaxed version of the time-frequency mask. Experimental results over noisy speech signals show that the proposed model is effective compared with the original local Gaussian model and is comparable to its extension, the multi-channel nonnegative matrix factorization.

In this letter, we develop a multichannel blind source separation algorithm based on a beamspace transform and the multichannel nonnegative matrix factorization (NMF) method. The conventional multichannel NMF algorithm performs well with multichannel mixing data, but there is still room for enhancement in multichannel real-world recording data. In this letter, we consider a beamspace-time-frequency domain data model for multichannel NMF method, and enhance the conventional method using a beamspace transform. Our decomposition algorithm is applied to 2-channel and 4-channel unsupervised audio source separation, using a dataset from the international Signal Separation Evaluation Campaign 2010 (SiSEC 2010). Our algorithm shows a better performance than the conventional NMF method in an evaluation results.

We consider inference in a general data-driven object-based model of multichannel audio data, assumed generated as a possibly underdetermined convolutive mixture of source signals. We work in the short-time Fourier transform (STFT) domain, where convolution is routinely approximated as linear instantaneous mixing in each frequency band. Each source STFT is given a model inspired from nonnegative matrix factorization (NMF) with the Itakura-Saito divergence, which underlies a statistical model of superimposed Gaussian components. We address estimation of the mixing and source parameters using two methods. The first one consists of maximizing the exact joint likelihood of the multichannel data using an expectation-maximization (EM) algorithm. The second method consists of maximizing the sum of individual likelihoods of all channels using a multiplicative update algorithm inspired from NMF methodology. Our decomposition algorithms are applied to stereo audio source separation in various settings, covering blind and supervised separation, music and speech sources, synthetic instantaneous and convolutive mixtures, as well as professionally produced music recordings. Our EM method produces competitive results with respect to state-of-the-art as illustrated on two tasks from the international Signal Separation Evaluation Campaign (SiSEC 2008).

Underdetermined blind source separation using normalized spatial covariance matrix and multichannel nonnegative matrix factorization

A probability model with Variational Bayesian Inference for the complex interference suppression in the acoustic array measurement

Changes in the concentration of free amino acids in biological tissues is a sign of impaired protein metabolism in patients with cancer. Recently, Raman spectroscopy has been used for early diagnostics of oncological diseases. The concentrations of individual components of biological tissue (for instance, the concentrations of amino acids) can be obtained by decomposing the tissue Raman spectrum. This study was designed to evaluate the effect of noise in the Raman spectra of individual amino acids on the result of the decomposition of the spectra of an amino acid mixture. As a decomposition method, we used Multivariate Curve Resolution-Alternating Least Squares (MCR–ALS) analysis and investigate experimental Raman spectra of amino acids and mathematically simulated Raman spectra of amino acid mixtures. Noise with different signal-to-noise ratios (SNR) was artificially added to both the experimental spectra of pure amino acids and the spectra of the mixtures. Concentration values for each amino acid obtained as a result of applying the MCR–ALS analysis have been compared with the corresponding true values and the correlation coefficients have been calculated. The results show a less pronounced negative effect of noise in the case when the spectra of pure amino acids (which were used as a basis for the MCR–ALS analysis) are noisy, and a more pronounced negative effect when the spectrum of the mixture is noisy. The accuracy of reconstruction of an amino acid is also negatively affected by strong background fluorescence in the amino acid spectrum. Moreover, the results indicate that using the basis spectra with a high SNR (SNR = 5) makes it possible to successfully estimate the amino acid concentrations in a mixture even when the Raman spectrum of the mixture is noisy and has a low SNR (SNR < 5).

Microphone array signal processing techniques are extensively used for sound source localisation, acoustical characterisation and sound source separation, which are related to audio analysis. However, the use of microphone arrays for auralisation, which is generally related to synthesis, has been limited so far. This paper proposes a method for binaural auralisation of multiple sound sources based on blind source separation (BSS) and binaural audio synthesis. A BSS algorithm is introduced that exploits the intensity vector directions in order to generate directional signals. The directional signals are then used in the synthesis of binaural recordings using head related transfer functions. The synthesised recordings subsume the indirect information about the auditory environment conveying the source positions and the acoustics similar to dummy head recordings. Test recordings were made with a compact microphone array and a dummy head in different indoor environments. Subjective listening tests were performed to compare the synthesized recordings with the actual ones for their performance in providing presence, spaciousness and envelopment.

Nonnegative matrix factorization (NMF) is often used for source separation of audio signals. In most of these algorithms, the initialization step of the NMF, which has a strong impact on the separation performance, is based on random values or deterministic methods such as singular value decomposition (SVD). Another deterministic initialization approach, which is used e.g. for score-informed source separation algorithms, makes use of synthesized magnitude spectra of harmonic notes. It was shown that this semantic method leads to good separation results in blind source separation (BSS) as well; not only for harmonic but also for percussive mixtures with some harmonic components. In this paper, we present an extension to the semantic approach to enhance the separation quality for arbitrary audio mixtures. We evaluate this extension in a BSS scenario and compare it to other initialization schemes.

In this work we proposed a new method that allows the blind source separation by the analysis of independent components known as FASTICA in the domain of Wavelet to observe his behavior on signs captured in a real environment. The problem that tries to be solved in Blind Source Separation (BSS) consists of recovering signs statistically independent. Nevertheless, certain difficulties appear when this system is applied to real signs, on the one hand the effect of the reverberation does that the mixtures gathered by the microphones are convolution mix; and on the other hand, these mixtures will not be totally independent. We did two experiments. With the first experiment we separated 2 audio signals with a very low percentage of error. With the second experiment we recorded 3 different audio sources with an array of 3 microphones, and then from one audio recorded source 3 signals were separated, we appreciate that in each source one signal was amplified and the other two signals were fallen down. From the results, the method that we proposed is able to separate from one mixed audio signal 2 or even 3 independent signals.