Extended semantic initialization for NMF-based audio source separation

This paper presents a robust variant of nonnegative matrix factorization (NMF) based on complex Student's t distributions (t-NMF) for source separation of single-channel audio signals. The Itakura-Saito divergence NMF (Gaussian NMF) is justified for this purpose under an assumption that the complex spectra of source signals and those of the mixture signal are complex Gaussian distributed (the additiv-ity of power spectra holds). In fact, however, the source spectra are often heavy-tailed distributed. When the source spectra are complex Cauchy distributed, for example, the mixture spectra are also complex Cauchy distributed (the additivity of amplitude spectra holds). Using the complex t distribution that includes the complex Gaussian and Cauchy distributions as its special cases, we propose t-NMF as a unified extension of Gaussian NMF and Cauchy NMF. Furthermore, we propose the corresponding variant of positive semidefinite tensor factorization based on multivariate complex t distributions (t-PSDTF). The experimental results showed that while t-NMF and t-PSDTF were comparative to Gaussian counterparts in terms of peak performance, they worked much better on average because they are insensitive to initialization and tend to avoid local optima.

Underdetermined blind separation of nonnegative dependent sources consists in decomposing a set of observed mixed signals into greater number of original nonnegative and dependent component (source) signals. That is an important problem for which very few algorithms exist. It is also practically relevant for contemporary metabolic profiling of biological samples, such as biomarker identification studies, where sources (a.k.a. pure components or analytes) are aimed to be extracted from mass spectra of complex multicomponent mixtures. This paper presents a method for underdetermined blind separation of nonnegative dependent sources. The method performs nonlinear mixture‐wise mapping of observed data in high‐dimensional reproducible kernel Hilbert space (RKHS) of functions and sparseness‐constrained nonnegative matrix factorization (NMF) therein. Thus, the original problem is converted into new one with increased number of mixtures, increased number of dependent sources, and higher‐order (error) terms generated by nonlinear mapping. Provided that amplitudes of original components are sparsely distributed, which is the case for mass spectra of analytes, sparseness‐constrained NMF in RKHS yields, with significant probability, improved accuracy relative to the case when the same NMF algorithm is performed on the original problem. The method is exemplified on numerical and experimental examples related respectively to extraction of 10 dependent components from five mixtures and to extraction of 10 dependent analytes from mass spectra of two to five mixtures. Thereby, analytes mimic complexity of components expected to be found in biological samples. Copyright © 2013 John Wiley & Sons, Ltd.

The fields of video games, simulations and virtual reality are now tending to develop increasingly high-performance, realistic and immersive technologies. Efforts are made in terms of sound devices and sound processing to synthesize realistic sound scenes in a 3-D environment. One of the greatest challenges is the ability to analyze a 3-D audio stream corresponding to a complex sound scene in its basic components (i.e. individual sound sources), to modify the scene (e.g. to change the locations of sound sources) and to resynthesize a modified 3-D audio stream. This situation is referred to as "spatial remix".Actually, the spatial remix problem is still an open field. Work in progress rely on sound separation algorithms to analyze a sound scene, but these techniques are not perfect and can damage the reconstructed source signals. These are referred to as "separation artefacts", including transient alteration of the target source and rejections of other sources into the target source. Objective and subjective evaluation of separation artefacts have been conducted [1], but these studies usually consider the separated source signals alone, i.e. when each source is listened to separately. This is different form the spatial remix problem, where all sources are listened to simultaneously.In that case, one may wonder if the separation artefacts can affect the spatial image of the synthesized 3-D sound scene. According to the perceptual mechanisms involved in spatial hearing, hypothesis can be made on the kind of spatial distortions that could occur in this context. Indeed, as transients are important cues to precisely localize sounds sources, its alteration may result in a localization blur or source widening. On the other hand, when separated sources are spatialized and played simultaneously, rejections of one source into another may also produce unwanted effects such as a feeling of moving sources and "phantom" sources emergence. This paper presents a new methodology to perceptually evaluate the spatial distortions that can occur in a spatial remix context. It consists in carrying out a localization test on complex scenes composed of three synthetic musical instruments played on a set of loudspeakers. In order to eliminate possible issues related to the spatial audio rendering device, we consider a simple case: We consider only three spatial positions, each corresponding to a single loudspeaker. Then, the spatial remix is restrained to a simple permutation of the source locations.The test is run through a virtual interface, using a head mounted display. The subject is placed in a simple visual virtual environment and is asked to surround with a remote the areas where each instrument is perceived. This experimental device allows the subject to report precisely both instruments position and size. A single instrument can also be spotted at multiple locations. Perceived source positions are approximated as ellipses from which center position and dimensions can easily be deduced. In order to quantify spatial distortions, the localization task is performed on both clean and degraded versions of the same musical extract. Localization performances in both cases are then compared taking the clean sources case as a reference. In this paper, the methodology is applied to assess the quality of Non-Negative Matrix Factorization source separation algorithm developped by Leglaive [2] which performs separation on convolutive mixtures.Our study reveals that the source separation process leads to perceptible degradations of the spatial image. Three main kinds of spatial distortions have been characterized. First, in the majority of degraded cases, "phantom" sources have been observed. This artifact mainly concerns percussive sources. The results also show a significant increase in the perceived width of the degraded sources. Finally, azimuth and elevation localization error is significantly higher in the case of scenes composed of separated sources.[1] V. Emiya, E. Vincent, N. Harlander and V. Hohmann, "Subjective and Objective Quality Assessment of Audio Source Separation," in <i>IEEE Transactions on Audio, Speech, and Language Processing</i>, vol. 19, no. 7, pp. 2046-2057, Sept. 2011.[2] S. Leglaive, R. Badeau and G. Richard, "Separating time-frequency sources from time-domain convolutive mixtures using non-negative matrix factorization," <i>2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)</i>, New Paltz, NY, 2017, pp. 264-268.

The methods that can be done for the estimation of the number of source signals and separation of source signals using time‐frequency information are proposed as a method of blind source separation. In these methods, some independent conditions are assumed in a time‐frequency domain, and observed signals are expanded to time‐frequency domain using wavelet analysis. However, these methods are not considered phase differences (difference of distance) between source signals and observed signals. A new method is proposed. The method can be done not only for the source separation but also the specification of the locations of source signals considering phase differences. In this method, it is necessary to assume a rather stronger independent condition than former methods. But, on account of this assumption, a function that becomes a real value only when phase differences are coincidence in time‐frequency domain is determined. Using this function, first of all, the number of source signals is calculated, then the locations of source signals and source signals are calculated. The effectiveness of the method is shown using actual synthetic signals.

One successful approach for audio source separation involves applying nonnegative matrix factorization (NMF) to a magnitude spectrogram regarded as a nonnegative matrix. This can be interpreted as approximating the observed spectra at each time frame as the linear sum of the basis spectra scaled by time-varying amplitudes. This paper deals with the problem of the unsupervised instrument-wise source separation of polyphonic signals based on an extension of the NMF approach. We focus on the fact that each piece of music is typically played on a handful of musical instruments, which allows us to assume that the spectra of the underlying audio events in a polyphonic signal can be grouped into a reasonably small number of clusters in the mel-frequency cepstral coefficient (MFCC) domain. Based on this assumption, we propose formulating factorization of a magnitude spectrogram and clustering of the basis spectra in the MFCC domain as a joint optimization problem and derive a novel optimization algorithm based on the majorization–minimization principle. Experimental results revealed that our method was superior to a two-stage algorithm that consists of performing factorization followed by clustering the basis spectra, thus showing the advantage of the joint optimization approach.

Blind single-channel source separation is a long-standing machine learning and signal processing problem. Traditional blind source separation (BSS) algorithms were proposed to solve this task utilizing multiple signal constraints. Generative adversarial network (GAN) are free from statistical constraints and samples, but the role of adversarial training in the task of BSS has not been fully demonstrated. Therefore, a new separation network model that enables to learn the known separated signal distribution from the stepwise fine estimation of the unknown mixture distribution was presented in this paper, and a self-attention mechanism was introduced to solve the problem of the blurring details of the generated image by the generator which preserves image details in the process of image separation. Compared with the existing single-channel blind source separation algorithm based on generative adversarial network-neural egg separation (NES), the detailed information of this new separation algorithm is more prominent, and the source signal in the mixed image has been separated more effectively, and has better separation performance than the classic blind source separation algorithms.

Most of the existing algorithms for blind sources separation have a limitation that sources are statistically independent. However, in many practical applications, the source signals are nonnegative and mutual statistically dependent signals. When the observations are nonnegative linear combinations of nonnegative sources, the correlation coefficients of the observations are larger than these of source signals. In this letter, a novel Nonnegative Matrix Factorization (NMF) algorithm with least correlated component constraints to blind separation of convolutive mixed sources is proposed. The algorithm relaxes the source independence assumption and has low-complexity algebraic computations. Simulation results on blind source separation including real face image data indicate that the sources can be successfully recovered with the algorithm.

This paper presents a new low-latency online blind source separation (BSS) algorithm. Although algorithmic delay of a frequency domain online BSS can be reduced simply by shortening the short-time Fourier transform (STFT) frame length, it degrades the source separation performance in the presence of reverberation. This paper proposes a method to solve this problem by integrating BSS with Weighted Prediction Error (WPE) based dereverberation. Although a simple cascade of online BSS after online WPE upgrades the separation performance, the overall optimality is not guaranteed. Instead, this paper extends a recently proposed batch processing algorithm that can jointly optimize dereverberation and separation so that it can perform online processing with low computational cost and little processing delay (< 12 ms). The results of a source separation experiment in a noisy car environment suggest that the proposed online method has better separation performance than the simple cascaded methods.

The issue of convolutive blind source separation (BSS) is addressed in this paper. Independent low-rank matrix analysis (ILRMA), unifying frequency-domain independent component analysis (FDICA) and nonnegative matrix factorization (NMF), is a method that has recently proposed to model low-rank structure of source spectra by using NMF in addition to independence between sources used in FDICA and independent vector analysis (IVA). Although ILRMA has been shown to provide better separation performance than FDICA and IVA, the frequency components of each source are assumed to be independent in ILRMA due to NMF modeling of source spectra, which may degrade its performance when the short-term Fourier transform (STFT) is unable to decorrelate the frequency components for each source. This paper therefore presents a new BSS method that unifies IVA and positive semidefinite tensor factorization (PSDTF). PSDTF models not only power spectra in the same way NMF does but also models the correlations between frequency bins in each source. The proposed method can be viewed as a multichannel extension of PSDTF and exploits both the independence between sources and the inter-frequency correlations as a clue for separating mixtures. Experimental results indicate the improved performance of our approach.

In the recent past, blind source separation (BSS) algorithms using multivariate statistical data analysis technique have been successfully used for source identification and separation in the field of biomedical and statistical signal processing. Recently numbers of different BSS techniques have been developed. With BSS methods being the feasible method for source separation and decomposition of biosignals, it is important to compare the different techniques and determine the most suitable method for the applications. This paper presents the performance of five BSS algorithms (SOBI, TDSEP, FastICA, JADE and Infomax) for decomposition of sEMG to identify subtle finger movements. It is observed that BSS algorithms based on second-order statistics (SOBI and TDSEP) gives better performance compared to algorithms based on higher-order statistics (FastICA, JADE and infomax).

In this paper we propose a new method for removing specular reflection from multispectral dermatological images which is based on blind source separation using Non-negative Matrix Factorization. The key idea of our method is based on a first step which consists to estimate the number of sources involved, by applying a Principal Component Analysis, then a first version of these sources by applying an Independent Component Analysis to our images. We exploit this first version of the sources in a second step to initialize our Non-negative Matrix Factorization algorithm instead of the random initialization that is used by most of the existing methods and that considerably affects their performances. In order to quantify numerically the performance of our method, we also propose a new protocol to artificially mix a specular reflection image with a diffuse reflection image. The tests effected on real and artificial multispectral dermatological images have shown the good performance of our method compared to two of the most used methods.KeywordsMultispectral dermatological imagesRGB imagesSpecular reflectionDiffuse reflectionBlind Source Separation (BSS)Principal Component Analysis (PCA)Independent Component Analysis (ICA)Non-negative Matrix Factorization (NMF)

Blind separation of non-stationary sources using continuous density hidden Markov models

In this paper, we present RT-GCC-NMF: a real-time (RT), two-channel blind speech enhancement algorithm that combines the non-negative matrix factorization (NMF) dictionary learning algorithm with the generalized cross-correlation (GCC) spatial localization method. Using a pre-learned universal NMF dictionary, RT-GCC-NMF operates in a frame-by-frame fashion by associating individual dictionary atoms to target speech or background interference based on their estimated time-delay of arrivals. We evaluate RT-GCC-NMF on two-channel mixtures of speech and real-world noise from the signal separation and evaluation campaign (SiSEC). We demonstrate that this approach generalizes to new speakers, acoustic environments, and recording setups from very little training data, and outperforms all but one of the algorithms from the SiSEC challenge in terms of overall perceptual evaluation methods for audio source separation (PEASS) scores and compares favourably to the ideal binary mask baseline. Over a wide range of input signal to noise ratios (SNRs), we show that this approach simultaneously improves the PEASS and SNR-based blind source separation eval objective quality metrics as well as the short-time objective intelligibility (STOI) and extended STOI objective speech intelligibility metrics. A flexible, soft masking function in the space of NMF activation coefficients offers real-time control of the tradeoff between interference suppression and target speaker fidelity. Finally, we use an asymmetric short-time Fourier transform to reduce the inherent algorithmic latency of RT-GCC-NMF from 64 ms to 2 ms with no loss in performance. We demonstrate that latencies within the tolerable range for hearing aids are possible on current hardware platforms.

During the last decade, distributed microphone arrays have been proposed in order to increase accuracy and spatial coverage of speaker localization systems operating in large and reverberant rooms. In principle, the framework provided by a distributed microphone network can also be applied effectively when using Blind Source Separation (BSS). Separation is commonly performed by processing the signals sampled at closely spaced microphones in a single adaptation step, for example by means of Independent Component Analysis (ICA). When the microphone spacing or the distance between source and microphones increase, the separation performance reduces due to spatial aliasing effects and to a reduced spatial coherence at microphones. In this paper we propose a new method, here referred to as Cooperative Wiener ICA (CW-ICA), which is able to apply BSS to signals acquired by a network of distributed microphone arrays. Different ICA adaptations are applied to the signals recorded by each array and are interconnected in order to constrain each adaptation to converge to a solution related to the same physical interpretation. A preliminary analysis on a network of two arrays shows that the proposed method can be applied successfully to source separation and localization tasks.

Fingerprint terahertz spectroscopy combined with machine learning for multicomponent mixture analysis