Nonnegative Matrix Factorization With Basis Clustering Using Cepstral Distance Regularization

Non-negative matrix factorization (NMF) is a powerful approach to single channel audio source separation. In a supervised setting, NMF is first applied to train the basis spectra of each sound source. At test time, NMF is applied to the spectrogram of a mixture signal using the pretrained spectra. The source signals can then be separated out using a Wiener filter. A typical way to train the basis spectra of each source is to minimize the objective function of NMF. However, the basis spectra obtained in this way do not ensure that the separated signal will be optimal at test time due to the inconsistency between the objective functions for training and separation (Wiener filtering). To address this, a framework called discriminative NMF (DNMF) has recently been proposed. In in this work a multiplicative update algorithm was proposed for the basis training, however one drawback is that the convergence is not guaranteed. To overcome this drawback, this paper proposes using a majorization-minimization principle to develop a convergence-guaranteed algorithm for DNMF. Experimental results showed that the proposed algorithm outperformed standard NMF and DNMF using a multiplicative update algorithm as regards both the signal-to-distortion and signal-to-interference ratios.

Multiple pitch estimation consists of estimating the fundamental frequencies and saliences of pitched sounds over short time frames of an audio signal. This task forms the basis of several applications in the particular context of musical audio. One approach is to decompose the short-term magnitude spectrum of the signal into a sum of basis spectra representing individual pitches scaled by time-varying amplitudes, using algorithms such as nonnegative matrix factorization (NMF). Prior training of the basis spectra is often infeasible due to the wide range of possible musical instruments. Appropriate spectra must then be adaptively estimated from the data, which may result in limited performance due to overfitting issues. In this paper, we model each basis spectrum as a weighted sum of narrowband spectra representing a few adjacent harmonic partials, thus enforcing harmonicity and spectral smoothness while adapting the spectral envelope to each instrument. We derive a NMF-like algorithm to estimate the model parameters and evaluate it on a database of piano recordings, considering several choices for the narrowband spectra. The proposed algorithm performs similarly to supervised NMF using pre-trained piano spectra but improves pitch estimation performance by 6% to 10% compared to alternative unsupervised NMF algorithms.

This paper proposes a sound event detection system that is trained using a minimally annotated data set of single sounds to identify and separate components of polyphonic sounds. The system uses a Feed Forward Neural Network with a single hidden layer that is pre-trained using an autoencoder. Single sounds, represented as Mel Frequency Cepstral Coefficient (MFCC) feature vectors, are used to train the neural network using back propagation. Polyphonic sounds are pre-processed using Principal Component Analysis (PCA) and Nonnegative Matrix Factorization (NMF) to obtain source separated sounds. These source separated sounds are then tested for sound classification using the feed-forward algorithm. Our system is able to achieve reasonable accuracy of source separation and detection with minimal training set. The ultimate goal of our system is to bootstrap from minimal data and learn new sounds leading to a better sound detection system.

The mismatch between the training conditions and the test conditions severely degrades the performance of speaker verification. Aiming at solving this problem, this paper presents a method which uses a combination of improved nonnegative matrix factorization (IMNMF) and mel frequency cepstral coefficients (MFCC) with feature warping for improving identity-vector (i-vector) speaker verification performance in noisy environment. Unlike the traditional nonnegative matrix factorization (NMF), IMNMF uses extra free basis vectors to capture the features which are not included in training data, and linear constraints on dictionary atoms. Feature warping is used to remove channel noises. Therefore, the proposed method can reduce distortion of reconstructed speech while enhancing the recovered speech quality. The performance of i-vector speaker verification is evaluated by using the short utterance database and the NOISEX-92 database. The experiment results indicate that the score level fusion of feature-warped IMNMF-MFCC and feature-warped MFCC is superior to the baseline system at the equal error rate under the majority of signal-to-noise ratios (SNRs).

This paper presents a single-channel source separation method that extends the ideas of Nonnegative Matrix Factorization (NMF). We interpret the approach of audio demixing via NMF as a cascade of a pooled analysis operator, given for example by the magnitude spectrogram, and a synthesis operators given by the matrix decomposition. Instead of imposing the temporal consistency of the decomposition through sophisticated structured penalties in the synthesis stage, we propose to change the analysis operator for a deep scattering representation, where signals are represented at several time resolutions. This new signal representation is invariant to smooth changes in the signal, consistent with its temporal dynamics. We evaluate the proposed approach in a speech separation task obtaining promising results.

This paper presents a robust variant of nonnegative matrix factorization (NMF) based on complex Student's t distributions (t-NMF) for source separation of single-channel audio signals. The Itakura-Saito divergence NMF (Gaussian NMF) is justified for this purpose under an assumption that the complex spectra of source signals and those of the mixture signal are complex Gaussian distributed (the additiv-ity of power spectra holds). In fact, however, the source spectra are often heavy-tailed distributed. When the source spectra are complex Cauchy distributed, for example, the mixture spectra are also complex Cauchy distributed (the additivity of amplitude spectra holds). Using the complex t distribution that includes the complex Gaussian and Cauchy distributions as its special cases, we propose t-NMF as a unified extension of Gaussian NMF and Cauchy NMF. Furthermore, we propose the corresponding variant of positive semidefinite tensor factorization based on multivariate complex t distributions (t-PSDTF). The experimental results showed that while t-NMF and t-PSDTF were comparative to Gaussian counterparts in terms of peak performance, they worked much better on average because they are insensitive to initialization and tend to avoid local optima.

In this paper, the problem of single microphone source separation via Nonnegative Matrix Factorization (NMF) by exploiting video information is addressed. Respective audio and video modalities coming from a single human speech usually have similar time changes. It means that changes in one of them usually corresponds to changes in the other one. So it is expected that activation coefficient matrices of their NMF decomposition are similar. Based on this similarity, in this paper the activation coefficient matrix of the video modality is used as an initialization for audio source separation via NMF. In addition, the mentioned similarity is used for post-processing and for clustering the rows of the activation coefficient matrix which were resulted from randomly initialized NMF. Simulation results confirm the effectiveness of the proposed multimodal approaches in single microphone source separation.

This paper addresses the problem of multichannel audio source separation in under-determined convolutive mixtures. We target a semi-blind scenario assuming that the mixing filters are known. The convolutive mixing process is exactly modeled using the time-domain impulse responses of the mixing filters. We propose a Student's t time-frequency source model based on non-negative matrix factorization (NMF). The Student's t distribution being heavy-tailed with respect to the Gaussian, it provides some flexibility in the modeling of the sources. We also study a simpler Student's t sparse source model within the same general source separation framework. The inference procedure relies on a variational expectation-maximization algorithm. Experiments show the advantage of using an NMF model compared with the sparse source model. While the Student's t NMF source model leads to slightly better results than our previous Gaussian one, we demonstrate the superiority of our method over two other approaches from the literature.

This paper presents a statistical method of audio source separation based on a nonparametric Bayesian extension of probabilistic latent component analysis (PLCA). A major approach to audio source separation is to use nonnegative matrix factorization (NMF) that approximates the magnitude spectrum of a mixture signal at each frame as the weighted sum of fewer source spectra. Another approach is to use PLCA that regards the magnitude spectrogram as a two-dimensional histogram of "sound quanta" and classifies each quantum into one of sources. While NMF has a physically-natural interpretation, PLCA has been used successfully for music signal analysis. To enable PLCA to estimate the number of sources, we propose Dirichlet process PLCA (DP-PLCA) and derive two kinds of learning methods based on variational Bayes and collapsed Gibbs sampling. Unlike existing learning methods for nonparametric Bayesian NMF based on the beta or gamma processes (BP-NMF and GaP-NMF), our sampling method can efficiently search for the optimal number of sources without truncating the number of sources to be considered. Experimental results showFed that DP-PLCA is superior to GaP-NMF in terms of source number estimation.

The fields of video games, simulations and virtual reality are now tending to develop increasingly high-performance, realistic and immersive technologies. Efforts are made in terms of sound devices and sound processing to synthesize realistic sound scenes in a 3-D environment. One of the greatest challenges is the ability to analyze a 3-D audio stream corresponding to a complex sound scene in its basic components (i.e. individual sound sources), to modify the scene (e.g. to change the locations of sound sources) and to resynthesize a modified 3-D audio stream. This situation is referred to as "spatial remix".Actually, the spatial remix problem is still an open field. Work in progress rely on sound separation algorithms to analyze a sound scene, but these techniques are not perfect and can damage the reconstructed source signals. These are referred to as "separation artefacts", including transient alteration of the target source and rejections of other sources into the target source. Objective and subjective evaluation of separation artefacts have been conducted [1], but these studies usually consider the separated source signals alone, i.e. when each source is listened to separately. This is different form the spatial remix problem, where all sources are listened to simultaneously.In that case, one may wonder if the separation artefacts can affect the spatial image of the synthesized 3-D sound scene. According to the perceptual mechanisms involved in spatial hearing, hypothesis can be made on the kind of spatial distortions that could occur in this context. Indeed, as transients are important cues to precisely localize sounds sources, its alteration may result in a localization blur or source widening. On the other hand, when separated sources are spatialized and played simultaneously, rejections of one source into another may also produce unwanted effects such as a feeling of moving sources and "phantom" sources emergence. This paper presents a new methodology to perceptually evaluate the spatial distortions that can occur in a spatial remix context. It consists in carrying out a localization test on complex scenes composed of three synthetic musical instruments played on a set of loudspeakers. In order to eliminate possible issues related to the spatial audio rendering device, we consider a simple case: We consider only three spatial positions, each corresponding to a single loudspeaker. Then, the spatial remix is restrained to a simple permutation of the source locations.The test is run through a virtual interface, using a head mounted display. The subject is placed in a simple visual virtual environment and is asked to surround with a remote the areas where each instrument is perceived. This experimental device allows the subject to report precisely both instruments position and size. A single instrument can also be spotted at multiple locations. Perceived source positions are approximated as ellipses from which center position and dimensions can easily be deduced. In order to quantify spatial distortions, the localization task is performed on both clean and degraded versions of the same musical extract. Localization performances in both cases are then compared taking the clean sources case as a reference. In this paper, the methodology is applied to assess the quality of Non-Negative Matrix Factorization source separation algorithm developped by Leglaive [2] which performs separation on convolutive mixtures.Our study reveals that the source separation process leads to perceptible degradations of the spatial image. Three main kinds of spatial distortions have been characterized. First, in the majority of degraded cases, "phantom" sources have been observed. This artifact mainly concerns percussive sources. The results also show a significant increase in the perceived width of the degraded sources. Finally, azimuth and elevation localization error is significantly higher in the case of scenes composed of separated sources.[1] V. Emiya, E. Vincent, N. Harlander and V. Hohmann, "Subjective and Objective Quality Assessment of Audio Source Separation," in <i>IEEE Transactions on Audio, Speech, and Language Processing</i>, vol. 19, no. 7, pp. 2046-2057, Sept. 2011.[2] S. Leglaive, R. Badeau and G. Richard, "Separating time-frequency sources from time-domain convolutive mixtures using non-negative matrix factorization," <i>2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)</i>, New Paltz, NY, 2017, pp. 264-268.

Nonnegative matrix factorization (NMF) is often used for source separation of audio signals. In most of these algorithms, the initialization step of the NMF, which has a strong impact on the separation performance, is based on random values or deterministic methods such as singular value decomposition (SVD). Another deterministic initialization approach, which is used e.g. for score-informed source separation algorithms, makes use of synthesized magnitude spectra of harmonic notes. It was shown that this semantic method leads to good separation results in blind source separation (BSS) as well; not only for harmonic but also for percussive mixtures with some harmonic components. In this paper, we present an extension to the semantic approach to enhance the separation quality for arbitrary audio mixtures. We evaluate this extension in a BSS scenario and compare it to other initialization schemes.

An additional feature processing algorithm using Non-negative Matrix Factorization (NMF) is proposed to be included during the conventional extraction of Mel-frequency cepstral coefficients (MFCC) for achieving noise robustness in HMM based speech recognition. The proposed approach reconstructs log-Mel filterbank outputs of speech data from a set of building blocks that form the bases of a speech subspace. The bases are learned using the standard NMF of training data. A variation of learning the bases is proposed, which uses histogram equalized activation coefficients during training, to achieve noise robustness. The proposed methods give up to 5.96% absolute improvement in recognition accuracy on Aurora-2 task over a baseline with standard MFCCs, and up to 13.69% improvement when combined with other feature normalization techniques like Histogram Equalization (HEQ) and Heteroscedastic Linear Discriminant Analysis (HLDA).

This paper proposes a novel method for simultaneously solving the problems of underdetermined blind source separation (BSS), source activity detection, dereverberation and direction-of-arrival (DOA) estimation by introducing an extension of the "multichannel factorial hidden Markov model (MFH-MM)." The MFHMM is an extension of the multichannel non-negative matrix factorization (NMF) modeL in which the basis spectra are allowed to vary over time according to the transitions of the hidden states. This model has allowed us to perform source separation, source activity detection and dereverberation in a unified manner. In our previous model, the spatial covariance of each source has been treated as a model parameter. This has led the entire generative model to have an unnecessarily high degree of freedom, and thus the parameter inference has been prone to getting trapped into undesired local optima. To reasonably restrict the solution space of the spatial covariance matrix of each source, we propose to describe it as a weighted sum of the fixed spatial covariance matrix corresponding to the discrete set of DOAs. Through the parameter inference, the proposed model allows us to simultaneously solve the problems of underdetermined BSS, source activity detection, dereverberation and DOA estimation. Experimental results revealed that the proposed method was superior to a previous method in terms of the signal-to-distortion ratios of separated signals.

Informed Source Separation (ISS) is a topic unifying the research fields of both source separation and source coding. Its main objective is to recover audio objects out of a mixture with a source separation step assisted by a set of compact parameters extracted with complete knowledge of the sources. ISS can be used for applications such as active listening and remixing of music (e.g. karaoke). In this paper, we propose a new ISS method which includes a semi-blind source separation (SBSS) step in the ISS decoder to decrease the amount of parameter bit rate. SBSS is conducted by fac-torizing the mixture in time-frequency domain by nonnegative matrix factorization (NMF). The transmitted parameters consist of a compact NMF initialization as well as residuals calculated in the NMF domain. We show in simulations that using SBSS in the decoder increases the separation quality and that our scheme improves the rate-distortion performance in comparison to a state-of-the art method.

The Short-time Fourier transform (STFT) is a popular time-frequency representation in many source separation problems. In this work, the sampled and discretized version of Discrete Gabor Transform (DGT) is proposed to replace STFT within the single-channel source separation problem of the Non-negative Matrix Factorization (NMF) framework. The result shows that NMF-DGT is better than NMF-STFT according to Signal-to-Interference Ratio (SIR), Signal-to-Artifact Ratio (SAR), and Signal-to-Distortion Ratio (SDR). In the supervised scheme, NMF-DGT has a SIR of 18.60 dB compared to 16.24 dB in NMF-STFT, SAR of 13.77 dB to 13.69 dB, and SDR of 12.45 dB to 11.16 dB. In the unsupervised scheme, NMF-DGT has a SIR of 0.40 dB compared to 0.27 dB by NMF-STFT, SAR of -10.21 dB to -10.36 dB, and SDR of -15.01 dB to -15.23 dB.