A Class of Manifold Regularized Multiplicative Update Algorithms for Image Clustering

The Gaussian and Gaussian-sigmoidal primaries are introduced as initialization functions in the nonnegative matrix factorization (NNMF) method to achieve more consistent error statistics. The proposed NNMF method with multiplicative update algorithm is employed for reconstruction of spectral data of Munsell and Colorchecker DC datasets from the corresponding CIEXYZ tristimulus values and the resultant achievements are compared with those obtained from PCA and the classical NNMF techniques. Both colorimetric and spectrophotometric errors between the original and the reconstructed spectra are used for the evaluation of reconstruction methods. While run to run variability of the classical NNMF method with multiplicative update algorithm is significantly remarkable, the stability of the suggested technique is fairly high, similar to the PCA method. The ensuing findings show that the embedding of Gaussian initializing vectors in NNMF method provide the all positive bases that enhance both the accuracy of Gaussian method and the reliability of NNMF technique.

Non-negative matrix factorization (NMF) is a powerful approach to single channel audio source separation. In a supervised setting, NMF is first applied to train the basis spectra of each sound source. At test time, NMF is applied to the spectrogram of a mixture signal using the pretrained spectra. The source signals can then be separated out using a Wiener filter. A typical way to train the basis spectra of each source is to minimize the objective function of NMF. However, the basis spectra obtained in this way do not ensure that the separated signal will be optimal at test time due to the inconsistency between the objective functions for training and separation (Wiener filtering). To address this, a framework called discriminative NMF (DNMF) has recently been proposed. In in this work a multiplicative update algorithm was proposed for the basis training, however one drawback is that the convergence is not guaranteed. To overcome this drawback, this paper proposes using a majorization-minimization principle to develop a convergence-guaranteed algorithm for DNMF. Experimental results showed that the proposed algorithm outperformed standard NMF and DNMF using a multiplicative update algorithm as regards both the signal-to-distortion and signal-to-interference ratios.

The objective of single-channel source separation is to accurately recover source signals from mixtures. Non-negative matrix factorization (NMF) is a popular approach for this task, yet previous NMF approaches have not optimized directly this objective, despite some efforts in this direction. Our paper introduces discriminative training of the NMF basis functions such that, given the coefficients obtained on a mixture, a desired source is optimally recovered. We approach this optimization by generalizing the model to have separate analysis and reconstruction basis functions. This generalization frees us to optimize reconstruction objectives that incorporate the filtering step and SNR performance criteria. A novel multiplicative update algorithm is presented for the optimization of the reconstruction basis functions according to the proposed discriminative objective functions. Results on the 2nd CHiME Speech Separation and Recognition Challenge task indicate significant gains in source-to-distortion ratio with respect to sparse NMF, exemplar-based NMF, as well as a previously proposed discriminative NMF criterion.

Nonnegative matrix factorization (NMF) by the multiplicative updates algorithm is a powerful machine learning method for decomposing a high-dimensional nonnegative matrix V into two nonnegative matrices, W and H, where V ~ WH. It has been successfully applied in the analysis and interpretation of large-scale data arising in neuroscience, computational biology, and natural language processing, among other areas. A distinctive feature of NMF is its nonnegativity constraints that allow only additive linear combinations of the data, thus enabling it to learn parts that have distinct physical representations in reality. In this letter, we describe an information-theoretic approach to NMF for signal-dependent noise based on the generalized inverse gaussian model. Specifically, we propose three novel algorithms in this setting, each based on multiplicative updates, and prove monotonicity of updates using the EM algorithm. In addition, we develop algorithm-specific measures to evaluate their goodness of fit on data. Our methods are demonstrated using experimental data from electromyography studies, as well as simulated data in the extraction of muscle synergies, and compared with existing algorithms for signal-dependent noise.

The nonnegative convex polyhedral cone (NCPC) learning is discussed in this paper. By exploiting the multiplicative update nonnegative quadratic programming, a multiplicative update algorithm is developed for NCPC learning. The proposed algorithm is promising for nonnegative matrix factorization (NMF) and we verify this by numerical experiments.

Semi-supervised graph regularized nonnegative matrix factorization with local coordinate for image representation

A solution for separating speech from music signal as a single channel source separation is Non-negative Matrix Factorization (NMF). In this approach spectrogram of each source signal is factorized as multiplication of two matrices which are known as basis and weight matrices. To achieve proper estimation of signal spectrogram, weight and basis matrices are updated iteratively. To estimate distance between signal and its estimation a cost function is used usually. Different cost functions have been introduced based on Kullback-Leibler (KL) and Itakura-Saito (IS) divergences. IS divergence is scale-invariant and so it is suitable for the conditions in which the coefficients of signal have a large dynamic range, for example in music short-term spectra. Based on this IS property, in this paper, we propose to use IS divergence as cost function of NMF in the training stage for music and on the other hand we suggest to use KL divergence as NMF cost function in the training stage for speech. Moreover, in the decomposition stage, we propose to use a linear combination of these two divergences in addition to a regularization term which considers temporal continuity information as a prior knowledge. Experimental results on one hour of speech and music, shows a good trade-off between signal to inference ratio (SIR) of speech and music in comparison to conventional NMF methods.

Non-negative Matrix Factorization (NMF) is widely used in image clustering; however, it has inherent limitations, including its unsupervised nature, lack of sparsity constraints, inability to leverage label information, and difficulty in capturing the geometric structure and nonlinear characteristics of data. To address these limitations, this paper proposes a Sparse Kernel Graph-regularized Discriminant Non-negative Matrix Factorization (SKGDNMF) algorithm. The algorithm innovatively adopts a dual normalization strategy, which involves column normalization for the basis matrix and row normalization for the coefficient matrix, and integrates spectral clustering to construct an end-to-end deep clustering framework. By applying sparse regularization to the coefficient matrix, the model’s robustness is significantly improved; this regularization forms complementary optimization with graph regularization, thereby effectively alleviating overfitting. A three-dimensional golden-section parameter optimization method is employed to determine key parameters, which enhances the algorithm’s practicality. Comparative experiments conducted on multiple datasets show that SKGDNMF significantly outperforms mainstream algorithms in terms of sparsity, robustness, and clustering performance, indicating its superior effectiveness for image clustering tasks.

Non-negative Matrix Factorization (NMF) has been frequently applied to image feature extraction and clustering. Especially in image clustering tasks, it can achieve the similar or better performance than most of the matrix factorization algorithms due to its parts-based representations in the brain. However, the features extracted by NMF are not sparse and localized enough and the error of factorization is not small enough. Semi-tensor product of matrices (STP) is a novel operation of matrix multiplication, it is a generalization of the conventional matrix product for allowing the dimensions of factor matrices to be unequal. STP can manage the data hierarchically and the inverse process of STP can separate the data hierarchically. Based on this character of STP, we propose the Non-Negative Matrix Semi-Tensor Factorization (NMSTF). In this algorithm, we use the inverse process of Semi-Tensor Product of matrices for non-negative matrix factorization. This algorithm effectively optimizes the above two problems in NMF. While achieving similar even better performance on image clustering tasks, the size of features extracted by STNMF is at least 50 % smaller than the ones’ extracted by NMF and the error of factorization reduces 30 % in average.

Non-negative matrix factorization for speech/music separation using source dependent decomposition rank, temporal continuity term and filtering

Non-negative matrix factorization (NMF) is a very popular dimensionality reduction method that has been widely used in computer vision and data clustering. However, NMF does not consider the intrinsic geometric information of a data set and also does not produce smooth and stable solutions. To resolve these problems, we propose a Graph regularized Lp Smooth Non-negative Matrix Factorization (GSNMF) method by incorporating graph regularization with Lp smooth constraint. The graph regularization can discover the hidden semantics and simultaneously respect the intrinsic geometric structure information of a data set. The Lp smooth constraint can combine the merits of isotropic (L2-norm) and anisotropic (L1-norm) diffusion smoothing, and produce a smooth and more accurate solution to the optimization problem. Experimental results on some data sets demonstrate that the proposed method outperforms related state-of-the-art NMF methods.

Nonnegative matrix factorization (NMF) provides a lower rank approximation of a matrix by a product of two nonnegative factors. NMF has been shown to produce clustering results that are often superior to those by other methods such as K-means. In this paper, we provide further interpretation of NMF as a clustering method and study an extended formulation for graph clustering called Symmetric NMF (SymNMF). In contrast to NMF that takes a data matrix as an input, SymNMF takes a nonnegative similarity matrix as an input, and a symmetric nonnegative lower rank approximation is computed. We show that SymNMF is related to spectral clustering, justify SymNMF as a general graph clustering method, and discuss the strengths and shortcomings of SymNMF and spectral clustering. We propose two optimization algorithms for SymNMF and discuss their convergence properties and computational efficiencies. Our experiments on document clustering, image clustering, and image segmentation support SymNMF as a graph clustering method that captures latent linear and nonlinear relationships in the data.

Reverberant signal separation using optimized complex sparse nonnegative tensor deconvolution on spectral covariance matrix

This article describes a spatio-temporal EEG/MEG source imaging (ESI) that extracts a parsimonious set of "atoms" or components, each the outer product of both a spatial and a temporal signature. The sources estimated are localized as smooth, minimally overlapping patches of cortical activation that are obtained by constraining spatial signatures to be nonnegative (NN), orthogonal, sparse, and smooth-in effect integrating ESI with NN-ICA. This constitutes a generalization of work by this group on the use of multiple penalties for ESI. A multiplicative update algorithm is derived being stable, fast and converging within seconds near the optimal solution. This procedure, spatio-temporal tomographic NN ICA (STTONNICA), is equally able to recover superficial or deep sources without additional weighting constraints as tested with simulations. STTONNICA analysis of ERPs to familiar and unfamiliar faces yields an occipital-fusiform atom activated by all faces and a more frontal atom that only is active with familiar faces. The temporal signatures are at present unconstrained but can be required to be smooth, complex, or following a multivariate autoregressive model.

The compression method, first proposed in 2012, is based on the non-negative tensor decomposition for interference hyper-spectral image data. As a tensor is generated by a huge amount of interference hyper-spectral images, the multiplicative update algorithm is made extremely complicated, and even unfeasible. To reduce the computational cost and speed up the convergence, this paper, based on the characteristics of interference hyper-spectral images, develops a new algorithm using different down-sampling factors for different non-negative wavelet sub-band tensors. The experimental results showed that this algorithm could significantly shorten the running time, while maintaining a good compression performance compared with the conventional methods.