Uniqueness of Nonnegative Matrix Factorizations by Rigidity Theory

Nonnegative matrix factorization and I-divergence alternating minimization

Nonnegative matrix factorization (NMF) provides a lower rank approximation of a matrix by a product of two nonnegative factors. NMF has been shown to produce clustering results that are often superior to those by other methods such as K-means. In this paper, we provide further interpretation of NMF as a clustering method and study an extended formulation for graph clustering called Symmetric NMF (SymNMF). In contrast to NMF that takes a data matrix as an input, SymNMF takes a nonnegative similarity matrix as an input, and a symmetric nonnegative lower rank approximation is computed. We show that SymNMF is related to spectral clustering, justify SymNMF as a general graph clustering method, and discuss the strengths and shortcomings of SymNMF and spectral clustering. We propose two optimization algorithms for SymNMF and discuss their convergence properties and computational efficiencies. Our experiments on document clustering, image clustering, and image segmentation support SymNMF as a graph clustering method that captures latent linear and nonlinear relationships in the data.

In this letter, we consider the nonnegative matrix factorization (NMF) problem and several NMF variants. Two approaches based on DC (difference of convex functions) programming and DCA (DC algorithm) are developed. The first approach follows the alternating framework that requires solving, at each iteration, two nonnegativity-constrained least squares subproblems for which DCA-based schemes are investigated. The convergence property of the proposed algorithm is carefully studied. We show that with suitable DC decompositions, our algorithm generates most of the standard methods for the NMF problem. The second approach directly applies DCA on the whole NMF problem. Two algorithms-one computing all variables and one deploying a variable selection strategy-are proposed. The proposed methods are then adapted to solve various NMF variants, including the nonnegative factorization, the smooth regularization NMF, the sparse regularization NMF, the multilayer NMF, the convex/convex-hull NMF, and the symmetric NMF. We also show that our algorithms include several existing methods for these NMF variants as special versions. The efficiency of the proposed approaches is empirically demonstrated on both real-world and synthetic data sets. It turns out that our algorithms compete favorably with five state-of-the-art alternating nonnegative least squares algorithms.

The paper discusses a pooling mechanism to induce subsampling in graph structured data and introduces it as a component of a graph convolutional neural network. The pooling mechanism builds on the Non-Negative Matrix Factorization (NMF) of a matrix representing node adjacency and node similarity as adaptively obtained through the vertices embedding learned by the model. Such mechanism is applied to obtain an incrementally coarser graph where nodes are adaptively pooled into communities based on the outcomes of the non-negative factorization. The empirical analysis on graph classification benchmarks shows how such coarsening process yields significant improvements in the predictive performance of the model with respect to its non-pooled counterpart.

Non-negative Matrix Factorization (NMF) has been frequently applied to image feature extraction and clustering. Especially in image clustering tasks, it can achieve the similar or better performance than most of the matrix factorization algorithms due to its parts-based representations in the brain. However, the features extracted by NMF are not sparse and localized enough and the error of factorization is not small enough. Semi-tensor product of matrices (STP) is a novel operation of matrix multiplication, it is a generalization of the conventional matrix product for allowing the dimensions of factor matrices to be unequal. STP can manage the data hierarchically and the inverse process of STP can separate the data hierarchically. Based on this character of STP, we propose the Non-Negative Matrix Semi-Tensor Factorization (NMSTF). In this algorithm, we use the inverse process of Semi-Tensor Product of matrices for non-negative matrix factorization. This algorithm effectively optimizes the above two problems in NMF. While achieving similar even better performance on image clustering tasks, the size of features extracted by STNMF is at least 50 % smaller than the ones’ extracted by NMF and the error of factorization reduces 30 % in average.

Nonnegative Matrix Factorization (NMF) is valuable in many applications of blind source separation, signal processing and machine learning. A number of algorithms that can infer nonnegative latent factors have been developed, but most of these assume a specific noise kernel. This is insufficient to deal with complex noise in real scenarios. In this paper, we present a hierarchical Dirichlet process nonnegative matrix factorization (DPNMF) model in which the Gaussian mixture model is used to approximate the complex noise distribution. Moreover, the model is cast in the nonparametric Bayesian framework by using Dirichlet process mixture to infer the necessary number of Gaussian components. We derive a mean-field variational inference algorithm for the proposed nonparametric Bayesian model. We first test the model on synthetic data sets contaminated by Gaussian, sparse and mixed noise. We then apply it to extract muscle synergies from the electromyographic (EMG) signal and to select discriminative features for motor imagery single-trial electroencephalogram (EEG) classification. Experimental results demonstrate that DPNMF performs better in extracting the latent nonnegative factors in comparison with state-of-the-art methods.

Hyperspectral unmixing refers to a source separation problem of decomposing a hyperspectral imagery (HSI) to estimate endmembers, and their corresponding abundances. Recently, matrix-vector nonnegative tensor factorization (MV-NTF) was proposed for unmixing to avoid structure information loss, which is caused by the HSI cube unfolding in nonnegative matrix factorization (NMF)-based methods. However, MV-NTF ignores local spatial information due to directly dealing with data as a whole, meanwhile, the forceful rank constraint in low-rank tensor decomposition loses some detailed structures. Unlike MV-NTF works at the original data, the pixel-based NMF is more adaptive to learn local spatial variations. Hence, from the perspective of multi-view, itis significant to utilize the complementary advantages of MV-NTF and NMF to fully preserve the intrinsic structure information, and exploit more detailed spatial information. In this article, we propose a sparsity-constrained coupled nonnegative matrix-tensor factorization (SCNMTF) model for unmixing, wherein MV-NTF and NMF are subtly coupled by sharing endmembers and abundances. Since the representations for abundances in MV-NTF and NMF are distinct, abundance sharing is achieved indirectly by introducing an auxiliary constraint. Furthermore, the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/2</sub> regularizer is adopted to promote the sparsity of abundances. A series of experiments on synthetic and real hyperspectral data demonstrate the effectiveness of the proposed SCNMTF method.

Obtaining an optimum data representation is a challenging issue that arises in many intellectual data processing techniques such as data mining, pattern recognition, and gene clustering. Many existing methods formulate this problem as a nonnegative matrix factorization (NMF) approximation problem. The standard NMF uses the least square loss function, which is not robust to outlier points and noises and fails to utilize prior label information to enhance the discriminability of representations. In this study, we develop a novel matrix factorization method called robust semisupervised nonnegative local coordinate factorization by integrating robust NMF, a robust local coordinate constraint, and local spline regression into a unified framework. We use the l2,1 norm for the loss function of the NMF and a local coordinate constraint term to make our method insensitive to outlier points and noises. In addition, we exploit the local and global consistencies of sample labels to guarantee that data representation is compact and discriminative. An efficient multiplicative updating algorithm is deduced to solve the novel loss function, followed by a strict proof of the convergence. Several experiments conducted in this study on face and gene datasets clearly indicate that the proposed method is more effective and robust compared to the state-of-the-art methods.

We have studied the feature extraction from sea ice SAR images based on non-negative factorization methods. The methods reported here are the sparseness-constrained non-negative matrix factorization (SC-NMF) and Non-negative tensor factorization (NTF). The studies performed show that these methods can be used to extract meaningful features from SAR images and that they can be used in sea ice SAR classification.

Exploring syndrome differentiation using non-negative matrix factorization and cluster analysis in patients with atopic dermatitis

Non-negative matrix factorization (NMF) has found numerous applications, due to its ability to provide interpretable decompositions. Perhaps surprisingly, existing results regarding its uniqueness properties are rather limited, and there is much room for improvement in terms of algorithms as well. Uniqueness and computational aspects of NMF are revisited here from a geometrical point of view. Both symmetric and asymmetric NMF are considered, the former being tantamount to element-wise non-negative square-root factorization of positive semidefinite matrices. New and insightful uniqueness results are derived, e.g., it is shown that a sufficient condition for uniqueness is that the conic hull of the latent factors is a superset of a particular second-order cone. Checking this is shown to be NP-complete; yet it offers insights on latent sparsity, as is also shown in a new necessary condition, to a smaller extent. On the computational side, a new efficient algorithm for symmetric NMF is proposed which uses Procrustes rotations. Simulation results show promising performance with respect to the state-of-art. The new algorithm is also applied to a clustering problem for co-authorship data, yielding meaningful and interpretable results.

Nonnegative matrix factorization (NMF), which can lead to nonsubtractive parts-based representation, has been demonstrated to be effective for dimensionality reduction of hyperspectral imagery (HSI). However, existing NMF methods applied to HSI use only a single spectral feature and do not take into consideration spatial information, such as texture or morphological features, while it has been widely acknowledged that exploiting multiple features can improve performance. Consequently, a variant of orthogonal NMF, which can not only achieve a nonnegative factorization but also exploit the complementary information that arises among heterogeneous features, is proposed for hyperspectral dimensionality reduction. The proposed method, which couples orthogonal NMF with a previous multiple-features-combining algorithm, yields a discriminative low-dimensional feature representation that matches the intuition that parts should sum to produce a whole. An efficient multiplicative updating procedure is derived, and its local convergence is guaranteed theoretically. Experimental results on two hyperspectral data sets demonstrate the effectiveness of the proposed method.

Non-negative Matrix Factorization (NMF) is a recently developed method for dimensionality reduction, feature extraction and data mining, etc. Currently no NMF algorithm holds both satisfactory efficiency for applications and enough ease of use. To improve the applicability of NMF, this paper proposes a new monotonic, fixed-point algorithm coined FastNMF by implementing least squares error-based non-negative factorization essentially according to the basic properties of parabola functions. The minimization problem corresponding to an operation in FastNMF can be analytically solved just by this algorithm, which is far beyond all existing algorithmspsila power. Therefore, FastNMF holds much higher efficiency, which is validated by a number of experimental results. For the simplicity of design philosophy, FastNMF is still one of NMF algorithms that are the easiest to use and the most comprehensible. Besides, theoretical analysis and experimental results also show that FastNMF tends to converge to better solutions than the popular multiplicative update-based algorithms.

Given any nonnegative matrix $A \in \mathbb{R}^{m \times n}$ , it is always possible to express A as the sum of a series of nonnegative rank-one matrices. Among the many possible representations of A, the number of terms that contributes the shortest nonnegative rank-one series representation is called the nonnegative rank of A. Computing the exact nonnegative rank and the corresponding factorization are known to be NP-hard. Even if the nonnegative rank is known a priori, no simple procedure exists presently that is able to perform the nonnegative factorization. Based on the Wedderburn rank reduction formula, this paper proposes a heuristic approach to tackle this difficult problem numerically. Starting with A, the idea is to recurrently extrat, whenever possible, a rank-one nonnegative portion from the previous matrix while keeping the residual nonnegative and lowering its rank by one. With a slight modification for symmetry, the method can equally be applied to another important class of completely positive matrices. No convergence can be guaranteed, but repeated restart might help alleviate the difficulty. Extensive numerical testing seems to suggest that the proposed algorithm might serve as a first-step numerical means for exploring the intriguing problem of nonnegative rank factorization.

Nonnegative matrix factorization (NMF) is widely used to analyze high-dimensional count data because, in contrast to real-valued alternatives such as factor analysis, it produces an interpretable parts-based representation. However, in applications such as spatial transcriptomics, NMF fails to incorporate known structure between observations. Here, we present nonnegative spatial factorization (NSF), a spatially-aware probabilistic dimension reduction model based on transformed Gaussian processes that naturally encourages sparsity and scales to tens of thousands of observations. NSF recovers ground truth factors more accurately than real-valued alternatives such as MEFISTO in simulations, and has lower out-of-sample prediction error than probabilistic NMF on three spatial transcriptomics datasets from mouse brain and liver. Since not all patterns of gene expression have spatial correlations, we also propose a hybrid extension of NSF that combines spatial and nonspatial components, enabling quantification of spatial importance for both observations and features. A TensorFlow implementation of NSF is available from https://github.com/willtownes/nsf-paper.