NMF revisited: New uniqueness results and algorithms

Nonnegative matrix factorization (NMF) provides a lower rank approximation of a matrix by a product of two nonnegative factors. NMF has been shown to produce clustering results that are often superior to those by other methods such as K-means. In this paper, we provide further interpretation of NMF as a clustering method and study an extended formulation for graph clustering called Symmetric NMF (SymNMF). In contrast to NMF that takes a data matrix as an input, SymNMF takes a nonnegative similarity matrix as an input, and a symmetric nonnegative lower rank approximation is computed. We show that SymNMF is related to spectral clustering, justify SymNMF as a general graph clustering method, and discuss the strengths and shortcomings of SymNMF and spectral clustering. We propose two optimization algorithms for SymNMF and discuss their convergence properties and computational efficiencies. Our experiments on document clustering, image clustering, and image segmentation support SymNMF as a graph clustering method that captures latent linear and nonlinear relationships in the data.

In this letter, we consider the nonnegative matrix factorization (NMF) problem and several NMF variants. Two approaches based on DC (difference of convex functions) programming and DCA (DC algorithm) are developed. The first approach follows the alternating framework that requires solving, at each iteration, two nonnegativity-constrained least squares subproblems for which DCA-based schemes are investigated. The convergence property of the proposed algorithm is carefully studied. We show that with suitable DC decompositions, our algorithm generates most of the standard methods for the NMF problem. The second approach directly applies DCA on the whole NMF problem. Two algorithms-one computing all variables and one deploying a variable selection strategy-are proposed. The proposed methods are then adapted to solve various NMF variants, including the nonnegative factorization, the smooth regularization NMF, the sparse regularization NMF, the multilayer NMF, the convex/convex-hull NMF, and the symmetric NMF. We also show that our algorithms include several existing methods for these NMF variants as special versions. The efficiency of the proposed approaches is empirically demonstrated on both real-world and synthetic data sets. It turns out that our algorithms compete favorably with five state-of-the-art alternating nonnegative least squares algorithms.

The significance of Non-negative Matrix Factorization ((NMF) in the field of automatic text summarization is rapidly increasing due to its interpretation and storage capabilities. Interpretation defines the ease at which the structure of high dimensional data can be understood. While, storage capability relates to the extent of data reduction process achieved by NMF. The parametric values that serve as input to the NMF process include initialization method, rank of factorization, sparseness measure and maximum iteration. These inputs are vital to the output produced by NMF. These parameters of NMF were not been considered in the existing literature on Automatic Text Summarization (ATS). This paper sheds light specifically on sparseness of NMF when applied to ATS and the impact it makes to the quality of the summary generated. In this study on NMF algorithm that supports sparseness is treated with various degree of sparsity and its performance impact on text summary generated is compared with other NMF algorithms without sparseness constraints.

Non-negative matrix factorization (NMF) is widely used in feature extraction and dimension reduction fields. Essentially, it is an optimization problem to determine two non-negative low rank matrices \(W_{m \times k}\) and \(H_{k \times n}\) for a given matrix \(A_{m \times n}\), satisfying \(A_{m \times n} \approx W_{m \times k}H_{k \times n}\). In this paper, a novel approach to improve the image decomposing and reconstruction effects by introducing the Singular Value Decomposing (SVD)-based initialization scheme of factor matrices W and H, and another measure called choosing rule to determine the optimum value of factor rank k, are proposed. The input image is first decomposed using SVD to get its singular values and corresponding eigenvectors. Then, the number of main components as the rank value k is extracted. Then, the singular values and corresponding eigenvectors are used to initialize W and H based on selected rank k. Finally, convergent results are obtained using multiplicative and additive update rules. However, iterative NMF algorithms’ convergence is very slow on most platforms limiting its practicality. To this end, a parallel implementation frame of described improved NMF algorithm using CUDA, a tool for algorithms parallelization on massively parallel processors, i.e., many-core graphics processors, is presented. Experimental results show that our approach can get better decomposing effect than traditional NMF implementations and dramatic accelerate rate comparing to serial schemes as well as existing distributed-system implementations.

Nonnegative matrix factorization and I-divergence alternating minimization

Nonnegative matrix factorizations are often encountered in data mining applications where they are used to explain datasets by a small number of parts. For many of these applications it is desirable that there exists a unique nonnegative matrix factorization up to trivial modifications given by scalings and permutations. This means that model parameters are uniquely identifiable from the data. Rigidity theory of bar and joint frameworks is a field that studies uniqueness of point configurations given some of the pairwise distances. The goal of this paper is to use ideas from rigidity theory to study uniqueness of nonnegative matrix factorizations in the case when nonnegative rank of a matrix is equal to its rank. We characterize infinitesimally rigid nonnegative factorizations, prove that a nonnegative factorization is infinitesimally rigid if and only if it is locally rigid and a certain matrix achieves its maximal possible Kruskal rank, and show that locally rigid nonnegative factorizations can be extended to globally rigid nonnegative factorizations. These results give so far the strongest necessary condition for the uniqueness of a nonnegative factorization. We also explore connections between rigidity of nonnegative factorizations and boundaries of the set of matrices of fixed nonnegative rank. Finally we extend these results from nonnegative factorizations to completely positive factorizations.

Non-negative Matrix Factorization (NMF) is a recently developed method for dimensionality reduction, feature extraction and data mining, etc. Currently no NMF algorithm holds both satisfactory efficiency for applications and enough ease of use. To improve the applicability of NMF, this paper proposes a new monotonic, fixed-point algorithm coined FastNMF by implementing least squares error-based non-negative factorization essentially according to the basic properties of parabola functions. The minimization problem corresponding to an operation in FastNMF can be analytically solved just by this algorithm, which is far beyond all existing algorithmspsila power. Therefore, FastNMF holds much higher efficiency, which is validated by a number of experimental results. For the simplicity of design philosophy, FastNMF is still one of NMF algorithms that are the easiest to use and the most comprehensible. Besides, theoretical analysis and experimental results also show that FastNMF tends to converge to better solutions than the popular multiplicative update-based algorithms.

Non-negative Matrix Factorization (NMF) has been frequently applied to image feature extraction and clustering. Especially in image clustering tasks, it can achieve the similar or better performance than most of the matrix factorization algorithms due to its parts-based representations in the brain. However, the features extracted by NMF are not sparse and localized enough and the error of factorization is not small enough. Semi-tensor product of matrices (STP) is a novel operation of matrix multiplication, it is a generalization of the conventional matrix product for allowing the dimensions of factor matrices to be unequal. STP can manage the data hierarchically and the inverse process of STP can separate the data hierarchically. Based on this character of STP, we propose the Non-Negative Matrix Semi-Tensor Factorization (NMSTF). In this algorithm, we use the inverse process of Semi-Tensor Product of matrices for non-negative matrix factorization. This algorithm effectively optimizes the above two problems in NMF. While achieving similar even better performance on image clustering tasks, the size of features extracted by STNMF is at least 50 % smaller than the ones’ extracted by NMF and the error of factorization reduces 30 % in average.

Nonnegative Matrix Factorization (NMF) approximates a given data matrix as a product of two low rank nonnegative matrices, usually by minimizing the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> or the KL distance between the data matrix and the matrix product. This factorization was shown to be useful for several important computer vision applications. We propose here a new NMF algorithm that minimizes the Earth Mover's Distance (EMD) error between the data and the matrix product. We propose an iterative NMF algorithm (EMD NMF) and prove its convergence. The algorithm is based on linear programming. We discuss the numerical difficulties of the EMD NMF and propose an efficient approximation. Naturally, the matrices obtained with EMD NMF are different from those obtained with L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> NMF. We discuss these differences in the context of two challenging computer vision tasks - texture classification and face recognition - and demonstrate the advantages of the proposed method.

Characterizing sediment sources by non-negative matrix factorization of detrital geochronological data

Hyperspectral unmixing refers to a source separation problem of decomposing a hyperspectral imagery (HSI) to estimate endmembers, and their corresponding abundances. Recently, matrix-vector nonnegative tensor factorization (MV-NTF) was proposed for unmixing to avoid structure information loss, which is caused by the HSI cube unfolding in nonnegative matrix factorization (NMF)-based methods. However, MV-NTF ignores local spatial information due to directly dealing with data as a whole, meanwhile, the forceful rank constraint in low-rank tensor decomposition loses some detailed structures. Unlike MV-NTF works at the original data, the pixel-based NMF is more adaptive to learn local spatial variations. Hence, from the perspective of multi-view, itis significant to utilize the complementary advantages of MV-NTF and NMF to fully preserve the intrinsic structure information, and exploit more detailed spatial information. In this article, we propose a sparsity-constrained coupled nonnegative matrix-tensor factorization (SCNMTF) model for unmixing, wherein MV-NTF and NMF are subtly coupled by sharing endmembers and abundances. Since the representations for abundances in MV-NTF and NMF are distinct, abundance sharing is achieved indirectly by introducing an auxiliary constraint. Furthermore, the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/2</sub> regularizer is adopted to promote the sparsity of abundances. A series of experiments on synthetic and real hyperspectral data demonstrate the effectiveness of the proposed SCNMTF method.

Non-negative matrix factorization (NMF) has been one of the most popular methods for feature learning in the field of machine learning and computer vision. Most existing works directly apply NMF on high-dimensional image datasets for computing the effective representation of the raw images. However, in fact, the common essential information of a given class of images is hidden in their low rank parts. For obtaining an effective low-rank data representation, we in this paper propose a non-negative low-rank matrix factorization (NLMF) method for image clustering. For the purpose of improving its robustness for the data in a manifold structure, we further propose a graph regularized NLMF by incorporating the manifold structure information into our proposed objective function. Finally, we develop an efficient alternating iterative algorithm to learn the low-dimensional representation of low-rank parts of images for clustering. Alternatively, we also incorporate robust principal component analysis into our proposed scheme. Experimental results on four image datasets reveal that our proposed methods outperform four representative methods.

Lee and Seung proposed nonnegative matrix factorization (NMF) algorithms to decompose patterns and images for structure retrieving. The NMF algorithms have been applied to various optimization problems. However, it is difficult to prove the convergence of this class of learning algorithms. This paper presents the global minima analysis of the NMF algorithms. In the analysis, invariant set is constructed so that the non-divergence of the algorithms can be guaranteed in the set. Using the features of linear equation systems and their solutions, the fixed points and convergence properties of the update algorithms are discussed in detail. The analysis shows that, although the cost function is not convex in both A and X together, it is possible to obtain the global minima from the particular learning algorithms. For different initializations, simulations are presented to confirm the analysis results.

In this work, we will consider the dimension reduction of the set of time series, such as economic data, to find the meaningful basis vector for the set of data, and indicate which data use which basis vector. Usually each of the time series is analyzed independently in economics but here we will analyze the set of time series simultaneously. Since some of the economic data are measured as positive values and we want to decompose them as a mixture of the parts, we will apply non-negative matrix factorization to the economic data. Non-negative matrix factorization can compress dimensions by approximating a non-negative matrix with the product of two non-negative matrices. The two non-negative matrices are called the coefficient matrix and the basis matrix, and the basis matrix can be considered as a dimensionally compressed matrix. If the standard non-negative matrix factorization is used for economic data, the basis matrix may not be smooth. We think that the basis vectors should be smooth except a few special economical incidents. In the proposed method, a Graph-based non-negative matrix factorization is introduced to regularize the basis matrix of the time series. A path graph for representing the time series of economic data is incorporated into the non-negative matrix factorization as regularization. As a result, basis vectors that maintains the time series of economic data are decomposed. Furthermore, we propose to introduce a sparsity in the non-negative matrix factorization. Traditionally, the sparsity incorporated into non-negative matrix factorization has been used for basis vectors. However, the proposed method introduces the sparsity for coefficient vectors. Thus the proposed method, which simultaneously incorporates the sparsity for the coefficient vectors and the smoothness for the basis vectors, can extract the smooth basis vectors and the original economical data are approximated as the weighted sum of the few bases vectors. This allows us to discover economic trends and the best-fit trends for each data at the same time.

Non-negative Matrix Factorisation (NMF) is a commonly used tool in many musical signal processing tasks, including Automatic Music Transcription (AMT). However unsupervised NMF is seen to be problematic in this context, and harmonically constrained variants of NMF have been proposed. While useful, the harmonic constraints may be constrictive in mixed signals. We have previously observed that recovery of overlapping signal elements using NMF is improved through introduction of a sparse coding step, and propose here the incorporation of a sparse coding step using the Hellinger distance into a NMF algorithm. Improved AMT results for unsupervised NMF are reported.