Sparse nonnegative matrix factorization with ℓ0-constraints

Sparse Non-negative Matrix Factorization (SNMF) based color unmixing for breast histopathological image analysis

Nonnegative Matrix Factorization (NMF), as a popular feature extraction technique, has recently attracted increasing attentions in high-dimensional data analysis, due to the strong ability of dimension reduction and semantic representation. In order to utilize the partial label information in practice, several semi-supervised NMF methods have been proposed. However, existing semi-supervised NMF variants only consider the global label constraint of data, but ignore the local label information embedded in geometric structure, which is very crucial for data representation. To address the above issue, we propose an improved NMF method, namely local and global regularized semi-supervised NMF (LGNMF), by considering the local and global label constraints simultaneously. Specifically, the local label constraint is depicted in a graph, which is pre-computed by metric learning. In addition, to explore the global label information, we construct an indicator matrix to restrict the coefficient matrix in LGNMF. For the formulated LGNMF, a multiplicative update rule (MUR) is developed. Extensive experiments on several real datasets demonstrate the superiority of the proposed method over the state-of-the-art methods in terms of clustering accuracy and normalized mutual information.

In this paper, we present a method for 3D mesh segmentation based on sparse non-negative matrix factorization (NMF). Image analysis techniques based on NMF have been shown to decompose images into semantically meaningful local features. Since the features and coefficients are represented in terms of non-negative values, the features contribute to the resulting images in an intuitive, additive fashion. Like spectral mesh segmentation, our method relies on the construction of an affinity matrix which depends on the geometric properties of the mesh. We show that segmentation based on the NMF is simpler to implement, and can result in more meaningful segmentation results than spectral mesh segmentation.

이 논문은 고차원의 데이터를 저 차원으로 줄이는 방법 중 하나인 특징추출에 대한 방법들의 특성을 비교한다. 비교대상 방법은 전통적인 PCA(Principal Component Analysis)방법과 시각피질의 특성을 보인다고 알려진 ICA(Independent Component Analysis), 국소기반인식을 구현한 NMF(Non-negative Matrix Factorization), 그리고 이의 성능을 개선한 sNMF(Sparse NMF)로 정하였다. 추출된 특징들의 특성을 시각적으로 확인하기 위하여 필기체 숫자 영상을 대상으로 특징추출을 수행하였으며, 인식기에 적용한 효과의 확인을 위하여 추출된 특징을 다층퍼셉트론에 학습시켜보았다. 각 방법의 특성을 비교한 결과는 응용하고자 하는 문제에서 어떤 특징을 추출하기 원하느냐에 따라 특징추출 방법을 선정할 때 유용할 것이다. In this paper, feature extraction methods, which is one field of reducing dimensions of high-dimensional data, are empirically investigated. We selected the traditional PCA(Principal Component Analysis), ICA(Independent Component Analysis), NMF(Non-negative Matrix Factorization), and sNMF(Sparse NMF) for comparisons. ICA has a similar feature with the simple cell of V1. NMF implemented a "parts-based representation in the brain" and sNMF is a improved version of NMF. In order to visually investigate the extracted features, handwritten digits are handled. Also, the extracted features are used to train multi-layer perceptrons for recognition test. The characteristic of each feature extraction method will be useful when applying feature extraction methods to many real-world problems.

Although nonnegative matrix factorization (NMF) favors a part-based and sparse representation of its input, there is no guarantee for this behavior. Several extensions to NMF have been proposed in order to introduce sparseness via the ℓ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> -norm, while little work is done using the more natural sparseness measure, the ℓ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sup> -pseudo-norm. In this work we propose two NMF algorithms with ℓ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sup> -sparseness constraints on the bases and the coefficient matrices, respectively. We show that classic NMF is a suited tool for ℓ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sup> -sparse NMF algorithms, due to a property we call sparseness maintenance. We apply our algorithms to synthetic and real-world data and compare our results to sparse NMF and nonnegative K-SVD.

While nonnegative matrix factorization (NMF) has successfully been applied for gain-robust multi-pitch detection, a method to track pitch values over time was not provided. We embed NMF-based pitch detection into a recently proposed pitch-tracking system, based on a factorial hidden Markov model (FHMM). The original system models speech spectra with Gaussian mixture models, which is sensitive to a gain mismatch between training and test data. We therefore combine the advantages of these two approaches and derive a gain-adaptive observation model for the FHMM. As training algorithm we use a modification of ℓ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sup> -sparse NMF, which represents the short-time spectrum with scalable basis vectors. In experiments we show that the new approach significantly increases the gain-robustness of the original tracking system.

In this paper, we present a method based on sparse non-negative matrix factorization (NMF) for brain tissue segmentation using diffusion MRI (DMRI) data. Unlike existing NMF-based approaches, in our method NMF is applied to the spherical mean data, computed on a per-shell basis, instead of the original diffusion-weighted images. This is motivated by the fact that the spherical mean is independent of the fiber orientation distribution and is only dependent on tissue microstructure. Applying NMF to the spherical mean data will hence allow tissue signal separation based solely on the microstructural properties, unconfounded by factors such as fiber dispersion and crossing. We show results explaining why applying NMF directly on the diffusion-weighted images fails and why our method is able to yield the expected outcome, producing tissue segmentation with greater accuracy.

Feature analysis is a fundamental research area in computer graphics; meanwhile, meaningful and part-aware feature bases are always demanding. This paper proposes a framework for conducting feature analysis on a three-dimensional (3D) model by introducing modified Non-negative Matrix Factorization (NMF) model into the graphical feature space and push forward further applications. By analyzing and utilizing the intrinsic ideas behind NMF, we propose conducting the factorization on feature matrices constructed based on descriptors or graphs, which provides a simple but effective way to raise compressed and scale-aware descriptors. In order to enable part-aware model analysis, we modify the NMF model to be sparse and constrained regarding to both bases and encodings, which gives rise to Sparse and Constrained Non-negative Matrix Factorization (SAC-NMF). Subsequently, by adapting the analytical components (including hidden variables, bases, and encodings) to design descriptors, several applications have been easily but effectively realized. The extensive experimental results demonstrate that the proposed framework has many attractive advantages, such as being efficient, extendable, and so forth.

Nonnegative matrix factorization (NMF) provides a way for finding a part-based representation of nonnegative data. An important property of NMF is that it can produce a sparse representation of the data; however, in some applications, especially in text clustering, the sparse representation always consists of separated words, which cannot explicitly express the meaning of the basis vector. This paper presents a new descriptive clustering algorithm based on NMF, called DC-NMF that can avoid this separated word problem. In our proposed method, we embrace the phrase-by-document matrix in addition to the commonly used term-by-document matrix. Then, we use conjunct gradient descent to minimize the mean squared error objective function. Finally, we describe each cluster with the highest weighted element corresponding to one particular phrase. Our experimental results indicate that our method can obtain more ldquopurerdquo clusters than other methods.

Accelerated sparse nonnegative matrix factorization for unsupervised feature learning

In this letter, we consider the nonnegative matrix factorization (NMF) problem and several NMF variants. Two approaches based on DC (difference of convex functions) programming and DCA (DC algorithm) are developed. The first approach follows the alternating framework that requires solving, at each iteration, two nonnegativity-constrained least squares subproblems for which DCA-based schemes are investigated. The convergence property of the proposed algorithm is carefully studied. We show that with suitable DC decompositions, our algorithm generates most of the standard methods for the NMF problem. The second approach directly applies DCA on the whole NMF problem. Two algorithms-one computing all variables and one deploying a variable selection strategy-are proposed. The proposed methods are then adapted to solve various NMF variants, including the nonnegative factorization, the smooth regularization NMF, the sparse regularization NMF, the multilayer NMF, the convex/convex-hull NMF, and the symmetric NMF. We also show that our algorithms include several existing methods for these NMF variants as special versions. The efficiency of the proposed approaches is empirically demonstrated on both real-world and synthetic data sets. It turns out that our algorithms compete favorably with five state-of-the-art alternating nonnegative least squares algorithms.

This demo presents an extremely efficient concept detection system based on a novel bag of words extraction method and sparse ensemble learning. We will show that the presented system can efficiently build the concept detectors upon millions of images, and achieve real-time concept detection on unseen images with the state-of-the-arts accuracy. To do so, we first develop an efficient bag of visual words (BoW) construction method based on sparse non-negative matrix factorization (NMF) and GPU enabled SIFT feature extraction. We then develop a sparse ensemble learning method to build the detection model, which drastically reduces learning time in order of magnitude over traditional methods like Support Vector Machine. The demo video of the system is available at YouTube: http://youtu.be/57obnlCxqAs

Non-negative matrix factorization (NMF) is widely used in feature extraction and dimension reduction fields. Essentially, it is an optimization problem to determine two non-negative low rank matrices \(W_{m \times k}\) and \(H_{k \times n}\) for a given matrix \(A_{m \times n}\), satisfying \(A_{m \times n} \approx W_{m \times k}H_{k \times n}\). In this paper, a novel approach to improve the image decomposing and reconstruction effects by introducing the Singular Value Decomposing (SVD)-based initialization scheme of factor matrices W and H, and another measure called choosing rule to determine the optimum value of factor rank k, are proposed. The input image is first decomposed using SVD to get its singular values and corresponding eigenvectors. Then, the number of main components as the rank value k is extracted. Then, the singular values and corresponding eigenvectors are used to initialize W and H based on selected rank k. Finally, convergent results are obtained using multiplicative and additive update rules. However, iterative NMF algorithms’ convergence is very slow on most platforms limiting its practicality. To this end, a parallel implementation frame of described improved NMF algorithm using CUDA, a tool for algorithms parallelization on massively parallel processors, i.e., many-core graphics processors, is presented. Experimental results show that our approach can get better decomposing effect than traditional NMF implementations and dramatic accelerate rate comparing to serial schemes as well as existing distributed-system implementations.

Nowadays, non-negative matrix factorization (NMF) based cluster analysis for multi-view data shows impressive behavior in machine learning. Usually, multi-view data have complementary information from various views. The main concern behind the NMF is how to factorize the data to achieve a significant clustering solution from these complementary views. However, NMF does not focus to conserve the geometrical structures of the data space. In this article, we intensify on the above issue and evolve a new NMF clustering method with manifold regularization for multi-view data. The manifold regularization factor is exploited to retain the locally geometrical structure of the data space and gives extensively common clustering solution from multiple views. The weight control term is adopted to handle the distribution of each view weight. An iterative optimization strategy depended on multiplicative update rule is applied on the objective function to achieve optimization. Experimental analysis on the real-world datasets are exhibited that the proposed approach achieves better clustering performance than some state-of-the-art algorithms.

Text embedding plays a crucial role in natural language processing (NLP). Among various approaches, nonnegative matrix factorization (NMF) is an effective method for this purpose. However, the standard NMF approach, fundamentally based on the bag-of-words model, fails to utilize the contextual information of documents and may result in a significant loss of semantics. To address this issue, we propose a new NMF scheme incorporating a regularization term based on the Wasserstein metric, which quantifies an approximation of a word-context matrix to leverage semantic information. Since the word-context matrix is typically symmetric and positive definite (SPD), the Wasserstein metric used in the regularization is related to a natural SPD manifold structure. We then build upon this manifold structure to integrate an explicit expression of the Wasserstein metric into the NMF framework. Additionally, we introduce modifications to the gradient computation to adapt three fundamental classes of numerical algorithms from ordinary NMF to tackle the Wasserstein-regularized NMF (NMF-WR) problem. To demonstrate the effectiveness of the NMF-WR model and the proposed algorithms, we conduct experiments on popular datasets, focusing on topic modeling and document clustering. The results indicate that the NMF-WR model exhibits superior performance compared to other conventional NMF models in processing text embeddings. These findings suggest that this novel NMF-WR framework not only enhances semantic representation but also underscores a commitment to the model’s reliability and interpretability.