Finding Rule-Interpretable Non-Negative Data Representation

Non-negative matrix factorization (NMF) was introduced by Paatero and Tapper in 1994 and it was a general way of reducing the dimension of the matrix with non-negative entries. Non-negative matrix factorization is very useful in many data analysis applications such as character recognition, text mining, and others. This paper aims to study the application in Chinese character recognition using non-negative matrix factorization. Python was used to carry out the LU factorization and non-negative matrix factorization of a Chinese character in Boolean Matrix. Preliminary analysis confirmed that the data size of and and are chosen for the NMF of the Boolean matrix. In this project, one hundred printed Chinese characters were selected, and all the Chinese characters can be categorized into ten categories according to the number of strokes , for . The Euclidean distance between the Boolean matrix of a Chinese character and the matrix after both LU factorization and NMF is calculated for further analysis. Paired t-test confirmed that the factorization of Chinese characters in the Boolean matrix using NMF is better than the LU factorization. Finally, ten handwritten Chinese characters were selected to test whether the program is able to identify the handwritten and the printed Chinese characters. Experimental results showed that 70% of the characters can be recognized via the least Euclidean distance obtained. NMF is suitable to be applied in Chinese character recognition since it can reduce the dimension of the image and the error between the original Boolean matrix and after NMF is less than 5%.

Non-negative matrix factorization (NMF) has been one of the most popular methods for feature learning in the field of machine learning and computer vision. Most existing works directly apply NMF on high-dimensional image datasets for computing the effective representation of the raw images. However, in fact, the common essential information of a given class of images is hidden in their low rank parts. For obtaining an effective low-rank data representation, we in this paper propose a non-negative low-rank matrix factorization (NLMF) method for image clustering. For the purpose of improving its robustness for the data in a manifold structure, we further propose a graph regularized NLMF by incorporating the manifold structure information into our proposed objective function. Finally, we develop an efficient alternating iterative algorithm to learn the low-dimensional representation of low-rank parts of images for clustering. Alternatively, we also incorporate robust principal component analysis into our proposed scheme. Experimental results on four image datasets reveal that our proposed methods outperform four representative methods.

Meta-learning is originally developed for supervised learning to enable models to generalize across tasks by leveraging prior experience. However, its potential in the unsupervised learning domain remains largely underexplored. To demonstrate the feasibility of meta-learning in unsupervised settings, we apply it to Non-negative Matrix Factorization (NMF), a widely used technique for decomposing non-negative data matrices into interpretable, lower-dimensional representations. While NMF has found applications in topic modeling, image processing, bioinformatics, and recommendation systems, it faces persistent challenges such as rank selection, optimization stability, uniqueness, and computational efficiency. Traditional approaches primarily focus on improving initialization strategies to enhance convergence and generalization. However, these methods often fail to exploit structural similarities across tasks. In this paper, we introduce a meta-learning paradigm for NMF that systematically learns optimal factorization parameters from small-scale tasks and transfers this knowledge to improve learning on larger tasks. By discovering fine structures in small tasks and leveraging them to guide factorization on more complex datasets, our approach directs the search process toward a more optimal and structured search space, reducing the risk of suboptimal solutions and improving model robustness. This meta-unsupervised learning framework enhances NMF's ability to uncover meaningful patterns while maintaining adaptability across different domains. Additionally, we evaluate the model under noisy conditions and demonstrate its robustness by filtering noise over learning epochs, further enhancing its interpretability and stability. By integrating meta-learning principles, our method not only improves optimization stability but also enhances interpretability and generalizability, bridging the gap between NMF-based models and advanced autonomous (unsupervised) learning strategies.

Discriminative concept factorization for data representation

Improving non-negative matrix factorizations through structured initialization

Obtaining an optimum data representation is a challenging issue that arises in many intellectual data processing techniques such as data mining, pattern recognition, and gene clustering. Many existing methods formulate this problem as a nonnegative matrix factorization (NMF) approximation problem. The standard NMF uses the least square loss function, which is not robust to outlier points and noises and fails to utilize prior label information to enhance the discriminability of representations. In this study, we develop a novel matrix factorization method called robust semisupervised nonnegative local coordinate factorization by integrating robust NMF, a robust local coordinate constraint, and local spline regression into a unified framework. We use the l2,1 norm for the loss function of the NMF and a local coordinate constraint term to make our method insensitive to outlier points and noises. In addition, we exploit the local and global consistencies of sample labels to guarantee that data representation is compact and discriminative. An efficient multiplicative updating algorithm is deduced to solve the novel loss function, followed by a strict proof of the convergence. Several experiments conducted in this study on face and gene datasets clearly indicate that the proposed method is more effective and robust compared to the state-of-the-art methods.

Nonnegative matrix factorization (NMF) provides a way for finding a part-based representation of nonnegative data. An important property of NMF is that it can produce a sparse representation of the data; however, in some applications, especially in text clustering, the sparse representation always consists of separated words, which cannot explicitly express the meaning of the basis vector. This paper presents a new descriptive clustering algorithm based on NMF, called DC-NMF that can avoid this separated word problem. In our proposed method, we embrace the phrase-by-document matrix in addition to the commonly used term-by-document matrix. Then, we use conjunct gradient descent to minimize the mean squared error objective function. Finally, we describe each cluster with the highest weighted element corresponding to one particular phrase. Our experimental results indicate that our method can obtain more ldquopurerdquo clusters than other methods.

Recently, network embedding has received a large amount of attention in network analysis. Although some network embedding methods have been developed from different perspectives, on one hand, most of the existing methods only focus on leveraging the plain network structure, ignoring the abundant attribute information of nodes. On the other hand, for some methods integrating the attribute information, only the lower-order proximities (e.g., microscopic proximity structure) are taken into account, which may suffer if there exists the sparsity issue and the attribute information is noisy. To overcome this problem, the attribute information and mesoscopic community structure are utilized. In this article, we propose a novel network embedding method termed Attributed Network Embedding with Micro-Meso structure, which is capable of preserving both the attribute information and the structural information including the microscopic proximity structure and mesoscopic community structure. In particular, both the microscopic proximity structure and node attributes are factorized by Nonnegative Matrix Factorization (NMF), from which the low-dimensional node representations can be obtained. For the mesoscopic community structure, a community membership strength matrix is inferred by a generative model (i.e., BigCLAM) or modularity from the linkage structure, which is then factorized by NMF to obtain the low-dimensional node representations. The three components are jointly correlated by the low-dimensional node representations, from which two objective functions (i.e., ANEM_B and ANEM_M) can be defined. Two efficient alternating optimization schemes are proposed to solve the optimization problems. Extensive experiments have been conducted to confirm the superior performance of the proposed models over the state-of-the-art network embedding methods.

Manifold learning algorithms have been demonstrated to be effective for hyperspectral data dimension reduction (DR). However, the low dimensional feature representation resulted by traditional manifold learning algorithms could not preserve the nonnegative property of the hyperspectral data, which leads inconsistency with the psychological intuition of “combining parts to form a whole”. In this paper, we introduce a nonnegative discriminative manifold learning (NDML) algorithm for hyperspectral data DR, which yields a discriminative and low dimensional feature representation, with psychological and physical evidence in the human brain. Our method benefits from both the nonnegative matrix factorization (NMF) algorithm and the discriminative manifold learning (DML) algorithm. We apply the NDML algorithm to hyperspectral remote sensing image classification on HYDICE dataset. Experimental results confirm the efficiency of the proposed NDML algorithm, compared with some existing manifold learning based DR methods.Keywordsmanifold learningnonnegative matrix factorizationdimension reductionhypsrspectral data

Nonnegative Matrix Factorization (NMF), as a popular feature extraction technique, has recently attracted increasing attentions in high-dimensional data analysis, due to the strong ability of dimension reduction and semantic representation. In order to utilize the partial label information in practice, several semi-supervised NMF methods have been proposed. However, existing semi-supervised NMF variants only consider the global label constraint of data, but ignore the local label information embedded in geometric structure, which is very crucial for data representation. To address the above issue, we propose an improved NMF method, namely local and global regularized semi-supervised NMF (LGNMF), by considering the local and global label constraints simultaneously. Specifically, the local label constraint is depicted in a graph, which is pre-computed by metric learning. In addition, to explore the global label information, we construct an indicator matrix to restrict the coefficient matrix in LGNMF. For the formulated LGNMF, a multiplicative update rule (MUR) is developed. Extensive experiments on several real datasets demonstrate the superiority of the proposed method over the state-of-the-art methods in terms of clustering accuracy and normalized mutual information.

In this paper, we present a novel method for obtaining a low dimensional representation of a complex brain network that: (1) can be interpreted in a neurobiologically meaningful way, (2) emphasizes group differences by accounting for label information, and (3) captures the variation in disease subtypes/severity by respecting the intrinsic manifold structure underlying the data. Our method is a supervised variant of non-negative matrix factorization (NMF), and achieves dimensionality reduction by extracting an orthogonal set of subnetworks that are interpretable, reconstructive of the original data, and also discriminative at the group level. In addition, the method includes a manifold regularizer that encourages the low dimensional representations to be smooth with respect to the intrinsic geometry of the data, allowing subjects with similar disease-severity to share similar network representations. While the method is generalizable to other types of non-negative network data, in this work we have used structural connectomes (SCs) derived from diffusion data to identify the cortical/subcortical connections that have been disrupted in abnormal neurological state. Experiments on a traumatic brain injury (TBI) dataset demonstrate that our method can identify subnetworks that can reliably classify TBI from controls and also reveal insightful connectivity patterns that may be indicative of a biomarker.

Nonnegative matrix factorization (NMF), which can lead to nonsubtractive parts-based representation, has been demonstrated to be effective for dimensionality reduction of hyperspectral imagery (HSI). However, existing NMF methods applied to HSI use only a single spectral feature and do not take into consideration spatial information, such as texture or morphological features, while it has been widely acknowledged that exploiting multiple features can improve performance. Consequently, a variant of orthogonal NMF, which can not only achieve a nonnegative factorization but also exploit the complementary information that arises among heterogeneous features, is proposed for hyperspectral dimensionality reduction. The proposed method, which couples orthogonal NMF with a previous multiple-features-combining algorithm, yields a discriminative low-dimensional feature representation that matches the intuition that parts should sum to produce a whole. An efficient multiplicative updating procedure is derived, and its local convergence is guaranteed theoretically. Experimental results on two hyperspectral data sets demonstrate the effectiveness of the proposed method.

Nonnegative matrix factorization (NMF) is known as a parts-based linear representation for nonnegative data. This method has been applied for blind source separation (BSS) when the sources are nonnegative. This paper presents a new NMF method for independent component analysis (ICA), which is useful for BSS without the nonnegativity constraint. Using this method, we transform the sources by their cumulative distribution functions (CDFs) and perform the nonparametric quantization to construct a nonnegative matrix where each entry represents the joint probability density of two transformed signals. The NMF procedure is accordingly realized to find the ICA demixing matrix. The independence between sources is maximized towards attaining the uniformity in the joint probability density. In the experiments on the separation of signal and music signals, we show the effectiveness of the proposed NMF-ICA compared to the infomax ICA and FastICA algorithms.

Sparse nonnegative matrix factorization with ℓ0-constraints

Non-negative matrix factorization (NMF) has found numerous applications, due to its ability to provide interpretable decompositions. Perhaps surprisingly, existing results regarding its uniqueness properties are rather limited, and there is much room for improvement in terms of algorithms as well. Uniqueness and computational aspects of NMF are revisited here from a geometrical point of view. Both symmetric and asymmetric NMF are considered, the former being tantamount to element-wise non-negative square-root factorization of positive semidefinite matrices. New and insightful uniqueness results are derived, e.g., it is shown that a sufficient condition for uniqueness is that the conic hull of the latent factors is a superset of a particular second-order cone. Checking this is shown to be NP-complete; yet it offers insights on latent sparsity, as is also shown in a new necessary condition, to a smaller extent. On the computational side, a new efficient algorithm for symmetric NMF is proposed which uses Procrustes rotations. Simulation results show promising performance with respect to the state-of-art. The new algorithm is also applied to a clustering problem for co-authorship data, yielding meaningful and interpretable results.