Semi-supervised graph regularized nonnegative matrix factorization with local coordinate for image representation

Nonnegative Matrix Factorization (NMF), as a popular feature extraction technique, has recently attracted increasing attentions in high-dimensional data analysis, due to the strong ability of dimension reduction and semantic representation. In order to utilize the partial label information in practice, several semi-supervised NMF methods have been proposed. However, existing semi-supervised NMF variants only consider the global label constraint of data, but ignore the local label information embedded in geometric structure, which is very crucial for data representation. To address the above issue, we propose an improved NMF method, namely local and global regularized semi-supervised NMF (LGNMF), by considering the local and global label constraints simultaneously. Specifically, the local label constraint is depicted in a graph, which is pre-computed by metric learning. In addition, to explore the global label information, we construct an indicator matrix to restrict the coefficient matrix in LGNMF. For the formulated LGNMF, a multiplicative update rule (MUR) is developed. Extensive experiments on several real datasets demonstrate the superiority of the proposed method over the state-of-the-art methods in terms of clustering accuracy and normalized mutual information.

Obtaining an optimum data representation is a challenging issue that arises in many intellectual data processing techniques such as data mining, pattern recognition, and gene clustering. Many existing methods formulate this problem as a nonnegative matrix factorization (NMF) approximation problem. The standard NMF uses the least square loss function, which is not robust to outlier points and noises and fails to utilize prior label information to enhance the discriminability of representations. In this study, we develop a novel matrix factorization method called robust semisupervised nonnegative local coordinate factorization by integrating robust NMF, a robust local coordinate constraint, and local spline regression into a unified framework. We use the l2,1 norm for the loss function of the NMF and a local coordinate constraint term to make our method insensitive to outlier points and noises. In addition, we exploit the local and global consistencies of sample labels to guarantee that data representation is compact and discriminative. An efficient multiplicative updating algorithm is deduced to solve the novel loss function, followed by a strict proof of the convergence. Several experiments conducted in this study on face and gene datasets clearly indicate that the proposed method is more effective and robust compared to the state-of-the-art methods.

Incremental semi-supervised graph learning NMF with block-diagonal

Label consistency-based deep semisupervised NMF for tumor recognition

Imaging genetics is an approach that explores the underlying mechanisms of brain disorders such as Alzheimer's disease (AD) by analyzing the correlation between neuroimaging and genetic data. Traditional non-negative matrix factorization (NMF) algorithms are based on linear assumptions, which limits the potential of nonlinear feature extraction among multi-omics data. This study proposes a novel joint-connectivity-based deep semi-supervised non-negative matrix factorization (JCB-DSNMF) model to overcome this limitation and incorporate prior knowledge from both within and between different modalities of data. The model effectively integrates physiological constraints such as connectivity to identify regions of interest (ROI), risk genes, and risk SNP loci associated with AD patients. JCB-DSNMF outperformed other NMF-based algorithms, such as JDSNMF and NMF, in identifying and predicting biologically relevant biomarkers closely related to AD from essential modules. The accuracy of the selected features was further validated by constructing a diagnostic model with high classification accuracy, achieving an AUC value of 0.8621 on the test set. In particular, the brain region Putamen_L and the gene RALGAPB achieved AUC values of 0.903 and 0.924, respectively, highlighting the importance of these features in early AD diagnosis.

To develop a sound-monitoring system for checking machine health, we propose an algorithm for extracting novel sound (i.e. abnormal sound) from the monaural signal including the learned noise. Conventional approaches use non-negative matrix factorization (NMF) or semi-supervised NMF (SSNMF) for separating the novel sound and the learned noise. However, the conventional approaches assume the low-rankness of the novel sound, which is not necessarily satisfied. In addition, SSNMF has a hyper parameter that cannot be tuned i.e. the number of atoms corresponding to the novel sound. To solve the problems of NMF and SSNMF, our algorithm applies a constraint for the novel sound, i.e. only the non-negativity without low-rankness. Our algorithm can be interpreted as a simultaneous optimization version revised from the state-of-the-art non-negative matrix underapproximation (NMU). Experimental results indicate that the extraction performance of the novel sound is higher than conventional approaches.

Non-negative matrix factorization (NMF) is widely used in feature extraction and dimension reduction fields. Essentially, it is an optimization problem to determine two non-negative low rank matrices \(W_{m \times k}\) and \(H_{k \times n}\) for a given matrix \(A_{m \times n}\), satisfying \(A_{m \times n} \approx W_{m \times k}H_{k \times n}\). In this paper, a novel approach to improve the image decomposing and reconstruction effects by introducing the Singular Value Decomposing (SVD)-based initialization scheme of factor matrices W and H, and another measure called choosing rule to determine the optimum value of factor rank k, are proposed. The input image is first decomposed using SVD to get its singular values and corresponding eigenvectors. Then, the number of main components as the rank value k is extracted. Then, the singular values and corresponding eigenvectors are used to initialize W and H based on selected rank k. Finally, convergent results are obtained using multiplicative and additive update rules. However, iterative NMF algorithms’ convergence is very slow on most platforms limiting its practicality. To this end, a parallel implementation frame of described improved NMF algorithm using CUDA, a tool for algorithms parallelization on massively parallel processors, i.e., many-core graphics processors, is presented. Experimental results show that our approach can get better decomposing effect than traditional NMF implementations and dramatic accelerate rate comparing to serial schemes as well as existing distributed-system implementations.

The significance of Non-negative Matrix Factorization ((NMF) in the field of automatic text summarization is rapidly increasing due to its interpretation and storage capabilities. Interpretation defines the ease at which the structure of high dimensional data can be understood. While, storage capability relates to the extent of data reduction process achieved by NMF. The parametric values that serve as input to the NMF process include initialization method, rank of factorization, sparseness measure and maximum iteration. These inputs are vital to the output produced by NMF. These parameters of NMF were not been considered in the existing literature on Automatic Text Summarization (ATS). This paper sheds light specifically on sparseness of NMF when applied to ATS and the impact it makes to the quality of the summary generated. In this study on NMF algorithm that supports sparseness is treated with various degree of sparsity and its performance impact on text summary generated is compared with other NMF algorithms without sparseness constraints.

Hyperspectral unmixing (HU) is a research hotspot of hyperspectral remote sensing technology. As a classical HU method, the nonnegative matrix factorization (NMF) unmixing method can decompose an observed hyperspectral data matrix into the product of two nonnegative matrices, i.e., endmember and abundance matrices. Because the objective function of NMF is the traditional least-squares function, NMF is sensitive to noise. In order to improve the robustness of NMF, this paper proposes a maximum likelihood estimation (MLE) based NMF model (MLENMF) for unmixing of hyperspectral images (HSIs), which substitutes the least-squares objective function in traditional NMF by a robust MLE-based loss function. Experimental results on a simulated and two widely used real hyperspectral data sets demonstrate the superiority of our MLENMF over existing NMF methods.

In order to improve the speech enhancement effect and overcome the weakness of falling into the local optimum of the traditional nonnegative matrix factorization, this paper presents speech enhancement methodology based on nonnegative matrix factorization combined with the improved genetic algorithm. The methodology consists of training stage and enhancement stage. In the training stage, instead of gradient descent, iterative learning is achieved by selecting several directions in super flat close to deflector cone as the mutation. By adding the mutation operator and simulated annealing operator to NMF, the proposed algorithm is used to acquire the accurate noise dictionary as prior information. In the de-noising stage, NMF-IGA is used to evaluate the dictionary matrix and the activation matrix of the de-noised speech. Then using the dictionary of de-noised speech to get the enhanced speech spectrogram and reconstruct the enhanced speech. It is verified that the proposed algorithm yields less residual noise and better speech quality than traditional nonnegative matrix factorization and Bayes nonnegative matrix factorization algorithm.

Non-negative matrix factorization (NMF) is an unsupervised learning method that can be exploited for parts-based image representation due to non-negativity constraints. However, singer-layer NMF cannot capture the latent hierarchical structure features from images, while deep features play more important roles in the image representation and recognition tasks. To overcome the limitation of NMF, this paper proposes a novel deep grouped NMF (DGNMF) approach to learn different level attributes of the data. It is interesting that DGNMF approach automatically makes the data from distinct classes share different basis images and the feature vectors among different classes are mutually orthogonal at the same layer. Meanwhile, to preserve the local information, our DGNMF model establishes the objective function with graph regularization, and its optimization problem is solved using gradient descent method. The developed DGNMF algorithm is proved to be convergent and is finally evaluated on face datasets for classification. Compared with some state-of-the-art deep NMF variants, the results demonstrate the proposed DGNMF algorithm achieves surpassing performances using different layer features.

Traditional Nonnegative Matrix Factorization (NMF) is a linear and unsupervised algorithm. This would limit the classification power of NMF for the complicated data. To overcome the above limitations of NMF, this paper proposes a novel supervised and nonlinear NMF algorithm based on kernel theory and discriminant analysis. We incorporate the class label information into the decomposition of NMF in the Reproducing Kernel Hilbert Space (RKHS). A new iterative algorithm for NMF is derived and the objective function is non-increasing under the update rules. The proposed method is evaluated on the ORL and Yale face databases. The experimental results demonstrate the the proposed method is superior to the state of-the-art algorithms.

Characterizing sediment sources by non-negative matrix factorization of detrital geochronological data

Non-negative matrix factorization (NMF) has been one of the most popular methods for feature learning in the field of machine learning and computer vision. Most existing works directly apply NMF on high-dimensional image datasets for computing the effective representation of the raw images. However, in fact, the common essential information of a given class of images is hidden in their low rank parts. For obtaining an effective low-rank data representation, we in this paper propose a non-negative low-rank matrix factorization (NLMF) method for image clustering. For the purpose of improving its robustness for the data in a manifold structure, we further propose a graph regularized NLMF by incorporating the manifold structure information into our proposed objective function. Finally, we develop an efficient alternating iterative algorithm to learn the low-dimensional representation of low-rank parts of images for clustering. Alternatively, we also incorporate robust principal component analysis into our proposed scheme. Experimental results on four image datasets reveal that our proposed methods outperform four representative methods.

Non-negative matrix factorization (NMF) has proved effective in many clustering and classification tasks. The classic ways to measure the errors between the original and the reconstructed matrix are $l_2$ distance or Kullback-Leibler (KL) divergence. However, nonlinear cases are not properly handled when we use these error measures. As a consequence, alternative measures based on nonlinear kernels, such as correntropy, are proposed. However, the current correntropy-based NMF only targets on the low-level features without considering the intrinsic geometrical distribution of data. In this paper, we propose a new NMF algorithm that preserves local invariance by adding graph regularization into the process of max-correntropy-based matrix factorization. Meanwhile, each feature can learn corresponding kernel from the data. The experiment results of Caltech101 and Caltech256 show the benefits of such combination against other NMF algorithms for the unsupervised image clustering.