Local Application of Non Negative Matrix Factorization Algorithm in Face Recognition

Nonnegative Matrix Factorization (NMF) is a widely used technique in many applications such as face recognition, motion segmentation, etc. It approximates the nonnegative data in an original high dimensional space with a linear representation in a low dimensional space by using the product of two nonnegative matrices. In many applications data are often partially corrupted with large additive noise. When the positions of noise are known, some existing variants of N-MF can be applied by treating these corrupted entries as missing values. However, the positions are often unknown in many real world applications, which prevents the usage of traditional NMF or other existing variants of NMF. This paper proposes a Robust Nonnegative Matrix Factorization (RobustNMF) algorithm that explicitly models the partial corruption as large additive noise without requiring the information of positions of noise. In particular, the proposed method jointly approximates the clean data matrix with the product of two nonnegative matrices and estimates the positions and values of outliers/noise. An efficient iterative optimization algorithm with a solid theoretical justification has been proposed to learn the desired matrix factorization. Experimental results demonstrate the advantages of the proposed algorithm.

Lee and Seung proposed nonnegative matrix factorization (NMF) algorithms to decompose patterns and images for structure retrieving. The NMF algorithms have been applied to various optimization problems. However, it is difficult to prove the convergence of this class of learning algorithms. This paper presents the global minima analysis of the NMF algorithms. In the analysis, invariant set is constructed so that the non-divergence of the algorithms can be guaranteed in the set. Using the features of linear equation systems and their solutions, the fixed points and convergence properties of the update algorithms are discussed in detail. The analysis shows that, although the cost function is not convex in both A and X together, it is possible to obtain the global minima from the particular learning algorithms. For different initializations, simulations are presented to confirm the analysis results.

Color face image is composed of color information of different channels. Color image provides more information for face recognition task compared with grey scale image. A novel approach coined block diagonal non-negative matrix factorization (BDNMF) is proposed for color face representation and recognition. The approach employs block diagonal matrix to encode color information of different channels. Block diagonal constraint is imposed on the non-negative matrix factorization algorithm to factorize matrices of different channels simultaneously. And block diagonal non-negative matrix factorization algorithm is exploited to extract facial features. Nearest neighborhood classifier is adopted to identify color face samples. Experimental results on CVL and CMU PIE color face databases verify the effectiveness of the proposed approach.

This paper attempts to represent the mapped data in the radial basis function (RBF) feature space under non-negativity constraints and develops a RBF kernel based non-negative matrix factorization (KNMF-RBF) algorithm. Based on an objective function with Frobenius norm, we obtain the multiplicative update rules of our KNMF-RBF approach using kernel theory and gradient descent method. The proposed KNMF-RBF method is theoretically shown to be convergence by means of the constructed auxiliary function. Our approach is successfully applied to face recognition in the FERET face database. Experimental results demonstrate that the proposed KNMF-RBF method surpasses some kernel based methods. The results are encouraging.

The effects of fatigue on synergy of selected lower limb muscles during running

Nonnegative matrix factorization (NMF) is a widely-used method for multivariate analysis of nonnegative data, the goal of which is decompose a data matrix into a basis matrix and an encoding variable matrix with all of these matrices allowed to have only nonnegative elements. In this paper we present simple algorithms for orthogonal NMF, where orthogonality constraints are imposed on basis matrix or encoding matrix. We develop multiplicative updates directly from the true gradient (natural gradient) in Stiefel manifold, whereas existing algorithms consider additive orthogonality constraints. Numerical experiments on face image data for a image representation task show that our orthogonal NMF algorithm preserves the orthogonality, while the goodness-of-fit (GOF) is minimized. We also apply our orthogonal NMF to a clustering task, showing that it works better than the original NMF, which is confirmed by experiments on several UCI repository data sets.

Non-negative matrix factorization (NMF) approximates a given matrix as a product of two non-negative matrix factors. Multiplicative algorithms deliver reliable results, but they show slow convergence for high-dimensional data and may be stuck away from local minima. Gradient descent methods have better behavior, but only apply to smooth losses. For non-smooth losses such as the Kullback-Leibler (KL) loss, surprisingly, these methods are lacking. In this paper, we propose a first-order primal-dual algorithm for non-negative decomposition problems (one of the two factors is fixed) with the KL distance. All required computations may be obtained in closed form and we provide an efficient heuristic way to select step-sizes. By using alternating optimization, our algorithm readily extends to NMF and, on synthetic or real world data, it is either faster than existing algorithms, or leads to improved local optima, or both.

This article uses the projected gradient method (PG) for a non-negative matrix factorization problem (NMF), where one or both matrix factors must have orthonormal columns or rows. We penalize the orthonormality constraints and apply the PG method via a block coordinate descent approach. This means that at a certain time one matrix factor is fixed and the other is updated by moving along the steepest descent direction computed from the penalized objective function and projecting onto the space of non-negative matrices. Our method is tested on two sets of synthetic data for various values of penalty parameters. The performance is compared to the well-known multiplicative update (MU) method from Ding (2006), and with a modified global convergent variant of the MU algorithm recently proposed by Mirzal (2014). We provide extensive numerical results coupled with appropriate visualizations, which demonstrate that our method is very competitive and usually outperforms the other two methods.

Hyperspectral unmixing is a powerful method of the remote sensing image mining that identifies the constituent materials and estimates the corresponding fractions from the mixture. We consider the application of nonnegative matrix factorization (NMF) for the mining and analysis of spectral data. In this paper, we develop two effective active set type NMF algorithms for hyperspectral unmixing. Because the factor matrices used in unmixing have sparse features, the active set strategy helps reduce the computational cost. These active set type algorithms for NMF is based on an alternating nonnegative constrained least squares (ANLS) and achieve a quadratic convergence rate under the reasonable assumptions. Finally, numerical tests demonstrate that these algorithms work well and that the function values decrease faster than those obtained with other algorithms.

ABSTRACTHyperspectral unmixing (HU) has drawn remarkable attention because it can decompose mixed pixels into a set of endmembers and abundance fractions. And the nonnegative matrix factorization (NMF) algorithm has been widely used for solving hyperspectral spectral unmixing problem. This letter proposes a Hessian graph regularized NMF (HGNMF) algorithm, which relies upon the construction of a Hessian graph representation, to solve the hyperspectral unmixing problem. According to the HGNMF algorithm, the smoothness in the estimated abundance maps is well promoted. Moreover, the optimized problem of HGNMF algorithm is also solved by employing the multiplicative updating rules. Compared to other algorithms, the proposed HGNMF algorithm demonstrates lots of advantages based on the simulated results of the synthetic data and real data sets.

Face recognition is a very old and popular biometric problem. In the last two decades, several challenges occur in face recognition such as expression, occlusion, illumination, pose variation, and low-quality image. Also, several techniques exist, such as Principle Component Analysis (PCA), Linear Discriminate Analysis (LDA), Local Non-Negative Matrix Factorization (LNME), Independent Component Analysis (ICA), and other variants, which finds good results under constrained scenarios. But, they give poor accuracy in an unconstrained scenarios such as illumination, pose variation, and occlusion. As Face recognition biometrics has gained great importance in occlusion condition, for example, a terrorists cover his face using facial accessories like sunglasses, scarves, or masks. This occlusion face recognition is categorized into two parts. 1) Feature extraction without occlusion detection face recognition. 2) Feature extraction with occlusion detection face recognition 3) occlusion recovery face recognition. Feature-based and fractal-based methods consider features around the eyes, nose, or mouth region to be used in the recognition face of the algorithm. This paper describes the different techniques and databases to handle the partial occlusion problem.

In environmental data analysis, source apportionment can be an important approach to extract useful information that might otherwise be hidden within the data. The United States Environmental Protection Agency (EPA) has developed an open-source python package, the Environmental Source Apportionment Toolkit (ESAT), which enables source apportionment modeling and error estimation workflows. ESAT is intended to replace Positive Matrix Factorization v5 (PMF5) that has substantial data size limitations. ESAT is currently in alpha testing with development plans for enhanced functionality and support of large datasets, High-performance Computing (HPC) execution through a command line interface (CLI), and a standalone desktop graphical user interface (GUI). The alpha product of ESAT is publicly available and offers a complete application programming interface (API) to replicate the workflows and functionality of PMF5, with examples provided through Jupyter Notebooks. The ESAT computing module currently contains two non-negative matrix factorization (NMF) algorithms for model training, with the module designed for other algorithms to be easily added. The two algorithms currently available are the least-squares NMF (LS-NMF) and weighted-semi NMF (WS-NMF). Each algorithm offers different benefits depending on project or data requirements. The ESAT python codebase has been optimized to run in a highly parallelized manner, with most of the numerical computations implemented in Rust, a low-level language comparable in performance to C. ESAT replicates the model error estimation methods of PMF5, namely bootstrap, displacement, and a hybrid method. To facilitate experimentation and testing, ESAT contains a synthetic dataset generator and model simulator that can evaluate how well ESAT can recreate synthetic factors and contributions. Continuous development of new features are tested and added to the python package on a regular basis. One such feature is the addition of an uncertainty perturbation workflow, which will run a collection of models while slightly perturbing the uncertainty matrix, and then evaluating the impact on the solution profiles and contributions. The alpha version of the ESAT python package is available for installation from pypi at https://pypi.org/project/esat/. Further testing and development of the alpha version will proceed to a full release in late 2025. The development of a GUI desktop application is currently planned to begin after the ESAT full release.

Nonnegative matrix factorization (NMF) is a popular method for multivariate analysis of nonnegative data, the goal of which is decompose a data matrix into a product of two factor matrices with all entries in factor matrices restricted to be nonnegative. NMF was shown to be useful in a task of clustering (especially document clustering). In this paper we present an algorithm for orthogonal nonnegative matrix factorization, where an orthogonality constraint is imposed on the nonnegative decomposition of a term-document matrix. We develop multiplicative updates directly from true gradient on Stiefel manifold, whereas existing algorithms consider additive orthogonality constraints. Experiments on several different document data sets show our orthogonal NMF algorithms perform better in a task of clustering, compared to the standard NMF and an existing orthogonal NMF.KeywordsNonnegative Matrix FactorizationDocument ClusterOrthogonality ConstraintMultiplicative UpdateProbabilistic Latent Semantic IndexingThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Nonnegative matrix factorization (NMF) is a data analysis technique used in a great variety of applications such as text mining, image processing, hyperspectral data analysis, computational biology, and clustering. In this letter, we consider two well-known algorithms designed to solve NMF problems: the multiplicative updates of Lee and Seung and the hierarchical alternating least squares of Cichocki et al. We propose a simple way to significantly accelerate these schemes, based on a careful analysis of the computational cost needed at each iteration, while preserving their convergence properties. This acceleration technique can also be applied to other algorithms, which we illustrate on the projected gradient method of Lin. The efficiency of the accelerated algorithms is empirically demonstrated on image and text data sets and compares favorably with a state-of-the-art alternating nonnegative least squares algorithm.

Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have become popular in modern data analysis problems due to their computational efficiency. Even though they have proved useful for many statistical models, the application of SG-MCMC to non-negative matrix factorization (NMF) models has not yet been extensively explored. In this study, we develop two parallel SG-MCMC algorithms for a broad range of NMF models. We exploit the conditional independence structure of the NMF models and utilize a stratified sub-sampling approach for enabling parallelization. We illustrate the proposed algorithms on an image restoration task and report encouraging results.