Orthogonal nonnegative matrix t-factorizations for clustering

Non-negative matrix factorization (NMF) has been one of the most popular methods for feature learning in the field of machine learning and computer vision. Most existing works directly apply NMF on high-dimensional image datasets for computing the effective representation of the raw images. However, in fact, the common essential information of a given class of images is hidden in their low rank parts. For obtaining an effective low-rank data representation, we in this paper propose a non-negative low-rank matrix factorization (NLMF) method for image clustering. For the purpose of improving its robustness for the data in a manifold structure, we further propose a graph regularized NLMF by incorporating the manifold structure information into our proposed objective function. Finally, we develop an efficient alternating iterative algorithm to learn the low-dimensional representation of low-rank parts of images for clustering. Alternatively, we also incorporate robust principal component analysis into our proposed scheme. Experimental results on four image datasets reveal that our proposed methods outperform four representative methods.

In this work, we will consider the dimension reduction of the set of time series, such as economic data, to find the meaningful basis vector for the set of data, and indicate which data use which basis vector. Usually each of the time series is analyzed independently in economics but here we will analyze the set of time series simultaneously. Since some of the economic data are measured as positive values and we want to decompose them as a mixture of the parts, we will apply non-negative matrix factorization to the economic data. Non-negative matrix factorization can compress dimensions by approximating a non-negative matrix with the product of two non-negative matrices. The two non-negative matrices are called the coefficient matrix and the basis matrix, and the basis matrix can be considered as a dimensionally compressed matrix. If the standard non-negative matrix factorization is used for economic data, the basis matrix may not be smooth. We think that the basis vectors should be smooth except a few special economical incidents. In the proposed method, a Graph-based non-negative matrix factorization is introduced to regularize the basis matrix of the time series. A path graph for representing the time series of economic data is incorporated into the non-negative matrix factorization as regularization. As a result, basis vectors that maintains the time series of economic data are decomposed. Furthermore, we propose to introduce a sparsity in the non-negative matrix factorization. Traditionally, the sparsity incorporated into non-negative matrix factorization has been used for basis vectors. However, the proposed method introduces the sparsity for coefficient vectors. Thus the proposed method, which simultaneously incorporates the sparsity for the coefficient vectors and the smoothness for the basis vectors, can extract the smooth basis vectors and the original economical data are approximated as the weighted sum of the few bases vectors. This allows us to discover economic trends and the best-fit trends for each data at the same time.

Profile-Decomposing Output of Multi-Channel Odor Sensor Array

Nonnegative matrix factorization (NMF) is the problem of approximating an input nonnegative matrix, $V$, as the product of two smaller nonnegative matrices, $W$ and $H$. In this paper, we introduce a general framework to design multiplicative updates (MU) for NMF based on $\beta$-divergences ($\beta$-NMF) with disjoint equality constraints, and with penalty terms in the objective function. By disjoint, we mean that each variable appears in at most one equality constraint. Our MU satisfy the set of constraints after each update of the variables during the optimization process, while guaranteeing that the objective function decreases monotonically. We showcase this framework on three NMF models, and show that it competes favorably the state of the art: (1)~$\beta$-NMF with sum-to-one constraints on the columns of $H$, (2) minimum-volume $\beta$-NMF with sum-to-one constraints on the columns of $W$, and (3) sparse $\beta$-NMF with $\ell_2$-norm constraints on the columns of $W$.

Positron emission tomography (PET) is an important imaging technique to visualize a number of functions in the brain or human body. For reconstructing PET images from the sinogram data, an inverse problem has to be solved using numerical optimizations such as expectation-maximization (EM)-based methods. However, the standard EM method suffers from measurement noise added in the sinogram data. In this paper, we propose a new simultaneous PET image reconstruction and parts extraction method using constrained non-negative matrix factorization. In contrast that the many existing methods reconstruct a single PET image independently, we reconstruct the time-series of PET images simultaneously from the time-series of sinograms using non-negative matrix factorization. Furthermore, we impose the smoothness constraint for the temporal feature, and the exclusive LASSO-based sparseness constraint for the spatial feature for robust image reconstruction and physically meaningful feature extraction.

Non-negative Matrix Factorization (NMF) has been frequently applied to image feature extraction and clustering. Especially in image clustering tasks, it can achieve the similar or better performance than most of the matrix factorization algorithms due to its parts-based representations in the brain. However, the features extracted by NMF are not sparse and localized enough and the error of factorization is not small enough. Semi-tensor product of matrices (STP) is a novel operation of matrix multiplication, it is a generalization of the conventional matrix product for allowing the dimensions of factor matrices to be unequal. STP can manage the data hierarchically and the inverse process of STP can separate the data hierarchically. Based on this character of STP, we propose the Non-Negative Matrix Semi-Tensor Factorization (NMSTF). In this algorithm, we use the inverse process of Semi-Tensor Product of matrices for non-negative matrix factorization. This algorithm effectively optimizes the above two problems in NMF. While achieving similar even better performance on image clustering tasks, the size of features extracted by STNMF is at least 50 % smaller than the ones’ extracted by NMF and the error of factorization reduces 30 % in average.

Guitar string separation is a novel and complicated task. Guitar notes are not pure steady-state signals, hence, we hypothesize that neither Non-Negative Matrix Factorization (NMF) nor Non-Negative Matrix Factor Deconvolution (NMFD) are optimal for separating them. Therefore, we separate steady-state and transient parts using Harmonic-Percussive Separation (HPS) as a preprocessing step. Then, we use NMF for factorizing the harmonic part and NMFD for deconvolving the percussive part. We make use of a hexaphonic guitar dataset which allows for objective evaluation. In addition, we compare several types of time-frequency mask and introduce an intuitive way to combine a binary mask with a ratio mask. We show that the HPS mask type has an effect on source estimation. Our proposed method achieved results comparable to NMF without HPS. Finally, we show that the optimal mask at the final separation stage depends on the estimation algorithm.

Non-negative matrix factorization (NMF) is widely used in feature extraction and dimension reduction fields. Essentially, it is an optimization problem to determine two non-negative low rank matrices \(W_{m \times k}\) and \(H_{k \times n}\) for a given matrix \(A_{m \times n}\), satisfying \(A_{m \times n} \approx W_{m \times k}H_{k \times n}\). In this paper, a novel approach to improve the image decomposing and reconstruction effects by introducing the Singular Value Decomposing (SVD)-based initialization scheme of factor matrices W and H, and another measure called choosing rule to determine the optimum value of factor rank k, are proposed. The input image is first decomposed using SVD to get its singular values and corresponding eigenvectors. Then, the number of main components as the rank value k is extracted. Then, the singular values and corresponding eigenvectors are used to initialize W and H based on selected rank k. Finally, convergent results are obtained using multiplicative and additive update rules. However, iterative NMF algorithms’ convergence is very slow on most platforms limiting its practicality. To this end, a parallel implementation frame of described improved NMF algorithm using CUDA, a tool for algorithms parallelization on massively parallel processors, i.e., many-core graphics processors, is presented. Experimental results show that our approach can get better decomposing effect than traditional NMF implementations and dramatic accelerate rate comparing to serial schemes as well as existing distributed-system implementations.

Hyperspectral unmixing (HU) is a research hotspot of hyperspectral remote sensing technology. As a classical HU method, the nonnegative matrix factorization (NMF) unmixing method can decompose an observed hyperspectral data matrix into the product of two nonnegative matrices, i.e., endmember and abundance matrices. Because the objective function of NMF is the traditional least-squares function, NMF is sensitive to noise. In order to improve the robustness of NMF, this paper proposes a maximum likelihood estimation (MLE) based NMF model (MLENMF) for unmixing of hyperspectral images (HSIs), which substitutes the least-squares objective function in traditional NMF by a robust MLE-based loss function. Experimental results on a simulated and two widely used real hyperspectral data sets demonstrate the superiority of our MLENMF over existing NMF methods.

Analysis of two-mode network data using nonnegative matrix factorization

Nonnegative matrix factorization and I-divergence alternating minimization

In this letter, we consider the nonnegative matrix factorization (NMF) problem and several NMF variants. Two approaches based on DC (difference of convex functions) programming and DCA (DC algorithm) are developed. The first approach follows the alternating framework that requires solving, at each iteration, two nonnegativity-constrained least squares subproblems for which DCA-based schemes are investigated. The convergence property of the proposed algorithm is carefully studied. We show that with suitable DC decompositions, our algorithm generates most of the standard methods for the NMF problem. The second approach directly applies DCA on the whole NMF problem. Two algorithms-one computing all variables and one deploying a variable selection strategy-are proposed. The proposed methods are then adapted to solve various NMF variants, including the nonnegative factorization, the smooth regularization NMF, the sparse regularization NMF, the multilayer NMF, the convex/convex-hull NMF, and the symmetric NMF. We also show that our algorithms include several existing methods for these NMF variants as special versions. The efficiency of the proposed approaches is empirically demonstrated on both real-world and synthetic data sets. It turns out that our algorithms compete favorably with five state-of-the-art alternating nonnegative least squares algorithms.

The significance of Non-negative Matrix Factorization ((NMF) in the field of automatic text summarization is rapidly increasing due to its interpretation and storage capabilities. Interpretation defines the ease at which the structure of high dimensional data can be understood. While, storage capability relates to the extent of data reduction process achieved by NMF. The parametric values that serve as input to the NMF process include initialization method, rank of factorization, sparseness measure and maximum iteration. These inputs are vital to the output produced by NMF. These parameters of NMF were not been considered in the existing literature on Automatic Text Summarization (ATS). This paper sheds light specifically on sparseness of NMF when applied to ATS and the impact it makes to the quality of the summary generated. In this study on NMF algorithm that supports sparseness is treated with various degree of sparsity and its performance impact on text summary generated is compared with other NMF algorithms without sparseness constraints.

Abstract. A new methodology for performing long-term source apportionment (SA) using positive matrix factorization (PMF) is presented. The method is implemented within the SoFi Pro software package and uses the multilinear engine (ME-2) as a PMF solver. The technique is applied to a 1-year aerosol chemical speciation monitor (ACSM) dataset from downtown Zurich, Switzerland. The measured organic aerosol mass spectra were analyzed by PMF using a small (14 d) and rolling PMF window to account for the temporal evolution of the sources. The rotational ambiguity is explored and the uncertainties of the PMF solutions were estimated. Factor–tracer correlations for averaged seasonal results from the rolling window analysis are higher than those retrieved from conventional PMF analyses of individual seasons, highlighting the improved performance of the rolling window algorithm for long-term data. In this study four to five factors were tested for every PMF window. Factor profiles for primary organic aerosol from traffic (HOA), cooking (COA) and biomass burning (BBOA) were constrained. Secondary organic aerosol was represented by either the combination of semi-volatile and low-volatility organic aerosol (SV-OOA and LV-OOA, respectively) or by a single OOA when this separation was not robust. This scheme led to roughly 40 000 PMF runs. Full visual inspection of all these PMF runs is unrealistic and is replaced by predefined user-selected criteria, which allow factor sorting and PMF run acceptance/rejection. The selected criteria for traffic (HOA) and BBOA were the correlation with equivalent black carbon from traffic (eBCtr) and the explained variation of m/z 60, respectively. COA was assessed by the prominence of a lunchtime concentration peak within the diurnal cycle. SV-OOA and LV-OOA were evaluated based on the fractions of m/z 43 and 44 in their respective factor profiles. Seasonal pre-tests revealed a non-continuous separation of OOA into SV-OOA and LV-OOA, in particular during the warm seasons. Therefore, a differentiation between four-factor solutions (HOA, COA, BBOA and OOA) and five-factor solutions (HOA, COA, BBOA, SV-OOA and LV-OOA) was also conducted based on the criterion for SV-OOA. HOA and COA contribute between 0.4–0.7 µg m−3 (7.8 %–9.0 %) and 0.7–1.2 µg m−3 (12.2 %–15.7 %) on average throughout the year, respectively. BBOA shows a strong yearly cycle with the lowest mean concentrations in summer (0.6 µg m−3, 12.0 %), slightly higher mean concentrations during spring and fall (1.0 and 1.5 µg m−3, or 15.6 % and 18.6 %, respectively), and the highest mean concentrations during winter (1.9 µg m−3, 25.0 %). In summer, OOA is separated into SV-OOA and LV-OOA, with mean concentrations of 1.4 µg m−3 (26.5 %) and 2.2 µg m−3 (40.3 %), respectively. For the remaining seasons the seasonal concentrations of SV-OOA, LV-OOA and OOA range from 0.3 to 1.1 µg m−3 (3.4 %–15.9 %), from 0.6 to 2.2 µg m−3 (7.7 %–33.7 %) and from 0.9 to 3.1 µg m−3 (13.7 %–39.9 %), respectively. The relative PMF errors modeled for this study for HOA, COA, BBOA, LV-OOA, SV-OOA and OOA are on average ±34 %, ±27 %, ±30 %, ±11 %, ±25 % and ±12 %, respectively.

Positive Matrix Factorization (PMF) is a powerful method for the apportionment of aerosol sources. Traditionally applied to chemical datasets, it has more recently been extended to physical datasets, focusing predominantly on size distributions of ultra-fine particles (Hopke et al., 2022). In this study, aerosol physical properties (particle size distributions, from ultrafine to coarse mode, and spectral aerosol absorption) were used as input to PMF (EPA 5.0) for identifying emission sources of two distinct sites, and results compared with those from chemical PMF. Located at the two extremes of the Italian territory, the sites of Aosta and Lecce represent two markedly different environments. The first lies in the mountainous area of the northwestern Alps, the second in a flat area at the southeastern edge of the peninsula, in a typical Mediterranean context. In previous studies, PMF of aerosol chemical composition was performed using chemical speciation of PM over the period 2019-2021 (Aosta, e.g. Diemoz et al., 2019) and 2017 (Lecce, Giannossa et al., 2022).  The PMF input dataset at the urban-background site in Aosta included aerosol size distributions measured by an Optical Particle Counter (OPC) across the 0.18–18 µm range and wavelength-dependent aerosol absorption from an AE33-aethalometer. At the urban-background observatory in Lecce, an aerosol size distribution range (0.02–10 µm) was available using both a Scanning Mobility Particle Sizer (SMPS) and an OPC, complemented by single-wavelength aerosol absorption (Multi-Angle Absorption Photometer, MAAP) providing eBC concentrations. Ancillary data included meteorological parameters and trace-gases concentration. Although with differences in the physical properties used, at both sites the physical PMF allowed the identification of 6 aerosol sources. In Aosta, the aerosol sources included fossil fuel combustion, biomass burning, secondary droplet mode, secondary condensation, dust, and coarse particles. In Lecce, the sources were nucleation (thanks to the additional use of the SMPS), traffic emissions, secondary nitrate, secondary sulfate, regional transport (sea spray and dust), and local resuspension (coarse particles). The work will present a comparison of the two PMF approaches (Chemistry- and Physics-based PMF) and advantages and limits of the different physical input datasets used at the two sites. For example, at the Lecce site, the information on ultrafine and fine particle distribution well captured features of the nucleation and traffic factors, while the absence of wavelength-dependent absorption coefficients limited the ability to distinguish biomass burning from fossil fuel sources, which was key for Aosta. Overall, the use of physical aerosol data as input to PMF proved to be an effective method for source apportionment and could usefully complement the chemical-PMF analysis. In fact, this approach offers significant advantages, such as the capability for quasi real-time monitoring and the relative ease of instruments use and data analysis compared to the ‘traditional’ chemical analysis of PM. The authors acknowledge the MUR for funding the research through the CIR01_00015-PER-ACTRIS-IT.  Diémoz, H.et al., https://doi.org/10.5194/acp-19-10129-2019, 2019.  Giannossa, L. et al., http://doi.10.1016/j.jenvman.2022.115752, 2022. Hopke, P. K. et al., https://doi.org/10.1016/j.scitotenv.2022.153104, 2022.