Introducing a weighted non-negative matrix factorization for image classification

The Non-negative Matrix Factorization technique (NMF) has been recently proposed for dimensionality reduction. NMF is able to produce a region- or part-based representation of objects and images. The positive space defined with NMF lacks a suitable metric and this paper experimentally compares NMF to Principal Component Analysis (PCA) in the context of classification, trying to determine the best distance metric for NMF. This paper introduces the use of the Earth Mover's Distance (EMD) as a relevant metric that takes into account the positive definition of the NMF bases, leading to better recognition results when the dimensionality of the problem is correctly chosen. PCA and NMF have also been tested under the presence of occlusions and, due to its part-based representation, NMF is able to improve on the PCA results.

Non-negative matrix factorization (NMF) has been one of the most popular methods for feature learning in the field of machine learning and computer vision. Most existing works directly apply NMF on high-dimensional image datasets for computing the effective representation of the raw images. However, in fact, the common essential information of a given class of images is hidden in their low rank parts. For obtaining an effective low-rank data representation, we in this paper propose a non-negative low-rank matrix factorization (NLMF) method for image clustering. For the purpose of improving its robustness for the data in a manifold structure, we further propose a graph regularized NLMF by incorporating the manifold structure information into our proposed objective function. Finally, we develop an efficient alternating iterative algorithm to learn the low-dimensional representation of low-rank parts of images for clustering. Alternatively, we also incorporate robust principal component analysis into our proposed scheme. Experimental results on four image datasets reveal that our proposed methods outperform four representative methods.

For the quantitation of cerebral blood flow (CBF) using H/sub 2//sup 15/O PET, the measurement of arterial input function (AIF) is essential. In this study, for the simplified quantitation, we present a method for the blind and noninvasive extraction of carotid input function (CIF) from dynamic PET images. On 8 healthy volunteers, the PET scans of C/sup 15/O and H/sub 2//sup 15/O were sequentially performed with arterial blood sampling using detector. With the inhalation of C/sup 15/O gas, dynamic PET data was acquired. And then after the injection of H/sub 2//sup 15/O, dynamic PET scans of H/sub 2//sup 15/O was perform. For 4 subjects, PET data for both rest and Acetazolamide (ACZ)-induced stress states were acquired, respectively. In the transverse dynamic images of C/sup 15/O and H/sub 2//sup 15/O, the regions of dynamic images including carotid artery were selected by masking. Non-negative matrix factorization (NMF) algorithm was used to extract the CIF from the selected dynamic images. The partial volume correction estimated from dynamic C/sup 15/O image, was applied to the extracted CIF. Whole brain CBF was estimated by kinetic analysis based on single compartment model. The error was analyzed using the area under the curve (AUC) and estimated CBF. NMF provided the good separation of the component of CIF from others in both images of C/sup 15/O and H/sub 2//sup 15/O. The shape of estimated CIF by NMF was similar with AIF measured by blood sampling. The AUC between the measured AIF and the estimated CIF was not so different at both rest and ACZ states. The difference of CBF before and after the injection of ACZ was also same between the measured AIF and the estimated CIF. These results show that NMF technique may be a tool for the noninvasive extraction of carotid input function, and this carotid input function might be used in the noninvasive quantitation of CBF using H/sub 2//sup 15/O PET.

This paper proposes a novel image fusion method based on weighted non-negative matrix factorization (WNMF) and region segmentation. Firstly, visible and infrared images are endowed different weight, and are taken as original data in WNMF analysis, by which the feature image can be extracted. Then the target regions in infrared image are segmented by region growing method and are extracted, then the pixels of the target regions replace the pixels of corresponding position in feature image. At last, the fusion image is obtained. The experimental results show that the method is better than previous ones, both in visual effect and objective evaluation criteria.

Non-negativity and dependence constrained sparse coding for image classification

BackgroundIn the last few years, the Non-negative Matrix Factorization(NMF) technique has gained a great interest among the Bioinformatics community, since it is able to extract interpretable parts from high-dimensional datasets. However, the computing time required to process large data matrices may become impractical, even for a parallel application running on a multiprocessors cluster.In this paper, we present NMF-mGPU, an efficient and easy-to-use implementation of the NMF algorithm that takes advantage of the high computing performance delivered by Graphics-Processing Units(GPUs). Driven by the ever-growing demands from the video-games industry, graphics cards usually provided in PCs and laptops have evolved from simple graphics-drawing platforms into high-performance programmable systems that can be used as coprocessors for linear-algebra operations. However, these devices may have a limited amount of on-board memory, which is not considered by other NMF implementations on GPU.ResultsNMF-mGPU is based on CUDA(Compute Unified Device Architecture), the NVIDIA’s framework for GPU computing. On devices with low memory available, large input matrices are blockwise transferred from the system’s main memory to the GPU’s memory, and processed accordingly. In addition, NMF-mGPU has been explicitly optimized for the different CUDA architectures. Finally, platforms with multiple GPUs can be synchronized through MPI(Message Passing Interface). In a four-GPU system, this implementation is about 120 times faster than a single conventional processor, and more than four times faster than a single GPU device (i.e., a super-linear speedup).ConclusionsApplications of GPUs in Bioinformatics are getting more and more attention due to their outstanding performance when compared to traditional processors. In addition, their relatively low price represents a highly cost-effective alternative to conventional clusters. In life sciences, this results in an excellent opportunity to facilitate the daily work of bioinformaticians that are trying to extract biological meaning out of hundreds of gigabytes of experimental information. NMF-mGPU can be used “out of the box” by researchers with little or no expertise in GPU programming in a variety of platforms, such as PCs, laptops, or high-end GPU clusters. NMF-mGPU is freely available at https://github.com/bioinfo-cnb/bionmf-gpu.

The global utilization of electric vehicles (EVs) is exponentially increasing due to the increased availability of cost-efficient EVs and infrastructure managements for the EVs. In spite of the increasing usage of EVs, the problem of EV usage patterns’ analysis and implementing sustainable infrastructure for the EV transportation is still under development. In addition to this, there is a challenging problem of long waiting hours in traffic signals. This study deals with these problems by proposing an architecture that includes EV usage pattern analysis using nonnegative matrix factorization (NMF) technique and renewable solar-powered wireless smart charging grid to effectively utilize or mitigate the long traffic signal waiting hours. The insights from the EV usage patterns are analyzed and presented showing the importance of usage pattern analysis alongside to the presented architecture of renewable solar-powered wireless EV-smart charging grid. These implementations improvise the usage of the EVs and enhancing the transportation experience, which in turn leads to the development of sustainable smart transportation.

Recent studies using Principal Components Analysis (PCA) support low-dimensional models of odor space, in which one or two dimensions - with hedonic valence featuring prominently - explain most odor variability. Here we use non-negative matrix factorization (NMF) - a nonlinear optimization method - to discover an alternative, reduced-dimensional representation of the Dravnieks odor database (144 odors x 146 descriptors). NMF is theoretically well-suited for these types of analyses, as odor profiling data is inherently non-negative (e.g. descriptors either apply, or do not). We divided the dataset into training and testing halves, and found that RMSD testing error attained a minimum for subspace choice of 25, motivating this as an upper bound for odor perceptual space dimensionality. More parsimonious representations were found by comparing reconstruction errors (fraction of unexplained variance) of NMF with reconstruction errors of PCA on scrambled data (PCAsd). For subspace sizes > 10, NMF error was indistinguishable from PCAsd error, indicating no gain in retaining more than 10 perceptual dimensions. As is typical of NMF basis sets, the 10 odor dimensions we obtain are sparse (only a small subset of the 146 descriptors apply), and categorical (represent a positive valued quality). Moreover, these 10 dimensions were near-orthogonal, with a mean angle of 73 degrees between all pairs of basis vectors. Investigating the distribution of odors in this 10-dimensional space, we find marked clustering (Figure 1), with each odor being well-defined by its membership in a single dimension, and to the exclusion of others. In ongoing work, we are using graph-kernel methods to define a rudimentary mapping between physicochemical features of odorants and the 10 descriptor dimensions.

Many problems in image representation and classification involve some form of dimensionality reduction. Non-negative matrix factorization (NMF) is a recently proposed unsupervised procedure for learning spatially localized, parts-based subspace representation of objects. Here we present an improvement of the classical NMF by combining with Log-Gabor wavelets to enhance its part-based learning ability. In addition, we compare the new method with principal component analysis (PCA) and locally linear embedding (LLE) proposed recently in Science. Finally, we apply the new method to several real world datasets and achieve good performance in representation and classification.

Background: Recent technological advances have drastically reduced the cost and increased the speed of DNA sequencing, developments that have been heralded as the beginning of a new era of personalized medicine. However, there still remains a pressing need to translate improved knowledge of cancer genomics into better clinical outcomes. Like many solid tumors, high grade serous ovarian carcinoma (HGS-OvCa) is a heterogeneous entity with a widely variable clinical course even among patients of the same stage and histological subtype. Most prior attempts to stratify tumors into clinically meaningful subtypes have utilized gene expression profiles; however as a whole such methods have not reliably discovered subtypes with significant differences in survival or response to chemotherapy. There has been much effort to stratify tumors based on their somatic mutations; however this has proved difficult due to the great heterogeneity and sparseness of somatic mutations in most solid tumors. For example, in the TCGA HGS-OvCa cohort the average tumor had 61 non-silent mutations spread out over the ~20,000 genes in the human genome; most genes were mutated in one tumor only. Standard consensus clustering approaches applied to this sparse mutational data have repeatedly failed to produce a clinically valuable stratification of patients. Methods: The technique of Network-based Stratification (NBS) combines genome-scale somatic mutation profiles with a genetic interaction network to produce a subdivision of patients into subtypes. Briefly, somatic mutations for each patient are represented as a profile of binary (1,0) states on genes, in which a ‘1’ indicates a mutated gene. For each patient independently we project the mutation profiles onto a human gene interaction network obtained from public databases. Next the technique of network propagation is applied to spread the influence of each subsampled mutation profile over its network neighborhood, resulting in a ‘network-smoothed’ profile. Following this ‘network-smoothing’, profiles are clustered into a predefined number of subtypes using the unsupervised technique of non-negative matrix factorization (NMF). This procedure is repeated for 1000 different subsamples using the technique of consensus clustering to promote robust cluster assignments. This method was repeated with three difference sources of network data: STRING, HumanNet, and PathwayCommons. Results: Using the NBS technique four distinct subtypes were identified from the TCGA HGS-OvCa cohort of 330 exome sequenced and clinically-annotated tumors. The median overall survival (OS) in months for these four subtypes was 36, 48, 56, and not-yet-reached respectively, which was highly significant (Logrank p = 1.59 x 10-5). Subtypes were also predictive of platinum free interval (PFI), (Logrank p = 0.046). No significant differences between subtypes were found in age, tumor stage at presentation, or percentage of patients with an optimal surgical resection, and a cox proportional hazards model was significant even after taking those factors into account (Likelihood ratio test p = 3.3x10-4). Comparing each subtype against the other three, the network of mutated genes for subtype 1, which had the worst overall survival and shortest platinum free interval, was enriched for over 20 genes in the fibroblast growth pathway. The network for subtype 2 was enriched in DNA damage response genes including ATM, ATR, BRCA1/2, RAD51 and Chek2, and also contained the majority of patients with BRCA1/2 germline mutations (15/20 and 5/6, respectively). Conclusions: Network-based Stratification (NBS) combines genome-scale somatic mutation profiles with a genetic interaction network to stratify a cohort of HGS-OvCa tumors into subtypes with significant differences is OS and PFI. Citation Format: Matan Hofree, John Paul Shen, Hannah Carter, Andrew Gross, Trey Ideker. Network-based stratification: Combining genome-scale somatic mutation profiles with genetic interaction networks to identify clinically relevant subtypes in high grade serous ovarian carcinoma. [abstract]. In: Proceedings of the AACR Special Conference on Advances in Ovarian Cancer Research: From Concept to Clinic; Sep 18-21, 2013; Miami, FL. Philadelphia (PA): AACR; Clin Cancer Res 2013;19(19 Suppl):Abstract nr A14.

Non-negative matrix factorization (NNMF) technique has been widely applicable for dimension reduction. NNMF has shown its usability to solve numerous challenges in areas such as signal processing, image classification, and text mining. Existing matrix factorization methods need a predefined value of rank of a base matrix. Our primary intention in this article is to automatically identify the optimum value of the rank to factorize the matrix. Here we propose an Incremental Cauchy NonNegative Matrix factorization (ICNNMF) which automatically finds the optimum rank and separates the overlapping sounds received from a single channel of a source. Classification is performed using an ensemble of one class fuzzy rule-based (FRB) classifiers. We evaluated the proposed method over the real data set, and the proposed model has shown better accuracy as compared to the traditional classifier. Moreover, the proposed matrix factorization method automatically determines the optimal value of the rank and adapt to new data.

We compared non-negative matrix factorization (NMF) and convolution kernel compensation techniques for high-density electromyogram decomposition. The experimental data were recorded from nine healthy persons during controlled single degree of freedom (DOF) wrist flexion-extension, supination-pronation, and ulnar-radial deviation movements. We assembled the identified motor units and NMF components into three groups. Those active mostly during the first and the second movement direction per DOF were placed in the G1 and G3 groups, respectively. The remaining components were nonspecific for movement direction and were placed in the G2 group. In ulnar and radial deviation, the relative energies of identified cumulative motor unit spike trains (CSTs) and NMF components were similarly distributed among the groups. In other two movement types, the energy of NMF components in the G2 group was significantly larger than the energy of CSTs. We further performed a coherence analysis between CSTs and sums of NMF components in each group. Both decompositions demonstrated a solid match, but only at frequencies <3 Hz. At higher frequencies, the coherence hardly exceeded the value of 0.5. Potential reasons for these discrepancies include the negative impact of motor unit action potential shapes and noise on NMF decomposition.

Positron Emission Tomography (PET) can measure dynamics of radioactive material ligand and is widely used for observing e.g. metabolism or the spatial densities of neuroreceptors in the brain. It is still an important research topic to improve the signal-to-noise ratio (SNR) of PET images reconstructed from measured sinograms. In this paper, we propose a four dimensional PET image reconstruction method, which applies a constrained non-negative matrix factorization (NMF) technique for the improvement of the SNR. According to the compartment model for pharmacokinetics analysis, a matrix representation of dynamical PET images can be decomposed into two matrices, which are non-negative and low-rank. Experimental results demonstrate that our proposed method improves the SNR compared with conventional methods.

이 논문은 고차원의 데이터를 저 차원으로 줄이는 방법 중 하나인 특징추출에 대한 방법들의 특성을 비교한다. 비교대상 방법은 전통적인 PCA(Principal Component Analysis)방법과 시각피질의 특성을 보인다고 알려진 ICA(Independent Component Analysis), 국소기반인식을 구현한 NMF(Non-negative Matrix Factorization), 그리고 이의 성능을 개선한 sNMF(Sparse NMF)로 정하였다. 추출된 특징들의 특성을 시각적으로 확인하기 위하여 필기체 숫자 영상을 대상으로 특징추출을 수행하였으며, 인식기에 적용한 효과의 확인을 위하여 추출된 특징을 다층퍼셉트론에 학습시켜보았다. 각 방법의 특성을 비교한 결과는 응용하고자 하는 문제에서 어떤 특징을 추출하기 원하느냐에 따라 특징추출 방법을 선정할 때 유용할 것이다. In this paper, feature extraction methods, which is one field of reducing dimensions of high-dimensional data, are empirically investigated. We selected the traditional PCA(Principal Component Analysis), ICA(Independent Component Analysis), NMF(Non-negative Matrix Factorization), and sNMF(Sparse NMF) for comparisons. ICA has a similar feature with the simple cell of V1. NMF implemented a "parts-based representation in the brain" and sNMF is a improved version of NMF. In order to visually investigate the extracted features, handwritten digits are handled. Also, the extracted features are used to train multi-layer perceptrons for recognition test. The characteristic of each feature extraction method will be useful when applying feature extraction methods to many real-world problems.

The growing availability of hyperspectral data from planetary missions ranging from CubeSats to flagship programs has led to many studies aiming to infer surface mineralogy through remote sensing techniques. The Moon Mineralogical Mapper (M³) aboard Chandrayaan-1 provides visible and near-infrared data (430–3000nm) at 140 m/pixel resolution and has been widely used to infer lunar surface composition [1]. A common strategy involves applying unmixing models, often Non-Negative Matrix Factorization (NNMF), to M³ images with varying reflectance and compositional properties, using laboratory reference spectra from the RELAB database for comparison [2,3]. These spectra from standard materials represent typical lunar minerals such as olivine, pyroxenes, plagioclase, iron-titanium oxides, and volcanic glasses.This study first focuses on reproducing previous efforts to estimate the lunar mineralogy, applied to the Apollo 17 landing site (Figure 1). Then it shifts toward highlighting the methodological uncertainties involved in interpreting hyperspectral imagery using unmixing models in remote sensing applications.Figure. 1 (Top)Taurus-Littrow Valley. (Bottom) Mean (dark) reflectance spectrum and standard deviation (red) of the Taurus-Littrow Valley hyperspectral image.In the initial stage, a spatial binning with 8X8 window was applied to reduce the signal-to-noise ratio of the image. Principal Component Analysis (PCA) was then used to reduce data dimensionality and highlight surface variations. Initially applied on the binned image, PCA revealed that the first component explained 98% of the variance, mainly reflecting topography and illumination differences. To minimize these effects and enhance chemical information, the first derivative of the spectral data was applied (same preprocessing was applied to reference spectra, Figure.2). A second PCA was then performed to determine the chemical rank of the matrix, which suggested that NNMF should be carried out with 3–4 principal components.Figure. 2 ​First derivative of reference spectra from RELAB.NNMF was performed to unmix and extract surface components, possibly revealing features that may correspond to geochemical variations. In this final stage, it was assumed that three principal components could be present within the image. Figure. 3 shows the spatial distribution of the combination of these three components (top), along with the corresponding “pure” spectra (bottom) extracted by the model after applying NNMF using endmember initialization and non-negativity constraints [4] applied to the contributions. Commonly, when these results were compared with preprocessed spectra from RELAB, compounds such as pyroxenes, olivine, volcanic-glass, may be identified (Figure. 3), with acceptable lack of fit and a total variance explanation exceeding 99%.Figure. 3 False RGB showing the concentrations of the three main components after applying NNMF model using endmember initialization and non-negativity constraints.However, while spectral unmixing models have been widely applied in remote sensing for decades, limited attention has been paid to the intrinsic ambiguities they involve. Although simpler ambiguities (such as those related to scale or sign) can often be addressed using basic constraints like non-negativity, rotational ambiguity remains a significant and underexplored source of uncertainty. For these reasons, it is important to highlight that solutions derived from NNMF (commonly referred to as Multivariate Curve Resolution with non-negativity constraints) depend considerably on the degrees of freedom allowed by the optimization algorithm (e.g., Alternating Least Squares, ALS). To visualize and quantify this issue, feasibility plots were employed through MCRbands and FACpack (Figure. 4), which map the set of all admissible solutions within the NNMF framework [5-7].Figure. 4 Feasibility plots for the three components calculated. Right, the contributions and left, the spectral profiles obtained. Contributions: Blue and red represents the solution obtained and the plausible solution that could also be considered as optimal.Figure 4 shows that any space covered between those solutions could be considered feasible. On the left, the spectra calculated and the feasible spectra both indicate a strong lack of specificity and uniqueness in the solutions, clearly stating that the model is not stable enough to be deemed an optimal solution. The extent of solution variability reveals that differences between the acceptable solution and real solutions for the considered species are considerable due to rotational freedom. In conclusion, we emphasize the importance of a more rigorous consideration of ambiguity in remote sensing applications. Additionally, the presence of regolith and volcanic rocks on the lunar surface makes it unlikely that a 140 m/pixel represents a pure mineral spectrum [8].Keywords: Remote sensing, Chemometrics, Unmixing models, Near Infrared. Acknowledgements: Work supported through the PAMMAT project: “Alteration processes in Mars and Moon Meteorites, and Terrestrial Analogues at different environments: Mars2020, Rosalind Franklin and Returned Samples from Mars and Moon” (Grant No. PID2022-142750OB-I00), funded by the Spanish Agency for Research (through the Spanish Ministry of Science and Innovation, MCIN, and the European Regional Development Fund, FEDER).