Binary Nonnegative Matrix Factorization Applied to Semi-conductor Wafer Test Sets

This paper deals with algorithms for positive semidefinite matrix factorization (PSDMF). PSDMF is a recently-proposed extension of nonnegative matrix factorization with applications in combinatorial optimization, among others. In this paper, we focus on improving the local convergence of an alternating block gradient (ABC) method for PSDMF in a noise-free setting by replacing the quadratic objective function with the Poisson log-likelihood. This idea is based on truncated Wirtinger flow (TWF), a phase retrieval (PR) method that trims outliers in the gradient and thus regularizes it. Our motivation is a recent result linking PR with PSDMF. Our numerical experiments validate that the numerical benefits of TWF may carry over to PSDMF despite the more challenging setting, when initialized within its region of convergence. We then extend TWF from PR to affine rank minimization (ARM), and show that although the outliers are no longer an issue in the ARM setting, PSDMF with the new objective function may still achieves a smaller error for the same number of iterations. In a broader view, our results indicate that a proper choice of objective function may enhance convergence of matrix (or tensor) factorization methods.

Real world sounds often exhibit non-stationary spectral characteristics such as those produced by a harpsichord or a guitar. The classical Non-negative Matrix Factorization (NMF) needs a number of atoms to accurately decompose the spectrogram of such sounds. An extension of NMF is proposed hereafter which includes time-frequency activations based on ARMA modeling. This leads to an efficient single-atom decomposition for a single audio event. The new algorithm is tested on real audio data and shows promising results.

This paper describes an ultimate covariance-aware multichannel extension of nonnegative matrix factorization (NMF) for blind source separation (BSS). A typical approach to BSS is to integrate a low-rank source model with a full-rank spatial model as multichannel NMF (MNMF) based on full-rank spatial covariance matrices (CMs) or its efficient version named FastMNMF based on jointly-diagonalizable spatial CMs do. The NMF-based phase-unaware source model, however, can deal with only the positive cooccurrence relations between time-frequency bins. To overcome this limitation, we propose an efficient multichannel extension of correlated tensor factorization (CTF) named FastMCTF based on jointly-diagonalizable temporal, frequency, and spatial CMs. Integration of the jointly-diagonalizable full-rank source model proposed by FastCTF with the jointly-diagonalizable full-rank spatial model proposed by FastMNMF enables us to completely consider the positive and negative covariance relations between frequency bins, time frames, and channels. We derive a convergence-guaranteed parameter estimation algorithm based on the multiplicative update and iterative projection and experimentally show the potential of the proposed method.

The growing availability of hyperspectral data from planetary missions ranging from CubeSats to flagship programs has led to many studies aiming to infer surface mineralogy through remote sensing techniques. The Moon Mineralogical Mapper (M³) aboard Chandrayaan-1 provides visible and near-infrared data (430–3000nm) at 140 m/pixel resolution and has been widely used to infer lunar surface composition [1]. A common strategy involves applying unmixing models, often Non-Negative Matrix Factorization (NNMF), to M³ images with varying reflectance and compositional properties, using laboratory reference spectra from the RELAB database for comparison [2,3]. These spectra from standard materials represent typical lunar minerals such as olivine, pyroxenes, plagioclase, iron-titanium oxides, and volcanic glasses.This study first focuses on reproducing previous efforts to estimate the lunar mineralogy, applied to the Apollo 17 landing site (Figure 1). Then it shifts toward highlighting the methodological uncertainties involved in interpreting hyperspectral imagery using unmixing models in remote sensing applications.Figure. 1 (Top)Taurus-Littrow Valley. (Bottom) Mean (dark) reflectance spectrum and standard deviation (red) of the Taurus-Littrow Valley hyperspectral image.In the initial stage, a spatial binning with 8X8 window was applied to reduce the signal-to-noise ratio of the image. Principal Component Analysis (PCA) was then used to reduce data dimensionality and highlight surface variations. Initially applied on the binned image, PCA revealed that the first component explained 98% of the variance, mainly reflecting topography and illumination differences. To minimize these effects and enhance chemical information, the first derivative of the spectral data was applied (same preprocessing was applied to reference spectra, Figure.2). A second PCA was then performed to determine the chemical rank of the matrix, which suggested that NNMF should be carried out with 3–4 principal components.Figure. 2 ​First derivative of reference spectra from RELAB.NNMF was performed to unmix and extract surface components, possibly revealing features that may correspond to geochemical variations. In this final stage, it was assumed that three principal components could be present within the image. Figure. 3 shows the spatial distribution of the combination of these three components (top), along with the corresponding “pure” spectra (bottom) extracted by the model after applying NNMF using endmember initialization and non-negativity constraints [4] applied to the contributions. Commonly, when these results were compared with preprocessed spectra from RELAB, compounds such as pyroxenes, olivine, volcanic-glass, may be identified (Figure. 3), with acceptable lack of fit and a total variance explanation exceeding 99%.Figure. 3 False RGB showing the concentrations of the three main components after applying NNMF model using endmember initialization and non-negativity constraints.However, while spectral unmixing models have been widely applied in remote sensing for decades, limited attention has been paid to the intrinsic ambiguities they involve. Although simpler ambiguities (such as those related to scale or sign) can often be addressed using basic constraints like non-negativity, rotational ambiguity remains a significant and underexplored source of uncertainty. For these reasons, it is important to highlight that solutions derived from NNMF (commonly referred to as Multivariate Curve Resolution with non-negativity constraints) depend considerably on the degrees of freedom allowed by the optimization algorithm (e.g., Alternating Least Squares, ALS). To visualize and quantify this issue, feasibility plots were employed through MCRbands and FACpack (Figure. 4), which map the set of all admissible solutions within the NNMF framework [5-7].Figure. 4 Feasibility plots for the three components calculated. Right, the contributions and left, the spectral profiles obtained. Contributions: Blue and red represents the solution obtained and the plausible solution that could also be considered as optimal.Figure 4 shows that any space covered between those solutions could be considered feasible. On the left, the spectra calculated and the feasible spectra both indicate a strong lack of specificity and uniqueness in the solutions, clearly stating that the model is not stable enough to be deemed an optimal solution. The extent of solution variability reveals that differences between the acceptable solution and real solutions for the considered species are considerable due to rotational freedom. In conclusion, we emphasize the importance of a more rigorous consideration of ambiguity in remote sensing applications. Additionally, the presence of regolith and volcanic rocks on the lunar surface makes it unlikely that a 140 m/pixel represents a pure mineral spectrum [8].Keywords: Remote sensing, Chemometrics, Unmixing models, Near Infrared. Acknowledgements: Work supported through the PAMMAT project: “Alteration processes in Mars and Moon Meteorites, and Terrestrial Analogues at different environments: Mars2020, Rosalind Franklin and Returned Samples from Mars and Moon” (Grant No. PID2022-142750OB-I00), funded by the Spanish Agency for Research (through the Spanish Ministry of Science and Innovation, MCIN, and the European Regional Development Fund, FEDER).

A majorization–minimization based solution to penalized nonnegative matrix factorization with orthogonal regularization

Non-negative matrix factorization (NMF) has been one of the most popular methods for feature learning in the field of machine learning and computer vision. Most existing works directly apply NMF on high-dimensional image datasets for computing the effective representation of the raw images. However, in fact, the common essential information of a given class of images is hidden in their low rank parts. For obtaining an effective low-rank data representation, we in this paper propose a non-negative low-rank matrix factorization (NLMF) method for image clustering. For the purpose of improving its robustness for the data in a manifold structure, we further propose a graph regularized NLMF by incorporating the manifold structure information into our proposed objective function. Finally, we develop an efficient alternating iterative algorithm to learn the low-dimensional representation of low-rank parts of images for clustering. Alternatively, we also incorporate robust principal component analysis into our proposed scheme. Experimental results on four image datasets reveal that our proposed methods outperform four representative methods.

Gene expression analysis of samples with mixed cell types only provides limited insight to the characteristics of specific tissues. In silico deconvolution can be applied to extract cell type specific expression, thus avoiding prohibitively expensive techniques such as cell sorting or single-cell sequencing. Non-negative matrix factorization (NMF) is a deconvolution method shown to be useful for gene expression data, in part due to its constraint of non-negativity. Unlike other methods, NMF provides the capability to deconvolve without prior knowledge of the components of the model. However, NMF is not guaranteed to provide a globally unique solution. In this work, we present FaStaNMF, a method that balances achieving global stability of the NMF results, which is essential for inter-experiment and inter-lab reproducibility, with accuracy and speed. Results: FaStaNMF was applied to four datasets with known ground truth, created based on publicly available data or by using our simulation infrastructure, RNAGinesis. We assessed FaStaNMF on three criteria - speed, accuracy, and stability, and it favorably compared to the standard approach of achieving reproduceable results with NMF. We expect that FaStaNMF can be applied successfully to a wide array of biological data, such as different tumor/immune and other disease microenvironments.

This paper describes the difference between normal microlenses and zero-space microlenses in image sensors. The investigation of both sensitivity and crosstalk has been implemented for normal microlenses and zero-space microlenses that are fabricated on a 2.8-μm pinned photodiode array of 640 (row) by 480 (column) based on an 0.18-μm -CMOS process. The simulation results show that the zero-space microlens technology increases the pixel sensitivity by about 75.61% and decreases the crosstalk by about 9.4% compared to the normal microlens technology. This paper presents the sensitivity analysis of both normal microlenses and zero-space microlenses based on the wafer test data from the Teradyne IP750, which is a special wafer test platform for CMOS sensors. The results of the statistical data of the wafer test show that the sensitivity of the zero-space microlenses has been improved by about 73.6% on the red pixel (69.6% on the green pixel and 76.3% on the blue pixel) compared with that of the normal microlenses. Using a similar statistical method, the crosstalk characteristics of both the zero-space microlenses and the normal microlenses have been measured as well. Compared with the normal microlenses, the testing results show that the crosstalk of green light in red light is reduced by about 7.244% and the crosstalk of blue light in red light is reduced by about 41.447% due to the use of the zero-space microlens technology. More importantly, this paper also provides the comparison and analysis of the real images taken by CMOS image sensors with the normal microlenses and the zero-space microlenses, respectively. From the comparison of image results, the image luminance of the zero-space microlenses is higher than that of the normal microlenses, and the image definition of the zero-space microlenses is clearer than that of the normal microlenses as well. So far, the silicon results show that the zero-space microlens technology is a better choice than the normal microlens technology for CMOS image sensors of a small-size pixel.

Energy disaggregation or Non-Intrusive Load Monitoring (NILM) addresses the issue of extracting device-level energy consumption information by monitoring the aggregated signal at one single measurement point without installing meters on each individual device. Energy disaggregation can be formulated as a source separation problem where the aggregated signal is expressed as linear combination of basis vectors in a matrix factorization framework. In this paper, an approach based on Sum-to-k constrained Non-negative Matrix Factorization (S2K-NMF) is proposed. By imposing the sum-to-k constraint and the non-negative constraint, S2K-NMF is able to effectively extract perceptually meaningful sources from complex mixtures. The strength of the proposed algorithm is demonstrated through two sets of experiments: Energy disaggregation in a residential smart home, and HVAC components energy monitoring in an industrial building testbed maintained at the Oak Ridge National Laboratory (ORNL). Extensive experimental results demonstrate the superior performance of S2K-NMF as compared to state-of-the-art decomposition-based disaggregation algorithms. The source code and our collected data (HVORUT) for studying NILM for HVAC units can be found at https://bitbucket.org/aicip/nonintrusive-load-monitoring.

Dramatic growth in the volume of data made a compact and informative representation of the data highly demanded in computer vision, information retrieval, and pattern recognition. Non-negative Matrix Factorization (NMF) is used widely to provide parts-based representations by factorizing the data matrix into non-negative matrix factors. Since non-negativity constraint is not sufficient to achieve robust results, variants of NMF have been introduced to exploit the geometry of the data space. While these variants considered the local invariance based on the manifold assumption, we propose Farness preserving Non-negative Matrix Factorization (FNMF) to exploits the geometry of the data space by considering non-local invariance which is applicable to any data structure. FNMF adds a new constraint to enforce the far points (i.e., non-neighbors) in original space to stay far in the new space. Experiments on different kinds of data (e.g., Multimedia, Earth Observation) demonstrate that FNMF outperforms the other variants of NMF.

Nonnegative matrix factorization (NMF) is known as a parts-based linear representation for nonnegative data. This method has been applied for blind source separation (BSS) when the sources are nonnegative. This paper presents a new NMF method for independent component analysis (ICA), which is useful for BSS without the nonnegativity constraint. Using this method, we transform the sources by their cumulative distribution functions (CDFs) and perform the nonparametric quantization to construct a nonnegative matrix where each entry represents the joint probability density of two transformed signals. The NMF procedure is accordingly realized to find the ICA demixing matrix. The independence between sources is maximized towards attaining the uniformity in the joint probability density. In the experiments on the separation of signal and music signals, we show the effectiveness of the proposed NMF-ICA compared to the infomax ICA and FastICA algorithms.

Graph clustering is a fundamental task in the network analysis, which is essential for many modern applications. In recent years, Nonnegative Matrix Factorization (NMF) has been effectively used to discover cluster structures due to its powerful interpretability property. In this paper, we introduce a clustering algorithm based on Semi-Nonnegative Matrix Factorization that is one of the well-known extensions of NMF. This factorization allows algorithms to capture more accurate (positive and negative) relationships among clusters and, thereby, to derive a latent factor that is even proper for clustering and also has much more responsibility in the regularization. Moreover, to improve the clustering, we define an asymmetric graph regularization to penalize the asymmetric similarity of nodes denoted by cluster memberships. Experimental results on four real-world datasets validate the effectiveness of the proposed method.

In this paper, we present a novel muscle synergy extraction method based on multivariate curve resolution–alternating least squares (MCR-ALS) to overcome the limitation of the nonnegative matrix factorization (NMF) method for extracting non-sparse muscle synergy, and we study its potential application for evaluating motor function of stroke survivors. Nonnegative matrix factorization (NMF) is the most widely used method for muscle synergy extraction. However, NMF is susceptible to components’ sparseness and usually provides inferior reliability, which significantly limits the promotion of muscle synergy. In this study, MCR-ALS was employed to extract muscle synergy from electromyography (EMG) data. Its performance was compared with two other matrix factorization algorithms, NMF and self-modeling mixture analysis (SMMA). Simulated data sets were utilized to explore the influences of the sparseness and noise on the extracted synergies. As a result, the synergies estimated by MCR-ALS were the most similar to true synergies as compared with SMMA and NMF. MCR-ALS was used to analyze the muscle synergy characteristics of upper limb movements performed by healthy (n = 11) and stroke (n = 5) subjects. The repeatability and intra-subject consistency were used to evaluate the performance of MCR-ALS. As a result, MCR-ALS provided much higher repeatability and intra-subject consistency as compared with NMF, which were important for the reliability of the motor function evaluation. The stroke subjects had lower intra-subject consistency and seemingly had more synergies as compared with the healthy subjects. Thus, MCR-ALS is a promising muscle synergy analysis method for motor function evaluation of stroke patients.

Using nonnegative/binary matrix factorization (NBMF), a matrix can be decomposed into a nonnegative matrix and a binary matrix. Our analysis of facial images, based on NBMF and using the Fujitsu Digital Annealer, leads to successful image reconstruction and image classification. The NBMF algorithm converges in fewer iterations than those required for the convergence of nonnegative matrix factorization (NMF), although both techniques perform comparably in image classification.

Positron emission tomography (PET) is an important imaging technique to visualize a number of functions in the brain or human body. For reconstructing PET images from the sinogram data, an inverse problem has to be solved using numerical optimizations such as expectation-maximization (EM)-based methods. However, the standard EM method suffers from measurement noise added in the sinogram data. In this paper, we propose a new simultaneous PET image reconstruction and parts extraction method using constrained non-negative matrix factorization. In contrast that the many existing methods reconstruct a single PET image independently, we reconstruct the time-series of PET images simultaneously from the time-series of sinograms using non-negative matrix factorization. Furthermore, we impose the smoothness constraint for the temporal feature, and the exclusive LASSO-based sparseness constraint for the spatial feature for robust image reconstruction and physically meaningful feature extraction.