Block-diagonal Approach for Non-Negative Linguistic Matrix and Tensor Factorization

In order to achieve best neutron flux-mapping within nuclear research reactors, we investigate in this paper, the application of blind source separation algorithms to extract different independent components that form the fission chamber output signals. More specifically, we simulated the fission chamber using python-based of Fission Chambers (pyFC) code suite. The resulting simulated signals are considered observation vectors. They are then, processed via nonnegative matrix and tensor factorization (NMF&NTF) methods to isolate the independent components that form a neutron signal. Thus, in order to select the most efficient one to analyze our set of data, various algorithms have been tested and classified according to their performance index of separability (PI). The extracted independent components have been used to characterize neutron signal through spectrogram time-frequency representation. Obtained results will be illustrated and discussed.

We have studied the feature extraction from sea ice SAR images based on non-negative factorization methods. The methods reported here are the sparseness-constrained non-negative matrix factorization (SC-NMF) and Non-negative tensor factorization (NTF). The studies performed show that these methods can be used to extract meaningful features from SAR images and that they can be used in sea ice SAR classification.

Non-negative Tensor Factorization (NTF) is a widely used technique for decomposing a non-negative value tensor into sparse and reasonably interpretable factors. However, NTF performs poorly when the tensor is extremely sparse, which is often the case with real-world data and higher-order tensors. In this paper, we propose Non-negative Multiple Tensor Factorization (NMTF), which factorizes the target tensor and auxiliary tensors simultaneously. Auxiliary data tensors compensate for the sparseness of the target data tensor. The factors of the auxiliary tensors also allow us to examine the target data from several different aspects. We experimentally confirm that NMTF performs better than NTF in terms of reconstructing the given data. Furthermore, we demonstrate that the proposed NMTF can successfully extract spatio-temporal patterns of people's daily life such as leisure, drinking, and shopping activity by analyzing several tensors extracted from online review data sets.

Smooth nonnegative matrix and tensor factorizations for robust multi-way data analysis

Non-negative matrix factorization (NMF) has been one of the most popular methods for feature learning in the field of machine learning and computer vision. Most existing works directly apply NMF on high-dimensional image datasets for computing the effective representation of the raw images. However, in fact, the common essential information of a given class of images is hidden in their low rank parts. For obtaining an effective low-rank data representation, we in this paper propose a non-negative low-rank matrix factorization (NLMF) method for image clustering. For the purpose of improving its robustness for the data in a manifold structure, we further propose a graph regularized NLMF by incorporating the manifold structure information into our proposed objective function. Finally, we develop an efficient alternating iterative algorithm to learn the low-dimensional representation of low-rank parts of images for clustering. Alternatively, we also incorporate robust principal component analysis into our proposed scheme. Experimental results on four image datasets reveal that our proposed methods outperform four representative methods.

Matrix factorization is a general technique that can extract latent factors from data. Recent studies applied matrix factorization to the problem of establishing which skills are required by question items, and for assessing student skills mastery from student performance data. A number of generic algorithms, such as Non-negative Matrix Factorization and Tensor factorization, are used in these studies to perform the factorization, but few have looked at optimizing these algorithms to the specific characteristics of student performance data. In this thesis, we explore how one such characteristic can lead to better factorization: the fact that items are learnt in a constrained order and allow such inferences as if a difficult item is succeeded, an easier one should also be succeeded. In particular, we want to address this question: can a partial order knowledge structure (POKS) be used to guide matrix factorization algorithms and lead to faster or better solutions to latent skills modelling?

Zero-shot learning, a special case of unsupervised domain adaptation where the source and target domains have disjoint label spaces, has become increasingly popular in the computer vision community. In this paper, we propose a novel zero-shot learning method based on discriminative sparse non-negative matrix factorization. The proposed approach aims to identify a set of common high-level semantic components across the two domains via non-negative sparse matrix factorization, while enforcing the representation vectors of the images in this common component-based space to be discriminatively aligned with the attribute-based label representation vectors. To fully exploit the aligned semantic information contained in the learned representation vectors of the instances, we develop a label propagation based testing procedure to classify the unlabeled instances from the unseen classes in the target domain. We conduct experiments on four standard zero-shot learning image datasets, by comparing the proposed approach to the state-of-the-art zero-shot learning methods. The empirical results demonstrate the efficacy of the proposed approach.

In this paper, we describe scalable parallel algorithms for symmetric sparse matrix factorization, analyze their performance and scalability, and present experimental results for up to 1,024 processors on a Gray T3D parallel computer. Through our analysis and experimental results, we demonstrate that our algorithms substantially improve the state of the art in parallel direct solution of sparse linear systems-both in terms of scalability and overall performance. It is a well known fact that dense matrix factorization scales well and can be implemented efficiently on parallel computers. In this paper, we present the first algorithms to factor a wide class of sparse matrices (including those arising from two- and three-dimensional finite element problems) that are asymptotically as scalable as dense matrix factorization algorithms on a variety of parallel architectures. Our algorithms incur less communication overhead and are more scalable than any previously known parallel formulation of sparse matrix factorization. Although, in this paper, we discuss Cholesky factorization of symmetric positive definite matrices, the algorithms can be adapted for solving sparse linear least squares problems and for Gaussian elimination of diagonally dominant matrices that are almost symmetric in structure. An implementation of one of our sparse Cholesky factorization algorithms delivers up to 20 GFlops on a Gray T3D for medium-size structural engineering and linear programming problems. To the best of our knowledge, this is the highest performance ever obtained for sparse Cholesky factorization on any supercomputer.

Many spectral unmixing approaches ranging from geometry, algebra to statistics have been proposed, in which nonnegative matrix factorization (NMF)-based ones form an important family. The original NMF-based unmixing algorithm loses the spectral and spatial information between mixed pixels when stacking the spectral responses of the pixels into an observed matrix. Therefore, various constrained NMF methods are developed to impose spectral structure, spatial structure, and spectral-spatial joint structure into NMF to enforce the estimated endmembers and abundances preserve these structures. Compared with matrix format, the third-order tensor is more natural to represent a hyperspectral data cube as a whole, by which the intrinsic structure of hyperspectral imagery can be losslessly retained. Extended from NMF-based methods, a matrix-vector nonnegative tensor factorization (NTF) model is proposed in this paper for spectral unmixing. Different from widely used tensor factorization models, such as canonical polyadic decomposition CPD) and Tucker decomposition, the proposed method is derived from block term decomposition, which is a combination of CPD and Tucker decomposition. This leads to a more flexible frame to model various application-dependent problems. The matrix-vector NTF decomposes a third-order tensor into the sum of several component tensors, with each component tensor being the outer product of a vector (endmember) and a matrix (corresponding abundances). From a formal perspective, this tensor decomposition is consistent with linear spectral mixture model. From an informative perspective, the structures within spatial domain, within spectral domain, and cross spectral-spatial domain are retreated interdependently. Experiments demonstrate that the proposed method has outperformed several state-of-the-art NMF-based unmixing methods.

Positron emission tomography (PET) is an important imaging technique to visualize a number of functions in the brain or human body. For reconstructing PET images from the sinogram data, an inverse problem has to be solved using numerical optimizations such as expectation-maximization (EM)-based methods. However, the standard EM method suffers from measurement noise added in the sinogram data. In this paper, we propose a new simultaneous PET image reconstruction and parts extraction method using constrained non-negative matrix factorization. In contrast that the many existing methods reconstruct a single PET image independently, we reconstruct the time-series of PET images simultaneously from the time-series of sinograms using non-negative matrix factorization. Furthermore, we impose the smoothness constraint for the temporal feature, and the exclusive LASSO-based sparseness constraint for the spatial feature for robust image reconstruction and physically meaningful feature extraction.

Based on the mining of micro- and macro-relationships of genetic variation and brain imaging data, imaging genetics has been widely applied in the early diagnosis of Alzheimer's disease (AD). However, effective integration of prior knowledge remains a barrier to determining the biological mechanism of AD. This paper proposes a new connectivity-based orthogonal sparse joint non-negative matrix factorization (OSJNMF-C) method based on integrating the structural magnetic resonance image, single nucleotide polymorphism and gene expression data of AD patients; the correlation information, sparseness, orthogonal constraint and brain connectivity information between the brain image data and genetic data are designed as constraints in the proposed algorithm, which efficiently improved the accuracy and convergence through multiple iterative experiments. Compared with the competitive algorithm, OSJNMF-C has significantly smaller related errors and objective function values than the competitive algorithm, showing its good anti-noise performance. From the biological point of view, we have identified some biomarkers and statistically significant relationship pairs of AD/mild cognitive impairment (MCI), such as rs75277622 and BCL7A, which may affect the function and structure of multiple brain regions. These findings will promote the prediction of AD/MCI.

The Gaussian and Gaussian-sigmoidal primaries are introduced as initialization functions in the nonnegative matrix factorization (NNMF) method to achieve more consistent error statistics. The proposed NNMF method with multiplicative update algorithm is employed for reconstruction of spectral data of Munsell and Colorchecker DC datasets from the corresponding CIEXYZ tristimulus values and the resultant achievements are compared with those obtained from PCA and the classical NNMF techniques. Both colorimetric and spectrophotometric errors between the original and the reconstructed spectra are used for the evaluation of reconstruction methods. While run to run variability of the classical NNMF method with multiplicative update algorithm is significantly remarkable, the stability of the suggested technique is fairly high, similar to the PCA method. The ensuing findings show that the embedding of Gaussian initializing vectors in NNMF method provide the all positive bases that enhance both the accuracy of Gaussian method and the reliability of NNMF technique.

This paper deals with algorithms for positive semidefinite matrix factorization (PSDMF). PSDMF is a recently-proposed extension of nonnegative matrix factorization with applications in combinatorial optimization, among others. In this paper, we focus on improving the local convergence of an alternating block gradient (ABC) method for PSDMF in a noise-free setting by replacing the quadratic objective function with the Poisson log-likelihood. This idea is based on truncated Wirtinger flow (TWF), a phase retrieval (PR) method that trims outliers in the gradient and thus regularizes it. Our motivation is a recent result linking PR with PSDMF. Our numerical experiments validate that the numerical benefits of TWF may carry over to PSDMF despite the more challenging setting, when initialized within its region of convergence. We then extend TWF from PR to affine rank minimization (ARM), and show that although the outliers are no longer an issue in the ARM setting, PSDMF with the new objective function may still achieves a smaller error for the same number of iterations. In a broader view, our results indicate that a proper choice of objective function may enhance convergence of matrix (or tensor) factorization methods.

There is currently an unprecedented demand for large-scale temporal data analysis due to the explosive growth of data. Dynamic topic modeling has been widely used in social and data sciences with the goal of learning latent topics that emerge, evolve, and fade over time. Previous work on dynamic topic modeling primarily employ the method of nonnegative matrix factorization (NMF), where slices of the data tensor are each factorized into the product of lower-dimensional nonnegative matrices. With this approach, however, information contained in the temporal dimension of the data is often neglected or underutilized. To overcome this issue, we propose instead adopting the method of nonnegative CANDECOMP/PARAPAC (CP) tensor decomposition (NNCPD), where the data tensor is directly decomposed into a minimal sum of outer products of nonnegative vectors, thereby preserving the temporal information. The viability of NNCPD is demonstrated through application to both synthetic and real data, where significantly improved results are obtained compared to those of typical NMF-based methods. The advantages of NNCPD over such approaches are studied and discussed. To the best of our knowledge, this is the first time that NNCPD has been utilized for the purpose of dynamic topic modeling, and our findings will be transformative for both applications and further developments.

The main aim of this paper is to develop a new algorithm for computing a nonnegative low multi‐rank tensor approximation for a nonnegative tensor. In the literature, there are several nonnegative tensor factorizations or decompositions, and their approaches are to enforce the nonnegativity constraints in the factors of tensor factorizations or decompositions. In this paper, we study nonnegativity constraints in tensor entries directly, and a low rank approximation for the transformed tensor by using discrete Fourier transformation matrix, discrete cosine transformation matrix, or unitary transformation matrix. This strategy is particularly useful in imaging science as nonnegative pixels appear in tensor entries and a low rank structure can be obtained in the transformation domain. We propose an alternating projections algorithm for computing such a nonnegative low multi‐rank tensor approximation. The convergence of the proposed projection method is established. Numerical examples for multidimensional images are presented to demonstrate that the performance of the proposed method is better than that of nonnegative low Tucker rank tensor approximation and the other nonnegative tensor factorizations and decompositions.