Incremental Condition Estimator for Parallel Sparse Matrix Factorization

In this paper, we describe scalable parallel algorithms for symmetric sparse matrix factorization, analyze their performance and scalability, and present experimental results for up to 1,024 processors on a Gray T3D parallel computer. Through our analysis and experimental results, we demonstrate that our algorithms substantially improve the state of the art in parallel direct solution of sparse linear systems-both in terms of scalability and overall performance. It is a well known fact that dense matrix factorization scales well and can be implemented efficiently on parallel computers. In this paper, we present the first algorithms to factor a wide class of sparse matrices (including those arising from two- and three-dimensional finite element problems) that are asymptotically as scalable as dense matrix factorization algorithms on a variety of parallel architectures. Our algorithms incur less communication overhead and are more scalable than any previously known parallel formulation of sparse matrix factorization. Although, in this paper, we discuss Cholesky factorization of symmetric positive definite matrices, the algorithms can be adapted for solving sparse linear least squares problems and for Gaussian elimination of diagonally dominant matrices that are almost symmetric in structure. An implementation of one of our sparse Cholesky factorization algorithms delivers up to 20 GFlops on a Gray T3D for medium-size structural engineering and linear programming problems. To the best of our knowledge, this is the highest performance ever obtained for sparse Cholesky factorization on any supercomputer.

Parallel solution of sparse linear least squares problems on distributed-memory multiprocessors

Non-negative Matrix Factorization (NMF) is widely used in image clustering; however, it has inherent limitations, including its unsupervised nature, lack of sparsity constraints, inability to leverage label information, and difficulty in capturing the geometric structure and nonlinear characteristics of data. To address these limitations, this paper proposes a Sparse Kernel Graph-regularized Discriminant Non-negative Matrix Factorization (SKGDNMF) algorithm. The algorithm innovatively adopts a dual normalization strategy, which involves column normalization for the basis matrix and row normalization for the coefficient matrix, and integrates spectral clustering to construct an end-to-end deep clustering framework. By applying sparse regularization to the coefficient matrix, the model’s robustness is significantly improved; this regularization forms complementary optimization with graph regularization, thereby effectively alleviating overfitting. A three-dimensional golden-section parameter optimization method is employed to determine key parameters, which enhances the algorithm’s practicality. Comparative experiments conducted on multiple datasets show that SKGDNMF significantly outperforms mainstream algorithms in terms of sparsity, robustness, and clustering performance, indicating its superior effectiveness for image clustering tasks.

This paper proposes parallel method of non-negative large sparse matrix factorization - very popular technique in computational linguistics. Proposed blocks factorization approach using Latent Dirichlet allocation, for speedup of non-negative sparse extra-large linguistic matrix and tensor factorization. The proposed method also supports the scalability of the model.

The explosive growth of data has caused users to spend considerable time and effort finding the items they need. Various recommender systems have been created to provide convenience for users. This paper proposes a low-rank and sparse matrix factorization with prior relations (LSMF-PR) recommendation model, which predicts users’ ratings for items through a sum of the learned low-rank matrix and sparse matrix. Thus, unlike traditional matrix factorization approaches, our method can alleviate the error propagation produced by intermediate outputs. The LSMF-PR integrates user relationships and item relationships as prior information. User relationships in different recommendation scenarios are extracted by the corresponding social relations of the users, and item relationships are obtained from the similarity of the item content. Therefore, the sparsity and cold start problems can be effectively reduced with prior information. Furthermore, our model has better interpretability since it reveals the low-rank and sparse features of the ratings. Experiments are conducted on four real-world datasets to validate the performance of our proposed method.

In this article, the fast algorithms for the discrete Tchebichef transform (DTT) are proposed for input sequences of lengths in the range from 3 to 8. At present, DTT is widely applied in signal processing, image compression, and video coding. The review of the articles related to fast DTT algorithms has shown that such algorithms are mainly developed for input signal lengths 4 and 8. However, several problems exist for which signal and image processing with different apertures is required. To avoid this shortcoming, the structural approach and a sparse matrix factorization are applied in this paper to develop fast real DTT algorithms for short-length input signals. According to the structural approach, the rows and columns of the transform matrix are rearranged, possibly by changing the signs of some rows or columns. Next, the matched submatrix templates are extracted from the matrix structure and decomposed into a matrix product to construct the factorization of an initial matrix. A sparse matrix factorization assumes that the butterfly architecture can be extracted from the transform matrix. Combining the structural approach with a sparse matrix factorization, we obtained the matrix representation with reduced computational complexity. Based on the obtained matrix representation, the fast algorithms were developed for the real DTT via the data flow graphs. The fast algorithms for integer DTT can be easily obtained using the constructed data flow graphs. To confirm the correctness of the designed algorithms, the MATLAB R2023b software was applied. The constructed factorizations of the real DTT matrices reduce the number of multiplication operations by 78% on average compared to the direct matrix-vector product at signal lengths in the range from 3 to 8. The number of additions decreased by 5% on average within the same signal length range.

Based on the mining of micro- and macro-relationships of genetic variation and brain imaging data, imaging genetics has been widely applied in the early diagnosis of Alzheimer's disease (AD). However, effective integration of prior knowledge remains a barrier to determining the biological mechanism of AD. This paper proposes a new connectivity-based orthogonal sparse joint non-negative matrix factorization (OSJNMF-C) method based on integrating the structural magnetic resonance image, single nucleotide polymorphism and gene expression data of AD patients; the correlation information, sparseness, orthogonal constraint and brain connectivity information between the brain image data and genetic data are designed as constraints in the proposed algorithm, which efficiently improved the accuracy and convergence through multiple iterative experiments. Compared with the competitive algorithm, OSJNMF-C has significantly smaller related errors and objective function values than the competitive algorithm, showing its good anti-noise performance. From the biological point of view, we have identified some biomarkers and statistically significant relationship pairs of AD/mild cognitive impairment (MCI), such as rs75277622 and BCL7A, which may affect the function and structure of multiple brain regions. These findings will promote the prediction of AD/MCI.

High-dimensional data often lie in low-dimensional subspaces instead of the whole space. Subspace clustering is a problem to analyze data that are from multiple low-dimensional subspaces and cluster them into the corresponding subspaces. In this work, we propose a $(k,k)$-sparse matrix factorization method for subspace clustering. In this method, data itself is considered as the 'dictionary', and each data point is represented as a linear combination of the basis of its cluster in the dictionary. Thus, the coefficient matrix is low-rank and sparse. With an appropriate permutation, it is also blockwise with each block corresponding to a cluster. With an assumption that each block is no more than $k$-by-$k$ in matrix recovery, we seek a low-rank and $(k,k)$-sparse coefficient matrix, which will be used for the construction of affinity matrix in spectral clustering. The advantage of our proposed method is that we recover a coefficient matrix with $(k,k)$-sparse and low-rank simultaneously, which is better fit for subspace clustering. Numerical results illustrate the effectiveness that it is better than SSC and LRR in real-world classification problems such as face clustering and motion segmentation.

Zero-shot learning, a special case of unsupervised domain adaptation where the source and target domains have disjoint label spaces, has become increasingly popular in the computer vision community. In this paper, we propose a novel zero-shot learning method based on discriminative sparse non-negative matrix factorization. The proposed approach aims to identify a set of common high-level semantic components across the two domains via non-negative sparse matrix factorization, while enforcing the representation vectors of the images in this common component-based space to be discriminatively aligned with the attribute-based label representation vectors. To fully exploit the aligned semantic information contained in the learned representation vectors of the instances, we develop a label propagation based testing procedure to classify the unlabeled instances from the unseen classes in the target domain. We conduct experiments on four standard zero-shot learning image datasets, by comparing the proposed approach to the state-of-the-art zero-shot learning methods. The empirical results demonstrate the efficacy of the proposed approach.

A wide range of problems in computer graphics and vision can be formulated as sparse least squares problems. For example, Laplacian mesh deformation, Least Squares Conformal Maps, Poisson image editing, and as-rigid-as-possible (ARAP) image warping involve solving a linear or non-linear sparse least squares problem. High performance is crucial in many of these applications for interactive user feedback. For these applications, we show that the matrices produced by factorization methods such as QR have a special structure: the off-diagonal blocks are low-rank. We leverage this property to produce a fast sparse approximate QR factorization, spaQR, for these matrices in near-linear time. In our benchmarks, spaQR shows up to 57% improvement over solving the normal equations using Cholesky and 63% improvement over a standard preconditioner with Conjugate Gradient Least Squares (CGLS).

In this paper, the generation of sequency-ordered complex Hadamard transform (SCHT) based on the complex Rademacher matrices is presented. The exponential form of SCHT is also derived, and the proof for the unitary property of SCHT is given. Using the sparse matrix factorization, the fast and efficient algorithm to compute the SCHT transform is developed, and its computation load is described. Certain properties of the SCHT matrices are derived and analyzed with the discussion of SCHT applications in spectrum analysis and image watermarking. Relations of SCHT with fast Fourier transform (FFT) and unified complex Hadamard transform (UCHT) are discussed.

A parallel version of the Householder algorithm with column pivoting is introduced for computing the QR factorization of a matrix. Local pivoting allows efficient implementation of the algorithm on a parallel machine; in particular, it is implemented on one with a distributed architecture. An inexpensive but reliable incremental condition estimator is used to control the selection of pivot columns by obtaining cheap estimates for the smallest singular value of the currently created upper triangular matrix R. Numerical experiments show that the local pivoting strategy behaves about as well as the traditional global pivoting strategy. They also show the advantages of incorporating the controlled pivoting strategy into the traditional QR algorithm to guard against the known pathological cases.

We study the problem of large-scale network embedding, which aims to learn\nlatent representations for network mining applications. Previous research shows\nthat 1) popular network embedding benchmarks, such as DeepWalk, are in essence\nimplicitly factorizing a matrix with a closed form, and 2)the explicit\nfactorization of such matrix generates more powerful embeddings than existing\nmethods. However, directly constructing and factorizing this matrix---which is\ndense---is prohibitively expensive in terms of both time and space, making it\nnot scalable for large networks.\n In this work, we present the algorithm of large-scale network embedding as\nsparse matrix factorization (NetSMF). NetSMF leverages theories from spectral\nsparsification to efficiently sparsify the aforementioned dense matrix,\nenabling significantly improved efficiency in embedding learning. The\nsparsified matrix is spectrally close to the original dense one with a\ntheoretically bounded approximation error, which helps maintain the\nrepresentation power of the learned embeddings. We conduct experiments on\nnetworks of various scales and types. Results show that among both popular\nbenchmarks and factorization based methods, NetSMF is the only method that\nachieves both high efficiency and effectiveness. We show that NetSMF requires\nonly 24 hours to generate effective embeddings for a large-scale academic\ncollaboration network with tens of millions of nodes, while it would cost\nDeepWalk months and is computationally infeasible for the dense matrix\nfactorization solution. The source code of NetSMF is publicly available\n(https://github.com/xptree/NetSMF).\n

This paper describes an efficient approach to sparse matrix factorization on vector supercomputers. The approach is suitable for application domains like circuit simulation that require the repeated direct solution of unsymmetric sparse linear systems of equations with identical zero-nonzero structure. An Overlap-Scatter data structure is used to represent the sparse matrix, enabling the use of multiple operand access modes to achieve higher performance than earlier proposed approaches. The superior performance of the new solver is demonstrated using a number of matrices derived from circuit simulation runs.

With the rapid growth of scientific publications, it is hard for researchers to acquire appropriate papers that meet their expectations. Recommendation system for scientific articles is an essential technology to overcome this problem. In this paper, we propose a novel low-rank and sparse matrix factorization-based paper recommendation (LSMFPRec) method for authors. The proposed method seamlessly combines low-rank and sparse matrix factorization method with fine-grained paper and author affinity matrixes that are extracted from heterogeneous scientific network. Thus, it can effectively alleviate the sparsity and cold start problems that exist in traditional matrix factorization based collaborative filtering methods. Moreover, LSMFPRec can significantly reduce the error propagated from intermediate outputs. In addition, the proposed method essentially captures the low-rank and sparse characteristics that exist in scientific rating activities; therefore, it can generate more reasonable predicted ratings for influential and uninfluential papers. The effectiveness of the proposed LSMFPRec is demonstrated by the recommendation evaluation conducted on the AAN and CiteULike data sets.