Low-rank matrix and tensor approximations for compression of machine-learning interatomic potentials.

The spontaneous internalization of electrostatic interactions by large atomic models in molten salts

Low rank approximation of matrices is an important tool in machine learning. Given a data matrix, low rank approximation helps to find factors, patterns, and provides concise representations for the data. Research on low rank approximation usually focuses on real matrices. However, in many applications data are binary (categorical) rather than continuous. This leads to the problem of low rank approximation of binary matrices. Here we are given a d x n binary matrix A and a small integer k < d. The goal is to find two binary matrices U and V of sizes d x k and k x n respectively, so that the Frobenius norm of A - U V is minimized. There are two models of this problem, depending on the definition of the dot product of binary vectors: The GF(2) model and the Boolean semiring model. Unlike low rank approximation of a real matrix which can be efficiently solved by Singular Value Decomposition, we show that approximation of a binary matrix is NP-hard, even for k=1.\nIn this paper, our main concern is the problem of Column Subset Selection (CSS), in which the low rank matrix U must be formed by k columns of the data matrix, and we are interested in the approximation ratio achievable by CSS for binary matrices. For the GF(2) model, we show that CSS has approximation ratio bounded by k/2+1+k/(2(2^k-1)) and this is asymptotically tight. For the Boolean model, it turns out that CSS is no longer sufficient to obtain a bound. We then develop a Generalized CSS (GCSS) procedure in which the columns of U are generated from Boolean formulas operating bitwise on selected columns of the data matrix. We show that the approximation ratio achieved by GCSS is bounded by 2^(k-1)+1, and argue that an exponential dependency on k is seems inherent.

Given the superposition of a low-rank matrix plus the product of a known fat\ncompression matrix times a sparse matrix, the goal of this paper is to\nestablish deterministic conditions under which exact recovery of the low-rank\nand sparse components becomes possible. This fundamental identifiability issue\narises with traffic anomaly detection in backbone networks, and subsumes\ncompressed sensing as well as the timely low-rank plus sparse matrix recovery\ntasks encountered in matrix decomposition problems. Leveraging the ability of\n$\\ell_1$- and nuclear norms to recover sparse and low-rank matrices, a convex\nprogram is formulated to estimate the unknowns. Analysis and simulations\nconfirm that the said convex program can recover the unknowns for sufficiently\nlow-rank and sparse enough components, along with a compression matrix\npossessing an isometry property when restricted to operate on sparse vectors.\nWhen the low-rank, sparse, and compression matrices are drawn from certain\nrandom ensembles, it is established that exact recovery is possible with high\nprobability. First-order algorithms are developed to solve the nonsmooth convex\noptimization problem with provable iteration complexity guarantees. Insightful\ntests with synthetic and real network data corroborate the effectiveness of the\nnovel approach in unveiling traffic anomalies across flows and time, and its\nability to outperform existing alternatives.\n

Low-rank matrix factorizations such as Principal Component Analysis (PCA), Singular Value Decomposition (SVD) and Non-negative Matrix Factorization (NMF) are a large class of methods for pursuing the low-rank approximation of a given data matrix. The conventional factorization models are based on the assumption that the data matrices are contaminated stochastically by some type of noise. Thus the point estimations of low-rank components can be obtained by Maximum Likelihood (ML) estimation or Maximum a posteriori (MAP). In the past decade, a variety of probabilistic models of low-rank matrix factorizations have emerged. The most significant difference between low-rank matrix factorizations and their corresponding probabilistic models is that the latter treat the low-rank components as random variables. This paper makes a survey of the probabilistic models of low-rank matrix factorizations. Firstly, we review some probability distributions commonly-used in probabilistic models of low-rank matrix factorizations and introduce the conjugate priors of some probability distributions to simplify the Bayesian inference. Then we provide two main inference methods for probabilistic low-rank matrix factorizations, i.e., Gibbs sampling and variational Bayesian inference. Next, we classify roughly the important probabilistic models of low-rank matrix factorizations into several categories and review them respectively. The categories are performed via different matrix factorizations formulations, which mainly include PCA, matrix factorizations, robust PCA, NMF and tensor factorizations. Finally, we discuss the research issues needed to be studied in the future.

Given the superposition of a low-rank matrix plus the product of a known fat compression matrix times a sparse matrix, the goal of this paper is to establish conditions under which exact recovery of the low-rank and sparse components becomes possible. This fundamental identifiability task subsumes compressed sensing and the timely low-rank plus sparse matrix recovery encountered in matrix decomposition problems. Leveraging the ability of l 1 - and nuclear norms to recover sparse and low-rank matrices, a convex program is formulated to estimate the unknowns. Analysis and simulations confirm that the said convex program can recover the unknowns for sufficiently low-rank and sparse enough components, along with a compression matrix possessing an isometry property.

Least Squares Support Vector Machine (LS-SVM) has been recently applied to non-parametric identification of Linear Parameter Varying (LPV) systems, described by the AutoRegressive with eXogenous input (ARX). However, the online application of LPV-ARX system in the LS-SVM setting requires high computational time, related to the number of training data used to compute the coefficients of the identified model, limiting the possibility to use the method to real-time applications. In this paper, the authors propose the Low-Rank (LR) matrix approximation and a pruning based approach to compute a sparse solution. In particular, the pruning algorithm is considered to compute off-line a sparse solution of Lagrangian multipliers and then speed up the testing stage, whereas the LR matrix approximation allows to speed up the training stage. The proposed approach has been tested by identifying a subsystem of a vehicle powertrain model by the input/output data collected from the simulation model. The proposed approach has been compared with respect to the standard approach based on LS-SVM. The methods are tested on the considered real-world problem and the proposed approach permits to reduce the execution time of about 77% on average in the considered identification problem, corresponding to a degradation of the identification result less than 0.2% with respect to the standard solution.

In this paper, we present a novel low-rank matrix factorization algorithm with adaptive graph regularizer (LMFAGR). We extend the recently proposed low-rank matrix with manifold regularization (MMF) method with an adaptive regularizer. Different from MMF, which constructs an affinity graph in advance, LMFAGR can simultaneously seek graph weight matrix and low-dimensional representations of data. That is, graph construction and low-rank matrix factorization are incorporated into a unified framework, which results in an automatically updated graph rather than a predefined one. The experimental results on some data sets demonstrate that the proposed algorithm outperforms the state-of-the-art low-rank matrix factorization methods.

Compressive sensing enables efficient acquisition of hyperspectral images (HSIs) by assuming high redundancy on natural scenes. Several reconstruction algorithms have been proposed to retrieve the underlying image, and most of them take advantage of the spatial and spectral correlations. However, reconstruction may not be necessary in certain applications such as land cover classification. Instead of knowing the full image, researchers are interested in features that could be extracted directly from the compressed measurements, which provide high inference capabilities. Low-rank (LR) matrix approximation has been widely used in feature extraction (FE), because it reduces the data dimension and computational cost. Therefore, in this paper, compressive hyperspectral imaging and FE are combined in a framework for HSI classification using an LR matrix approximation model. In the proposed framework, the compressed measurements are acquired from a single-pixel spectrometer. Instead of using the traditional high-complexity reconstruction model, an LR matrix factorization problem is formulated. The LR problem maximizes the posterior distribution with respect to the feature space and coefficients, and it is numerically solved based on an alternating optimization strategy. By incorporating spatial information, the numerical procedure minimizes the total variational of the feature coefficients subject to an orthogonality constraint for the feature space. Experiments on real HSIs show that the proposed approach can provide equally competitive classification results when compared to the traditional approach that performs FE and classification on the recovered images.

The explosive growth of data has caused users to spend considerable time and effort finding the items they need. Various recommender systems have been created to provide convenience for users. This paper proposes a low-rank and sparse matrix factorization with prior relations (LSMF-PR) recommendation model, which predicts users’ ratings for items through a sum of the learned low-rank matrix and sparse matrix. Thus, unlike traditional matrix factorization approaches, our method can alleviate the error propagation produced by intermediate outputs. The LSMF-PR integrates user relationships and item relationships as prior information. User relationships in different recommendation scenarios are extracted by the corresponding social relations of the users, and item relationships are obtained from the similarity of the item content. Therefore, the sparsity and cold start problems can be effectively reduced with prior information. Furthermore, our model has better interpretability since it reveals the low-rank and sparse features of the ratings. Experiments are conducted on four real-world datasets to validate the performance of our proposed method.

Understanding the physical origin of mechanisms in random alloys that lead to the formation of microstructures requires an understanding of their average behavior and, equally important, the role of local fluctuations around the average. Material properties of random alloys can be computed using direct simulations on random configurations. However, some properties are very difficult to compute, for others it is not even fully understood how to compute them using random sampling, in particular, interaction energies between multiple defects. To that end, we develop an atomistic model that does the averaging on the level of interatomic potentials. With such an average interatomic interaction model the problem of averaging via random sampling is bypassed since the problem of computing material properties on random configurations reduces to the problem of computing material properties on single crystals, the average alloy, which can be done using standard techniques.To be predictive, we develop our average model on the class of linear machine-learning interatomic potentials (MLIPs). To that end, using tools from higher-order statistics, we derive an analytic expansion of average many-body per-atom energies of linear MLIPs in terms of average tensor products of the feature vectors of the underlying machine-learning model that scales linearly with the size of an atomic neighborhood. In order to avoid forming higher-order tensors composed of products of feature vectors, we develop an implementation using equivariant tensor network (ETN) potentials, a class of linear MLIPs, that contracts the feature vectors to small-sized tensors, and then takes the average. We validate the average ETN potential by demonstrating the convergence of direct Monte Carlo simulations to the exact value for properties of the NbMoTaW medium-entropy alloy. Simulating average properties of dislocations has thus far only been possible with average embedded atom method potentials that, however, predict artificial polarized cores. Here, we show that average ETN potentials predict the compact screw dislocation core structure, in agreement with density functional theory. Hence, we anticipate that our model will become useful for understanding mechanistic origins of material properties and for developing predictive models of mechanical properties of random alloys.

With the rapid growth of scientific publications, it is hard for researchers to acquire appropriate papers that meet their expectations. Recommendation system for scientific articles is an essential technology to overcome this problem. In this paper, we propose a novel low-rank and sparse matrix factorization-based paper recommendation (LSMFPRec) method for authors. The proposed method seamlessly combines low-rank and sparse matrix factorization method with fine-grained paper and author affinity matrixes that are extracted from heterogeneous scientific network. Thus, it can effectively alleviate the sparsity and cold start problems that exist in traditional matrix factorization based collaborative filtering methods. Moreover, LSMFPRec can significantly reduce the error propagated from intermediate outputs. In addition, the proposed method essentially captures the low-rank and sparse characteristics that exist in scientific rating activities; therefore, it can generate more reasonable predicted ratings for influential and uninfluential papers. The effectiveness of the proposed LSMFPRec is demonstrated by the recommendation evaluation conducted on the AAN and CiteULike data sets.

With the increasing popularity of streaming tensor data such as videos and audios, tensor factorization and completion have attracted much attention recently in this area. Existing work usually assume that streaming tensors only grow in one mode. However, in many real-world scenarios, tensors may grow in multiple modes (or dimensions), i.e., multi-aspect streaming tensors. Standard streaming methods cannot directly handle this type of data elegantly. Moreover, due to inevitable system errors, data may be contaminated by outliers, which cause significant deviations from real data values and make such research particularly challenging. In this paper, we propose a novel method for Outlier-Robust Multi-Aspect Streaming Tensor Completion and Factorization (OR-MSTC), which is a technique capable of dealing with missing values and outliers in multi-aspect streaming tensor data. The key idea is to decompose the tensor structure into an underlying low-rank clean tensor and a structured-sparse error (outlier) tensor, along with a weighting tensor to mask missing data. We also develop an efficient algorithm to solve the non-convex and non-smooth optimization problem of OR-MSTC. Experimental results on various real-world datasets show the superiority of the proposed method over the baselines and its robustness against outliers.

Low rank approximation in simulations of quantum algorithms

Fixed-precision randomized quaternion singular value decomposition algorithm for low-rank quaternion matrix approximations

<abstract><p>Trace norm regularization is a widely used approach for learning low rank matrices. A standard optimization strategy is based on formulating the problem as one of low rank matrix factorization which, however, leads to a non-convex problem. In practice this approach works well, and it is often computationally faster than standard convex solvers such as proximal gradient methods. Nevertheless, it is not guaranteed to converge to a global optimum, and the optimization can be trapped at poor stationary points. In this paper we show that it is possible to characterize all critical points of the non-convex problem. This allows us to provide an efficient criterion to determine whether a critical point is also a global minimizer. Our analysis suggests an iterative meta-algorithm that dynamically expands the parameter space and allows the optimization to escape any non-global critical point, thereby converging to a global minimizer. The algorithm can be applied to problems such as matrix completion or multitask learning, and our analysis holds for any random initialization of the factor matrices. Finally, we confirm the good performance of the algorithm on synthetic and real datasets.</p></abstract>