A Generalized CUR Decomposition for Matrix Pairs

Low rank approximation of matrices is an important tool in machine learning. Given a data matrix, low rank approximation helps to find factors, patterns, and provides concise representations for the data. Research on low rank approximation usually focuses on real matrices. However, in many applications data are binary (categorical) rather than continuous. This leads to the problem of low rank approximation of binary matrices. Here we are given a d x n binary matrix A and a small integer k < d. The goal is to find two binary matrices U and V of sizes d x k and k x n respectively, so that the Frobenius norm of A - U V is minimized. There are two models of this problem, depending on the definition of the dot product of binary vectors: The GF(2) model and the Boolean semiring model. Unlike low rank approximation of a real matrix which can be efficiently solved by Singular Value Decomposition, we show that approximation of a binary matrix is NP-hard, even for k=1.\nIn this paper, our main concern is the problem of Column Subset Selection (CSS), in which the low rank matrix U must be formed by k columns of the data matrix, and we are interested in the approximation ratio achievable by CSS for binary matrices. For the GF(2) model, we show that CSS has approximation ratio bounded by k/2+1+k/(2(2^k-1)) and this is asymptotically tight. For the Boolean model, it turns out that CSS is no longer sufficient to obtain a bound. We then develop a Generalized CSS (GCSS) procedure in which the columns of U are generated from Boolean formulas operating bitwise on selected columns of the data matrix. We show that the approximation ratio achieved by GCSS is bounded by 2^(k-1)+1, and argue that an exponential dependency on k is seems inherent.

In this thesis we analyze and develop methods based on model order reduction (MOR) for the solution of optimization problems constrained by time-dependent partial differential equations (PDEs). The methods combine a direct solution approach with model reduction via proper orthogonal decomposition (POD) and the discrete empirical interpolation method (DEIM). The reduced-order models (ROMs) are used to approximate the high-dimensional dynamic systems originating from a spatial discretization of a PDE. However, when used in an optimization algorithm, conventional POD/DEIM ROMs often lack the ability to give adequate approximations of the gradient. We propose methods for a suitable enhancement of the ROMs for the optimization purpose which are based on the inclusion of derivative information in the POD and DEIM subspaces. We distinguish two types of error between quantities evaluated with the high-dimensional model and its ROM approximation in dependency on the optimization variable q: The reconstruction error which is evaluated with the same q0 which is used constructing the ROM and the prediction error which assesses approximations at q with a ROM constructed at q0 different to q. The novel reconstruction results we present include estimates for solutions of the adjoint equation and the sensitivity equations as well as for the gradient of the objective function. Based on the estimates we explain how the POD and DEIM bases should be extended with either adjoint or sensitivity information. The enhanced ROMs allow control of the reconstruction error for the objective and its gradient up to machine precision. Moreover, we propose a POD prediction estimate for the objective of the optimization problem in a neighborhood of q where the ROM is constructed. In case of sensitivity-extended POD and DEIM bases we give an analogous result for solutions of the states. The derivative-extended ROMs are then used to develop adaptive algorithms for the solution of optimal control and parameter estimation problems which results in great runtime improvements for the optimization while ensuring high approximation quality of the solution of the original problem. For the parameter estimation case a novel a posteriori error estimate is proposed which assesses the quality of suboptimal solutions obtained with the ROM. A further fundamental contribution is a discussion of discretize-then-optimize (DTO) vs. optimize-then-discretize (OTD) approaches in the context of MOR for optimization. We analyze advantages and disadvantages of both approaches and discuss to which extent our methods exhibit properties of either strategy. We also give examples of representative optimization problems in which standard POD/DEIM ROMs show an inacceptable behavior and can be successfully solved by derivative-extended ROMs. We have further implemented the developed methods emphasizing an efficient realization which is important for the investigation of the MOR potential. We showcase the practical performance of the proposed algorithms and the superiority of derivative-extended over conventional ROMs on two academic and one industry-relevant application which exhibit a variety of challenges for the model reduction approach in optimization.

We propose a new algorithm for the computation of a singular value decomposition (SVD) low-rank approximation of a matrix in the matrix product operator (MPO) format, also called the tensor train matrix format. Our tensor network randomized SVD (TNrSVD) algorithm is an MPO implementation of the randomized SVD algorithm that is able to compute dominant singular values and their corresponding singular vectors. In contrast to the state-of-the-art tensor-based alternating least squares SVD (ALS-SVD) and modified alternating least squares SVD (MALS-SVD) matrix approximation methods, TNrSVD can be up to 13 times faster while achieving better accuracy. In addition, our TNrSVD algorithm also produces accurate approximations in particular cases where both ALS-SVD and MALS-SVD fail to converge. We also propose a new algorithm for the fast conversion of a sparse matrix into its corresponding MPO form, which is up to 509 times faster than the standard tensor train SVD method while achieving machine precision accuracy. The efficiency and accuracy of both algorithms are demonstrated in numerical experiments.

The work studies model reduction method for nonlinear systems based on proper orthogonal decomposition (POD) and discrete empirical interpolation method (DEIM). Instead of using the classical DEIM to directly approximate the nonlinear term of a system, our approach extracts the main part of the nonlinear term with a linear approximation before approximating the residual with the DEIM. We construct the linear term by Taylor series expansion and dynamic mode decomposition (DMD), respectively, so as to obtain a more accurate reconstruction of the nonlinear term. In addition, a novel error prediction model is devised for the POD-DEIM reduced systems by employing neural networks with the aid of error data. The error model is cheaply computable and can be adopted as a remedy model to enhance the reduction accuracy. Finally, numerical experiments are performed on two nonlinear problems to show the performance of the proposed method.

An accelerated least-squares approach is introduced in this work by incorporating a greedy point selection method with randomized singular value decomposition (rSVD) to reduce the computational complexity of missing data reconstruction. The rSVD is used to speed up the computation of a low-dimensional basis that is required for the least-squares projection by employing randomness to generate a small matrix instead of a large matrix from high-dimensional data. A greedy point selection algorithm, based on the discrete empirical interpolation method, is then used to speed up the reconstruction process in the least-squares approximation. The accuracy and computational time reduction of the proposed method are demonstrated through three numerical experiments. The first two experiments consider standard testing images with missing pixels uniformly distributed on them, and the last numerical experiment considers a sequence of many incomplete two-dimensional miscible flow images. The proposed method is shown to accelerate the reconstruction process while maintaining roughly the same order of accuracy when compared to the standard least-squares approach.

Efficient quaternion CUR decomposition based on discrete empirical interpolation method

The Proper Orthogonal Decomposition (POD) combined with the (Discrete) Empirical Interpolation Method (DEIM) has been successfully used to reduce the computation time of nonlinear Finite Element (FE) problems. The DEIM enables to alleviate the complexity cost of the nonlinear terms. In this communication, the Best Points Interpolation Method (BPIM) is compared with the DEIM in the case of nonlinear magnetostatics.

This study aims to develop a linear nonintrusive reduced-order model (NIROM) for facilitating combined fast parametric analysis and reconstruction of unsteady aerodynamic flow variables by using a novel signal interpolation approach, which is combined with a subspace-identification-based method. The NIROM is then applied to airfoil flutter and gust interaction problems where the assumption of linearized dynamics gives reasonably accurate estimates. In the traditional subspace-identification method, the training datasets of force coefficient signals corresponding to each flow parameter (like the Mach number and angle of attack) are required to generate the linear time-invariant system. The generation of the training signal for each flow parameter is the most computationally expensive process in any parametric reduced-order model (ROM). In the present work, the discrete empirical interpolation method (DEIM) combined with the radial basis function (RBF) method is used for the interpolation of training signals at a specific flow parameter to reduce the overall computational cost. The conventional nonintrusive approaches are generally constrained by the number of output channels, and hence not suitable for the reconstruction of unsteady surface force distribution or flowfield variables. A few suitable sensor locations using the DEIM are selected as output channels of the subspace-identification method to realize the full datasets of surface force distribution over the airfoil. Finally, a suitable algorithmic structure is proposed to achieve unsteady distributive flow-variable reconstruction and parametric variation simultaneously. The current signal interpolation-based NIROM is applied to a wing section with two- and four-degree-of-freedom structural motions.

A matrix DEIM technique for model reduction of nonlinear parametrized problems in cardiac mechanics

The present paper proposes a novel non-intrusive model reduction approach based on Proper Orthogonal Decomposition (POD), the Discrete Empirical Interpolation Method (DEIM) and Radial Basis Function (RBF) networks to efficiently predict production of oil and gas reservoirs. Provided a representative set of training reservoir scenarios, either POD or DEIM allows for effectively projecting input parameters (e.g., permeability, porosity), states (e.g., pressure, saturations) and outputs (e.g., well production curves) into a much lower dimension that retains the main features contained in the simulation system. In this work, these projections are applied across multiple levels to be able to collapse a large number of spatio-temporal correlations. It is observed that these projections can be effectively performed at a large extent regardless of the underlying geological complexity and operational constraints associated with the reservoir model. The RBF network provides a powerful means for developing learning functions from input-output relationships described by the reservoir dynamics entailed by multiple combinations of inputs and controls. In order to achieve a high degree of predictability from the resulting reduced model, the RBF network exploits locality by a means of Gaussian basis functions that are maximal at the sampled point and decrease monotonically with distance. Compared to multilayer perceptron networks (i.e., traditional artificial neural networks) RBF networks require less training and are less sensitive to the presence of noise in the data. In this regard, POD or DEIM acts as a data filter that additionally aids at designing a more compact RBF network representation suitable for targeting fast reservoir predictions. Numerical results show significant accelerations with respect to running the original simulation model on a set of field-motivated cases.

The design of commercial air transportation vehicles heavily relies on understanding and modeling fluid flows, which pose computational challenges due to their complexity and high degrees of freedom. To overcome these challenges, we propose a novel approach based on machine learning (ML) to construct reduced-order models (ROMs) using an autoencoder neural network coupled with a discrete empirical interpolation method (DEIM). This methodology combines the interpolation of nonlinear functions identified based on selected interpolation points using DEIM with an ML-based clustering algorithm that provides accurate predictions by spanning a low-dimensional subspace at a significantly lower computational cost. In this study, we demonstrate the effectiveness of our approach by the calculation of transonic flows over the National Advisory Committee of Aeronautics 0012 airfoil and the National Aeronautics and Space Administration Common Research Model wing. All the results confirm that the ROM captures high-dimensional parameter variations efficiently and accurately in transonic regimes, in which the nonlinearities are induced by shock waves, demonstrating the feasibility of the ROM for nonlinear aerodynamics problems with varying flow conditions.

The proper orthogonal decomposition (POD) is an efficient model order reduction method, which is frequently coupled with the discrete empirical interpolation method (DEIM) to solve nonlinear electromagnetic problems. A drawback of this method is that instabilities can occur related to the reduction operator of the nonlinear part. In this contribution, different DEIMs and the Gappy POD are employed and analyzed. Consecutively, the methods are employed to efficiently estimate the behavior of a permanent magnet synchronous machine in terms of global quantities, such as torque and iron losses.

The objective of this paper is to reduce the computational effort in reservoir flooding optimization problems by a combination of different optimization parametrization methods and model order reduction techniques. We compare three different parametrization methods that reduce the cardinality of the original infinite set of control-decision variables to a finite set. The three methods include a traditional piece-wise constant (PWC) approximation, a polynomial approximation by Chebyshev orthogonal polynomials and a piece-wise polynomial approximation by cubic Spline interpolation. The Proper Orthogonal Decomposition with Discrete Empirical Interpolation Method (POD-DEIM) accomplishes the reduced order modeling (ROM).. We compare a gradient-free global stochastic search approach and a gradient-based local search approach. We used Particle Swarm Optimization (PSO) as a gradient-free algorithm and Interior-Point Optimization (IPOPT) with L-BFGS method as a gradient-based algorithm. First, we compare the performances of the three parametrization methods solved by each optimizer, using fine scale simulations for an increasing level of parametrization refinement. Then, in the second part of this paper, we combine the parametrization methods with the reduced modeling workflow. For a given level of parametrization refinement, we compare the performance of each parametrization method coupled with POD-DEIM, and solved by each optimizer. In this part, we introduce an online training procedure, where the first optimization iteration is used to construct the snapshot matrix. The results demonstrate how refining the control approximation with more decision variables per well lead to better NPV values, but with a higher computational cost. The best NPV was achieved using the highest refining level with Chebyshev polynomial approximation. Both polynomial and piece-wise polynomial approximations served as better training sets for POD-DEIM leading to a more accurate and fast reduced model. With the strategy proposed, POD-DEIM showed the best optimization accuracy for Chebyshev polynomial with the gradient-free optimizer, thus permitting the use of the model reduction methodology for global-stochastic search methods. However, the gradient-based approach seems to consistently outperform the gradient-free approach in terms of NPV and number of iterations for the cases shown.

During lamination, residual thermo-mechanical stresses are induced in the encapsulated solar cells composing photovoltaic (PV) modules. Depending on the material and geometrical configuration of the layers of the laminate, this residual stress field can be beneficial since it may lead to a compressive stress state in Silicon and therefore crack closure effects in the presence of cracks, with a recovery of electrical conductivity in cracked solar cells. It is therefore important to investigate the distribution of thermo-mechanical stresses within the PV laminate with a view to optimizing the coupling between the electrical response and elastic deformation in the operation of PV modules. A promising approach proposed in the present thesis regards the prediction of residual stresses in composite laminates by using a shear-lag theory to model the epoxy-vinil-acetate polymeric layers, accounting for their thermo-visco-elastic response. Moreover, it will be shown that thermomechanical formulations for stress analysis of a PV laminate lead to a system of higher order ordinary differential equations or partial differential equations in which the exact solutions may be impossible to be determined in closed form and hence numerical schemes become desirable. However, the computational cost associated with the implementation of the numerical scheme may be significantly expensive. Therefore, a method to reduce the computational complexity is expected to be very important. To this aim, Model Order Reduction (MOR) techniques are applied hierarchically, first to the thermal system of a PV module in service, and then extended to coupled thermo-mechanical problems. A combination of proper orthogonal decomposition (POD) and discrete empirical interpolation method (DEIM) with a modified formulation is proposed for the first-order thermal equations of photovoltaic system during service and a new coupled second-order Krylov based formulation is developed for model order reduction of the coupled thermo-mechanical model of the photovoltaic module. The results of these reduction schemes show a huge computational gain in the reduced system solutions and a high accuracy of the reduced system outputs

Dynamical low-rank approximation allows for solving large-scale matrix differential equations (MDEs) with significantly fewer degrees of freedom and has been applied to a growing number of applications. However, most existing techniques rely on explicit time integration schemes. In this work, we introduce a cost-effective Newton’s method for the implicit time integration of stiff, nonlinear MDEs on low-rank matrix manifolds. Our methodology is focused on MDEs resulting from the discretization of random partial differential equations (PDEs), where the columns of the MDE can be solved independently. Cost-effectiveness is achieved by solving the MDE at the minimum number of entries required for a rank- r approximation. We present a novel cross low-rank approximation that requires solving the parametric PDE at r strategically selected parameters and O ( r ) grid points using Newton’s method. The selected random samples and grid points adaptively vary over time and are chosen using the discrete empirical interpolation method or similar techniques. The proposed methodology is developed for high-order implicit multi-step and Runge–Kutta schemes and incorporates rank adaptivity, allowing for dynamic rank adjustment over time to control error. Several analytical and PDE examples, including the stochastic Burgers’ and Gray–Scott equations, demonstrate the accuracy and efficiency of the presented methodology.