Updating the Triangular Factorization of a Matrix

Semi-symmetric three-way arrays are essential tools in blind source separation (BSS) particularly in independent component analysis (ICA). These arrays can be built by resorting to higher order statistics of the data. The canonical polyadic (CP) decomposition of such semi-symmetric three-way arrays allows us to identify the so-called mixing matrix, which contains the information about the intensities of some latent source signals present in the observation channels. In addition, in many applications, such as the magnetic resonance spectroscopy (MRS), the columns of the mixing matrix are viewed as relative concentrations of the spectra of the chemical components. Therefore, the two loading matrices of the three-way array, which are equal to the mixing matrix, are nonnegative. Most existing CP algorithms handle the symmetry and the nonnegativity separately. Up to now, very few of them consider both the semi-nonnegativity and the semi-symmetry structure of the three-way array. Nevertheless, like all the methods based on line search, trust region strategies, and alternating optimization, they appear to be dependent on initialization, requiring in practice a multi-initialization procedure. In order to overcome this drawback, we propose two new methods, called JD LU + and JD QR + , to solve the problem of CP decomposition of semi-nonnegative semi-symmetric three-way arrays. Firstly, we rewrite the constrained optimization problem as an unconstrained one. In fact, the nonnegativity constraint of the two symmetric modes is ensured by means of a square change of variable. Secondly, a Jacobi-like optimization procedure is adopted because of its good convergence property. More precisely, the two new methods use LU and QR matrix factorizations, respectively, which consist in formulating high-dimensional optimization problems into several sequential polynomial and rational subproblems. By using both LU and QR matrix factorizations, we aim at studying the influence of the used matrix factorization. Numerical experiments on simulated arrays emphasize the advantages of the proposed methods especially the one based on LU factorization, in the presence of high-variance model error and of degeneracies such as bottlenecks. A BSS application on MRS data confirms the validity and improvement of the proposed methods.

Multiple LU factorizations of a singular matrix

Matrix factorizations or matrix decompositions are methods that represent a matrix as a product of two or more matrices. There are various types of matrix factorizations such as LU factorization, Cholesky factorization, singular value decomposition etc. Matrix factorization is widely used in pattern recognition, image denoising, data clustering etc. Motivated by these applications, some properties and applications of various types of matrix factorizations are studied. One of the purposes of matrix factorization is to ease the computation. Thus, comparisons in term of computation time of various matrix factorizations in different areas are carried out.

LU decomposition and Toeplitz decomposition of a neural network

Exact solving of systems of linear equations (SLEs) is a fundamental subroutine within number theory, formal verification of mathematical proofs, and exact-precision mathematical programming. Moreover, efficient exact SLE solution methods could be valuable for a growing body of science and engineering applications where current fixed-precision standards have been deemed inadequate. This article contains key derivations relating, and computational tests comparing, two exact direct solution frameworks: roundoff-error-free (REF) LU factorization and rational arithmetic LU factorization. Specifically, both approaches solve the linear system Ax = b by factoring the matrix A into the product of a lower triangular (L) and upper triangular (U) matrix, A = LU . Most significantly, the featured findings reveal that the integer-preserving REF factorization framework solves dense SLEs one order of magnitude faster than the exact rational arithmetic approach while requiring half the memory. Since rational LU is utilized for basic solution validation in exact linear and mixed-integer programming, these results offer preliminary evidence of the potential of the REF factorization framework to be utilized within this specific context. Additionally, this article develops and analyzes an efficient streamlined version of Edmonds’s Q-matrix approach that can be implemented as another basic solution validation approach. Further experiments demonstrate that the REF factorization framework also outperforms this alternative integer-preserving approach in terms of memory requirements and computational effort. General purpose codes to solve dense SLEs exactly via any of the aforementioned methods have been made available to the research and academic communities.

Given a multiport transfer function A , the question arises whether it can be realized using feedback loops of shortest possible length. The most elegant way to achieve this is to realize A in cascade form, with each subsystem in the cascade of smallest possible degree. Any transfer matrix of finite degree is a matrix whose entries are rational functions of the complex variable p , and a cascade synthesis of such a p -matrix will bear heavily on the properties of the factorization of rational p -matrices into simpler factors. We will call a p -matrix of the first degree an elementary factor. It is shown that 1) a nonsingular rational p -matrix can be factored into a minimal product of elementary factors provided either all poles or zeros are of the first order but not necessarily of the first degree; 2) that any nonsingular matrix can be factored into a product of elementary factors with the bound on the number of factors being a function of the degree of the matrix; and 3) that there are matrices which cannot be factored out minimally. Factorization procedures in special cases were first deduced by Belevitch and Youla, and later, using an improved criterion for degree reduction, by the first author. These procedures all use unitary or so-called J -unitary elementary factors. The theory presented here uses a general type of elementary factor to attack the problem of cascade decomposition of a general nonsingular transfer function. Thus the question whether a general (rational) multiport can be synthesized by means of feedback loops of length one has to be answered negatively although a very large number of systems can, and a criterion is given to distinguish the two cases. Also, an algorithm and an example are presented to exhibit how the factorization (if it exists) can be constructed.

We present algorithms to determine the number of nonzeros in each row and column of the factors of a sparse matrix, for both the QR factorization and the LU factorization with partial pivoting. The algorithms use only the nonzero structure of the input matrix, and run in time nearly linear in the number of nonzeros in that matrix. They may be used to set up data structures or schedule parallel operations in advance of the numerical factorization. The row and column counts we compute are upper bounds on the actual counts. If the input matrix is strong Hall and there is no coincidental numerical cancellation, the counts are exact for QR factorization and are the tightest bounds possible for LU factorization. These algorithms are based on our earlier work on computing row and column counts for sparse Cholesky factorization, plus an efficient method to compute the column elimination tree of a sparse matrix without explicitly forming the product of the matrix and its transpose.

Explicit formula for the inverse of a tridiagonal matrix by backward continued fractions

Neutrosophic numbers and matrices extend classical fuzzy set theory by incorporating three components to represent uncertainty: truth membership, indeterminacy membership, and falsity membership degrees. Real neutrosophic matrices consist of neutrosophic numbers as their entries, offering a structure to model uncertain and imprecise information across various applications. This paper investigates operations on real neutrosophic matrices, such as addition, multiplication, and scalar multiplication, and defines how these operations function within the neutrosophic framework. The paper also examines the use of LU decomposition as an efficient method for solving systems of real neutrosophic linear algebraic equations. LU decomposition is a numerical technique that breaks down a matrix into lower and upper triangular matrices, enabling the solution of linear systems through forward and backward substitution.

A new approach to the application of direct method in petroleum reservoir simulation is introduced, This new approach is based on a modified form of the alternating diagonal ordering scheme that will rearrange each type of elements along different secondary diagonals in the coefficient matrix. In a typical 2-dimensional 3-phase petroleum reservoir simulation problem of 345 grid system, the new method is substantially faster than the band algorithm applied to the matrix resulting from the standard ordering. Introduction In petroleum reservoir simulation, a linear system of equations is obtained. The coefficient matrix of such a linear system takes different forms depending on the ordering of the grid system. The resulting system of equations can be solved either by direct methods or by iterative methods. Direct solutions are more reliable, but they normally require larger storage space and more computer time. Therefore, a development of a better direct method approach is needed. The ordering technique can be improved by the use of processes which are geared specifically to the direct solution. The method is the generate-and-solve algorithm used by Woo, et al. The drawback of this technique is the fact it requires large computer storage and overhead work to store and generate the nonlooping code. The yet known systematic optimal ordering scheme in petroleum reservoir simulation is the alternating diagonal scheme developed by Price and Coats. However, there is a major drawback to this ordering scheme. In the alternating diagonal scheme, the elements in the upper right and lower left corners of the coefficient matrix do not form perfect secondary diagonals. Thus, the solution of the resulting matrix requires band algorithm which treats the zeros within the band as nonzeros. The treatment of the zeros as nonzeros makes the band matrix technique require larger storage space and more computational labor. In this paper a modification to the alternating diagonal ordering is proposed. This modification will rearrange each type of elements along different secondary diagonals in the coefficient matrix. By this rearrangement no new element will be introduced in the process of factorization except in the lower right process of factorization except in the lower right corner of the composite matrix. Also the location of the original and new elements are known beforehand. This enables us to develop a general procedure to solve the system of equations in reservoir simulation. This general procedure eliminates the overhead calculations and extra storage needed for the generate-and-solve algorithm. Also this general procedure would eliminate the extra calculations required procedure would eliminate the extra calculations required on the zero elements in the band matrix algorithm. A NEW DIRECT SOLUTION APPROACH The new direct solution approach will be outlined to solve the following system of equations (1) The new approach begins with the definition of a new ordering scheme. Then the LU factorization will be applied to this new ordering. Finally the implementation of the new ordering scheme in typical reservoir simulators will be outlined. RESTRICTED ALTERNATING DIAGONAL ORDERING Here a modification to the alternating diagonal ordering that produces perfect secondary diagonals in the original A matrix is proposed. This ordering will be called restricted alternating diagonal (RAD). Figure 1 depicts the RAD ordering. Figure 2 shows the resulting coefficient matrix corresponding to RAD ordering scheme. The numbering of this scheme must go along the shortest diagonal to reduce the band width of the new nonzero elements that will be generated later in the process of factorization. process of factorization.

The LU factorization is an important numerical algorithm for solving system of linear equations. This paper proposes a novel approach for computing the LU factorization in parallel on multicore architectures. It improves the overall performance and also achieves the numerical quality of the standard LU factorization with partial pivoting. While the update of the trailing submatrix is computationally intensive and highly parallel, the inherently problematic portion of the LU factorization is the panel factorization due to its memory-bound characteristic and the atomicity of selecting the appropriate pivots. We remedy this in our new approach to LU factorization of (narrow and tall) panel submatrices. We use a parallel fine-grained recursive formulation of the factorization. It is based on conflict-free partitioning of the data and lock-less synchronization mechanisms. Our implementation lets the overall computation naturally flow with limited contention. Our recursive panel factorization provides the necessary performance increase for the inherently problematic portion of the LU factorization of square matrices. A large panel width results in larger Amdahl's fraction as our experiments have revealed which is consistent with related efforts. The performance results of our implementation reveal superlinear speedup and far exceed what can be achieved with equivalent MKL and/or LAPACK routines.

Application demands and grand challenges in numerical simulation require for both highly capable computing platforms and efficient numerical solution schemes. Power constraints and further miniaturization of modern and future hardware give way for multi- and manycore processors with increasing fine-grained parallelism and deeply nested hierarchical memory systems -- as already exemplified by recent graphics processing units. Accordingly, numerical schemes need to be adapted and re-engineered in order to deliver scalable solutions across diverse processor configurations. Portability of parallel software solutions across emerging hardware platforms is another challenge. This work investigates multi-coloring and re-ordering schemes for block Gauß-Seidel methods and, in particular, for incomplete LU factorizations with and without fill-ins. We consider two matrix re-ordering schemes that deliver flexible and efficient parallel preconditioners. The general idea is to generate block decompositions of the system matrix such that the diagonal blocks are diagonal itself. In such a way, parallelism can be exploited on the block-level in a scalable manner. Our goal is to provide widely applicable, out-of-the-box preconditioners that can be used in the context of finite element solvers. We propose a new method for anticipating the fill-in pattern of ILU($p$) schemes which we call the power($q$)-pattern method. This method is based on an incomplete factorization of the system matrix $A$ subject to a predetermined pattern given by the matrix power $|A|^(p+1)$ and its associated multi-coloring permutation $. We prove that the obtained sparsity pattern is a superset of our modified ILU($p$) factorization applied to pi A pi^(-1). As a result, this modified ILU($p$) applied to multi-colored system matrix has no fill-ins in its diagonal blocks. This leads to an inherently parallel execution of triangular ILU($p$) sweeps. In addition, we describe the integration of the preconditioners into the HiFlow$^3$ open-source finite element package that provides a portable software solution across diverse hardware platforms. On this basis, we conduct performance analysis across a variety of test problems on multi-core CPUs and GPUs that proves efficiency, scalability and flexibility of our approach. Our preconditioners achieve a solver acceleration by a factor of up to 1.5, 8 and 85 for three different test problems. The GPU versions of the preconditioned solver are by a factor of up to 4 faster than an OpenMP parallel version on eight cores.

SummarySequentially linear analysis (SLA), an event‐by‐event procedure for finite element (FE) simulation of quasi‐brittle materials, is based on sequentially identifying a critical integration point in the FE model, to reduce its strength and stiffness, and the corresponding critical load multiplier (λcrit), to scale the linear analysis results. In this article, two strategies are proposed to efficiently reuse previous stiffness matrix factorisations and their corresponding solutions in subsequent linear analyses, since the global system of linear equations representing the FE model changes only locally. The first is based on a direct solution method in combination with the Woodbury matrix identity, to compute the inverse of a low‐rank corrected stiffness matrix relatively cheaply. The second is a variation of the traditional incomplete LU preconditioned conjugate gradient method, wherein the preconditioner is the complete factorisation of a previous analysis step's stiffness matrix. For both the approaches, optimal points at which the factorisation is recomputed are determined such that the total analysis time is minimised. Comparison and validation against a traditional parallel direct sparse solver, with regard to a two‐dimensional (2D) and three‐dimensional (3D) benchmark study, illustrates the improved performance of the Woodbury‐based direct solver over its counterparts, especially for large 3D problems.

In the previous installment, we looked at some generalpurpose linear solvers—some approaches to solving systems of simultaneous linear algebraic equations for cases in which there are no distinctive features offered by the coefficient matrix used in the description of the system. We concentrated on Gaussian elimination and LU decomposition. The computational effort required to solve an n-square system via either of those techniques is O(n3), although, as was pointed out, solution via LU decomposition requires only O(n2) arithmetic operations after the factorization is complete. Thus, solving a system of linear equations for more than one right-hand side (i.e., for more than one forcing vector) enjoys significant computational economy when LU decomposition is employed instead of Gaussian elimination.

Preface. Conventions and Notations. 1 An Introduction To Maple. 1.1 The Commands . 1.2 Programming. 2 Linear Systems of Equations and Matrices. 2.1 Linear Systems of Equations. 2.2 Augmented Matrix of a Linear System and Row Operations. 2.3 Some Matrix Arithmetic. 3 Gauss-Jordan Elimination and Reduced Row Echelon Form. 3.1 Gauss-Jordan Elimination and rref. 3.2 Elementary Matrices. 3.3 Sensitivity of Solutions to Error in the Linear System. 4 Applications of Linear Systems and Matrices. 4.1 Applications of Linear Systems to Geometry. 4.2 Applications of Linear Systems to Curve Fitting. 4.3 Applications of Linear Systems to Economics. 4.4 Applications of Matrix Multiplication to Geometry. 4.5 An Application of Matrix Multiplication to Economics. 5 Determinants, Inverses and Cramer s Rule. 5.1 Determinants and Inverses from the Adjoint Formula. 5.2 Determinants by Expanding Along Any Row or Column . 5.3 Determinants Found by Triangularizing Matrices. 5.4 LU Factorization. 5.5 Inverses from rref. 5.6 Cramer s Rule. 6 Basic Linear Algebra Topics. 6.1 Vectors. 6.2 Dot Product. 6.3 Cross Product. 6.4 Vector Projection. 7 A Few Advanced Linear Algebra Topics. 7.1 Rotations in Space. 7.2 Rolling a Circle Along a Curve. 7.3 The TNB Frame. 8 Independence, Basis and Dimension for Subspaces of Rn. 8.1 Subspaces of Rn. 8.2 Independent and Dependent Sets of Vectors in Rn. 8.3 Basis and Dimension for Subspaces of Rn. 8.4 Vector Projection onto a Subspace of Rn. 8.5 The Gram-Schmidt Orthonormalization Process. 9 Linear Maps from Rn to Rm. 9.1 Basics About Linear Maps. 9.2 The Kernel and Image Subspaces of a Linear Map. 9.3 Composites of Two Linear maps and Inverses. 9.4 Change of Bases for the Matrix Representation of a Linear Map. 10 The Geometry of Linear and Affine Maps. 10.1 The Effect of a Linear Map on Area and Arclength in Two Dimensions. 10.2 The Decomposition of Linear Maps into Rotations, Reflections and Rescalings in R2. 10.3 The Effect of Linear Maps on Volume, Area and Arclength in R3. 10.4 Rotations, Reflections and Rescalings in Three Dimensions. 10.5 Affine Maps. 11 Least Squares Fits and Pseudoinverses. 11.1 Pseudoinverse to a Non-Square Matrix and Almost Solving an Overdetermined Linear System. 11.2 Fits and Pseudoinverses. 11.3 Least Squares Fits and Pseudoinverses. 12 Eigenvalues and Eigenvectors. 12.1 What Are Eigenvalues and Eigenvectors, and Why Do We Need Them? 12.2 Summary of Definitions and Methods for Computing Eigenvalues and Eigenvectors as well as the Exponential of a Matrix. 12.3 Applications of the Diagonalizability of Square Matrices. 12.4 Solving a Square First Order Linear. System of Differential Equations ... 12.5 Basic Facts About Eigenvalues and Eigenvectors, and Diagonalizability. 12.6 The Geometry of the Ellipse Using Eigenvalues and Eigenvectors. 12.7 A Maple Eigen-Procedure. Bibliography. Indices. Keyword Index. Index of Maple Commands and Packages.