Unified Newton’s method for matrix decompositions

This paper is an updated and extended version of the paper "The QR Decomposition and the Singular Value Decomposition in the Symmetrized Max-Plus Algebra" (B. De Schutter and B. De Moor, SIAM J. Matrix Anal. Appl., 19 (1998), pp. 378--406). The max-plus algebra, which has maximization and addition as its basic operations, can be used to describe and analyze certain classes of discrete-event systems, such as flexible manufacturing systems, railway networks, and parallel processor systems. In contrast to conventional algebra and conventional (linear) system theory, the max-plus algebra and the max-plus-algebraic system theory for discrete-event systems are at present far from fully developed, and many fundamental problems still have to be solved. Currently, much research is going on to deal with these problems and to further extend the max-plus algebra and to develop a complete max-plus-algebraic system theory for discrete-event systems. In this paper we address one of the remaining gaps in the max-plus algebra by considering matrix decompositions in the symmetrized max-plus algebra. The symmetrized max-plus algebra is an extension of the max-plus algebra obtained by introducing a max-plus-algebraic analogue of the $-$-operator. We show that we can use well-known linear algebra algorithms to prove the existence of max-plus-algebraic analogues of basic matrix decompositions from linear algebra such as the QR decomposition, the singular value decomposition, the Hessenberg decomposition, the LU decomposition, and so on. These max-plus-algebraic matrix decompositions could play an important role in the max-plus-algebraic system theory for discrete-event systems.

QR and LU decompositions are the most important matrix decomposition algorithms. Many studies work on accelerating these algorithms by FPGA or ASIC in a case by case style. In this paper, we propose a unified framework for the matrix decomposition algorithms, combining three QR decomposition algorithms and LU algorithm with pivoting into a unified linear array structure. The QR and LU decomposition algorithms exhibit the same two-level loop structure and the same data dependency. Utilizing the similarities in loop structure and data dependency of matrix decomposition, we unify a fine-grained algorithm for all four matrix decomposition algorithms. Furthermore, we present a unified coprocessor structure with a scalable linear array of processing elements (PEs), in which four types of PEs are same in the structure of memory channels and PE connections, but the only difference exists in the internal structure of data path. Our unified co-processor, which is IEEE 32-bit floating-point precision, is implemented and mapped onto a Xilinx Virtex5 FPGA chip. Experimental results show that our co-processors can achieve speedup of 2.3 to 14.9 factors compared to a Pentium Dual CPU with double SSE threads.

Chapter 9 - Numerical Techniques

Applications using face biometric are ubiquitous in various domains. We propose an efficient method using Discrete Wavelet Transform (DWT), Extended Directional Binary codes (EDBC), three matrix decompositions and Singular Value Decomposition (SVD) for face recognition. The combined effect of Schur, Hessenberg and QR matrix decompositions are utilized with existing algorithm. The discrimination power between two different persons is justified using Average Overall Deviation (AOD) parameter. Fused EDBC and SVD features are considered for performance calculation. City-block and Euclidean Distance (ED) measure is used for matching. Performance is improved on YALE, GTAV and ORL face databases compared with existing methods.

In Gaussian elimination and LU factorization we solve a linear system by transforming it to triangular form. These are not the only kind of transformations that can be used for such a task. Matrices with orthonormal columns, called unitary matrices can be used to reduce a square matrix to upper triangular form and more generally a rectangular matrix to upper triangular (also called upper trapezoidal) form. This lead to a decomposition of a rectangular matrix known as a QR decomposition and a reduced form which we refer to as a QR factorization. The QR decomposition and factorization will be used in later chapters to solve least squares- and eigenvalue problems.

Lyapunov characteristic exponents are indicators of the nature and of the stability properties of solutions of differential equations. The estimation of Lyapunov exponents of algebraic multiplicity greater than 1 is troublesome. In this work, we intuitively derive an interpretation of higher multiplicity Lyapunov exponents in forms that occur in simple linear time invariant problems of engineering relevance. We propose a method to determine them from the real Schur decomposition of the state transition matrix of the linear, nonautonomous problem associated with the fiducial trajectory. So far, no practical way has been found to formulate the method as an algorithm capable of mitigating over- or underflow in the numerical computation of the state transition matrix. However, this interesting approach in some practical cases is shown to provide quicker convergence than usual methods like the discrete QR and the continuous QR and Singular Value Decomposition (SVD)methods when Lyapunov exponents with multiplicity greater than one are present.

The protection of sensitive and confidential data become a challenging task in the present scenario as more and more digital data is stored and transmitted between the end users. The privacy is vitally necessary in case of medical data, which contains the important information of the patients. In this article, a novel biometric inspired medical encryption technique is proposed based on newly introduced parameterized all phase orthogonal transformation (PR-APBST), singular value, and QR decomposition. The proposed technique utilizes the biometrics of the patient/owner to generate a key management system to obtain the parameters involved in the proposed technique. The medical image is then encrypted employing PR-APBST, QR and singular value decomposition and is ready for secure transmission or storage. Finally, a reliable decryption process is employed to reconstruct the original medical image from the encrypted image. The validity and feasibility of the proposed framework have been demonstrated using an extensive experiments on various medical images and security analysis.

Matrix factorizations or matrix decompositions are methods that represent a matrix as a product of two or more matrices. There are various types of matrix factorizations such as LU factorization, Cholesky factorization, singular value decomposition etc. Matrix factorization is widely used in pattern recognition, image denoising, data clustering etc. Motivated by these applications, some properties and applications of various types of matrix factorizations are studied. One of the purposes of matrix factorization is to ease the computation. Thus, comparisons in term of computation time of various matrix factorizations in different areas are carried out.

In this paper, a novel image watermarking scheme is proposed, based on Contourlet Transform (CT) and Matrix Factorization methods namely, Schur Factorization and Singular Value Decomposition (SVD). The original image and watermark are both decomposed by CT. The lowest frequency coefficients of original image after CT are factorized using SVD and that of watermark after CT are decomposed by Schur Factorization. The schur form of watermark is further factorized using SVD. The modified coefficients of singular values of watermark are embedded in the singular values of original image followed by inverse SVD and inverse CT computation that results in the watermarked image. The experimental results demonstrate that the scheme is resistant against signal processing attacks such as median filter, salt and pepper noise, Gaussian noise, compression and geometric attacks.KeywordsDigital watermarkingContourlet TransformSchur factorizationSVD factorization

The main aim of this paper is to compare the effectiveness of numerical techniques used for spatial multibody dynamics simulation by applying the natural complement method. In the paper, seven numerical schemes are considered: zero eigenvalue formulation, Pseudo Upper Triangular Decomposition, Schur decomposition, Singular Value Decomposition, QR decomposition, coordinate partitioning and Wang-Huston formulation. In order to illustrate the effectiveness of these schemes, two McPherson struts are considered. Simulations are performed with four error tolerance values and for three stabilization cases. Some suggestions on possible applications of the selected methods are formulated.

This paper presents a first attempt to perform the singular value decomposition of an infinite block-Hankel matrix. It is shown that singular value decomposition (SVD) of an infinite block-Hankel matrix can be reduced to the SVD of a matrix with finite dimension. The formulation is solely in terms of the coefficients of the transfer function matrix (TFM).The most of importance is that this method leads to a new formulation of L∞-optimization problems, Hankel-norm model reduction problems and a new algorithm for minimal balanced realization of multivariable systems

The classification of objects or quantities in all fields of science depends on the quality of the features used for classifying them. This includes, for example, classification of phenomena described by nonstationary processes such as electrocardiograms, seismic geophysics signals, submarine transient acoustic signals, and speech signals for recognition. This paper presents a new matrix decomposition that is used to obtain a set of principal features from time-frequency representations for classifying nonstationary time series processes. This new matrix decomposition is based on a transformation of the orthonormal basis from singular value decomposition (SVD). The new basis set yields extrema of the first moment for each vector in the new basis set. These basis sets for time and frequency can then be used to construct features relating to the location and spread of each energy density highlight in the time-frequency plane. This new matrix decomposition is presented in this paper along with a simple example to illustrate its application.

1. Linear systems: Introduction to Linear Systems. Gauss Elimination. Numerical Solutions. 2. Vectors: Vector Operations. Dot Product. Span. Linear Independence. The Product [Ax]. Cross Product. Lines, Planes, and Hyperplanes. 3. Matrices: Matrix Operations. Matrix Inverse. Elementary and Invertible Matrices. LU Factorization. 4. Vector spaces: Subspaces of [R to the n Power]. Vector Spaces. Linear Independence Bases. Dimension. Coordinate Vectors and Change of Basis. Rank and Nullity. Applications to Coding Theory. 5. Linear transformations: Matrix Transformations. Linear Transformations. Kernel and Range. The Matrix of a Linear Transformation. The Algebra of Linear Transformations. 6. Determinants: Determinants Cofactor Expansion. Properties of Determinants. The Adjoint Cramer's Rule. Determinants with Permutations. 7. Eigenvalues and eigenvectors: Eigenvalues and Eigenvectors. Diagonalization. Approximations of Eigenvalues and Eigenvectors. Applications to Dynamical Systems. Applications to Markov Chains. 8. Dot and inner products: Orthogonal Sets and Matrices. Orthogonal Projections Gram-Schmidt Process. The QR Factorization. Least Squares. Orthogonalization of Symmetric Matrices. Quadratic Forms and Conic Sections. The Singular Value Decomposition (SVD). Inner Products. Appendices: Answers to selected exercises Each chapter concludes with Applications, Mini-Projects, and Computer Exercises.

We present algorithms to determine the number of nonzeros in each row and column of the factors of a sparse matrix, for both the QR factorization and the LU factorization with partial pivoting. The algorithms use only the nonzero structure of the input matrix, and run in time nearly linear in the number of nonzeros in that matrix. They may be used to set up data structures or schedule parallel operations in advance of the numerical factorization. The row and column counts we compute are upper bounds on the actual counts. If the input matrix is strong Hall and there is no coincidental numerical cancellation, the counts are exact for QR factorization and are the tightest bounds possible for LU factorization. These algorithms are based on our earlier work on computing row and column counts for sparse Cholesky factorization, plus an efficient method to compute the column elimination tree of a sparse matrix without explicitly forming the product of the matrix and its transpose.

The modern graphics processing unit (GPU) found in many standard personal computers is a highly parallel math processor capable of nearly 1 TFLOPS peak throughput at a cost similar to a high-end CPU and an excellent FLOPS/watt ratio. High-level linear algebra operations are computationally intense, often requiring O(N3) operations and would seem a natural fit for the processing power of the GPU. Our work is on CULA, a GPU accelerated implementation of linear algebra routines. We present results from factorizations such as LU decomposition, singular value decomposition and QR decomposition along with applications like system solution and least squares. The GPU execution model featured by NVIDIA GPUs based on CUDA demands very strong parallelism, requiring between hundreds and thousands of simultaneous operations to achieve high performance. Some constructs from linear algebra map extremely well to the GPU and others map poorly. CPUs, on the other hand, do well at smaller order parallelism and perform acceptably during low-parallelism code segments. Our work addresses this via hybrid a processing model, in which the CPU and GPU work simultaneously to produce results. In many cases, this is accomplished by allowing each platform to do the work it performs most naturally.