Linear algebra with applications

(NOTE: Every chapter ends with Questions to Guide Your Review, Practice Exercises, and Additional Exercises.) P. Preliminaries. Real Numbers and the Real Line. Coordinates, Lines, and Increments. Functions. Shifting Graphs. Trigonometric Functions. 1. Limits and Continuity. Rates of Change and Limits. Rules for Finding Limits. Target Values and Formal Definitions of Limits. Extensions of the Limit Concept. Continuity. Tangent Lines. 2. Derivatives. The Derivative of a Function. Differentiation Rules. Rates of Change. Derivatives of Trigonometric Functions. The Chain Rule. Implicit Differentiation and Rational Exponents. Related Rates of Change. 3. Applications of Derivatives. Extreme Values of Functions. The Mean Value Theorem. The First Derivative Test for Local Extreme Values. Graphing with y e and y . Limits as x a a, Asymptotes, and Dominant Terms. Optimization. Linearization and Differentials. Newton's Method. 4. Integration. Indefinite Integrals. Differential Equations, Initial Value Problems, and Mathematical Modeling. Integration by Substitution--Running the Chain Rule Backward. Estimating with Finite Sums. Riemann Sums and Definite Integrals. Properties, Area, and the Mean Value Theorem. Substitution in Definite Integrals. Numerical Integration. 5. Applications of Integrals. Areas Between Curves. Finding Volumes by Slicing. Volumes of Solids of Revolution--Disks and Washers. Cylindrical Shells. Lengths of Plan Curves. Areas of Surfaces of Revolution. Moments and Centers of Mass. Work. Fluid Pressures and Forces. The Basic Pattern and Other Modeling Applications. 6. Transcendental Functions. Inverse Functions and Their Derivatives. Natural Logarithms. The Exponential Function. ax and logax. Growth and Decay. L'Hopital's Rule. Relative Rates of Growth. Inverse Trigonomic Functions. Derivatives of Inverse Trigonometric Functions Integrals. Hyperbolic Functions. First Order Differential Equations. Euler's Numerical Method Slope Fields. 7. Techniques of Integration. Basic Integration Formulas. Integration by Parts. Partial Fractions. Trigonometric Substitutions. Integral Tables and CAS. Improper Integrals. 8. Infinite Series. Limits of Sequences of Numbers. Theorems for Calculating Limits of Sequences. Infinite Series. The Integral Test for Series of Nonnegative Terms. Comparison Tests for Series of Nonnegative Terms. The Ratio and Root Tests for Series of Nonnegative Terms. Alternating Series, Absolute and Conditional Convergence. Power Series. Taylor and Maclaurin Series. Convergence of Taylor Series Error Estimates. Applications of Power Series. 9. Conic Sections, Parametrized Curves, and Polar Coordinates. Conic Sections and Quadratic Equations. Classifying Conic Sections by Eccentricity. Quadratic Equations and Rotations. Parametrizations of Plan Curves. Calculus with Parametrized Curves. Polar Coordinates. Graphing in Polar Coordinates. Polar Equations for Conic Sections. Integration in Polar Coordinates. 10. Vectors and Analytic Geometry in Space. Vectors in the Plane. Cartesian (Rectangular) Coordinates and Vectors in Space. Dot Products. Cross Products. Lines and Planes in Space. Cylinders and Quadric Surfaces. Cylindrical and Spherical Coordinates. 11. Vector-Valued Functions and Motion in Space. Vector-Valued Functions and Space Curves. Modeling Projectile Motion. Arc Length and the Unit Tangent Vector T. Curvature, Torison, and the TNB Frame. Planetary Motion and Satellites. 12. Multivariable Functions and Partial Derivatives. Functions of Several Variables. Limits and Continuity. Partial Derivatives. Differentiability, Linearization, and Differentials. The Chain Rule. Partial Derivatives with Constrained Variables. Directional Derivatives, Gradient Vectors, and Tangent Planes. Extreme Values and Saddle Points. Lagrange Multipliers. Taylor's Formula. 13. Multiple Integrals. Double Integrals. Areas, Moments, and Centers of Mass. Double Integrals in Polar Form. Triple Integrals in Rectangular Coordinates. Masses and Moments in Three Dimensions. Triple Integrals in Cylindrical and Spherical Coordinates. Substitutions in Multiple Integrals. 14. Integration in Vector Fields. Line Integrals. Vector Fields, Work, Circulation, and Flux. Path Independence, Potential Functions, and Conservative Fields. Green's Theorem in the Plane. Surface Area and Surface Integrals. Parametrized Surfaces. Stokes's Theorem. The Divergence Theorem and a Unified Theory. Appendices. Mathematical Induction. Proofs of Limit Theorems in Section 1.2. Complex Numbers. Simpson's One-Third Rule. Cauchy's Mean Value Theorem and the Stronger Form of L'Hopital's Rule. Limits that Arise Frequently. The Distributive Law for Vector Cross Products. Determinants and Cramer's Rule. Euler's Theorem and the Increment Theorem.

This paper is an updated and extended version of the paper "The QR Decomposition and the Singular Value Decomposition in the Symmetrized Max-Plus Algebra" (B. De Schutter and B. De Moor, SIAM J. Matrix Anal. Appl., 19 (1998), pp. 378--406). The max-plus algebra, which has maximization and addition as its basic operations, can be used to describe and analyze certain classes of discrete-event systems, such as flexible manufacturing systems, railway networks, and parallel processor systems. In contrast to conventional algebra and conventional (linear) system theory, the max-plus algebra and the max-plus-algebraic system theory for discrete-event systems are at present far from fully developed, and many fundamental problems still have to be solved. Currently, much research is going on to deal with these problems and to further extend the max-plus algebra and to develop a complete max-plus-algebraic system theory for discrete-event systems. In this paper we address one of the remaining gaps in the max-plus algebra by considering matrix decompositions in the symmetrized max-plus algebra. The symmetrized max-plus algebra is an extension of the max-plus algebra obtained by introducing a max-plus-algebraic analogue of the $-$-operator. We show that we can use well-known linear algebra algorithms to prove the existence of max-plus-algebraic analogues of basic matrix decompositions from linear algebra such as the QR decomposition, the singular value decomposition, the Hessenberg decomposition, the LU decomposition, and so on. These max-plus-algebraic matrix decompositions could play an important role in the max-plus-algebraic system theory for discrete-event systems.

Preface. Conventions and Notations. 1 An Introduction To Maple. 1.1 The Commands . 1.2 Programming. 2 Linear Systems of Equations and Matrices. 2.1 Linear Systems of Equations. 2.2 Augmented Matrix of a Linear System and Row Operations. 2.3 Some Matrix Arithmetic. 3 Gauss-Jordan Elimination and Reduced Row Echelon Form. 3.1 Gauss-Jordan Elimination and rref. 3.2 Elementary Matrices. 3.3 Sensitivity of Solutions to Error in the Linear System. 4 Applications of Linear Systems and Matrices. 4.1 Applications of Linear Systems to Geometry. 4.2 Applications of Linear Systems to Curve Fitting. 4.3 Applications of Linear Systems to Economics. 4.4 Applications of Matrix Multiplication to Geometry. 4.5 An Application of Matrix Multiplication to Economics. 5 Determinants, Inverses and Cramer s Rule. 5.1 Determinants and Inverses from the Adjoint Formula. 5.2 Determinants by Expanding Along Any Row or Column . 5.3 Determinants Found by Triangularizing Matrices. 5.4 LU Factorization. 5.5 Inverses from rref. 5.6 Cramer s Rule. 6 Basic Linear Algebra Topics. 6.1 Vectors. 6.2 Dot Product. 6.3 Cross Product. 6.4 Vector Projection. 7 A Few Advanced Linear Algebra Topics. 7.1 Rotations in Space. 7.2 Rolling a Circle Along a Curve. 7.3 The TNB Frame. 8 Independence, Basis and Dimension for Subspaces of Rn. 8.1 Subspaces of Rn. 8.2 Independent and Dependent Sets of Vectors in Rn. 8.3 Basis and Dimension for Subspaces of Rn. 8.4 Vector Projection onto a Subspace of Rn. 8.5 The Gram-Schmidt Orthonormalization Process. 9 Linear Maps from Rn to Rm. 9.1 Basics About Linear Maps. 9.2 The Kernel and Image Subspaces of a Linear Map. 9.3 Composites of Two Linear maps and Inverses. 9.4 Change of Bases for the Matrix Representation of a Linear Map. 10 The Geometry of Linear and Affine Maps. 10.1 The Effect of a Linear Map on Area and Arclength in Two Dimensions. 10.2 The Decomposition of Linear Maps into Rotations, Reflections and Rescalings in R2. 10.3 The Effect of Linear Maps on Volume, Area and Arclength in R3. 10.4 Rotations, Reflections and Rescalings in Three Dimensions. 10.5 Affine Maps. 11 Least Squares Fits and Pseudoinverses. 11.1 Pseudoinverse to a Non-Square Matrix and Almost Solving an Overdetermined Linear System. 11.2 Fits and Pseudoinverses. 11.3 Least Squares Fits and Pseudoinverses. 12 Eigenvalues and Eigenvectors. 12.1 What Are Eigenvalues and Eigenvectors, and Why Do We Need Them? 12.2 Summary of Definitions and Methods for Computing Eigenvalues and Eigenvectors as well as the Exponential of a Matrix. 12.3 Applications of the Diagonalizability of Square Matrices. 12.4 Solving a Square First Order Linear. System of Differential Equations ... 12.5 Basic Facts About Eigenvalues and Eigenvectors, and Diagonalizability. 12.6 The Geometry of the Ellipse Using Eigenvalues and Eigenvectors. 12.7 A Maple Eigen-Procedure. Bibliography. Indices. Keyword Index. Index of Maple Commands and Packages.

The protection of sensitive and confidential data become a challenging task in the present scenario as more and more digital data is stored and transmitted between the end users. The privacy is vitally necessary in case of medical data, which contains the important information of the patients. In this article, a novel biometric inspired medical encryption technique is proposed based on newly introduced parameterized all phase orthogonal transformation (PR-APBST), singular value, and QR decomposition. The proposed technique utilizes the biometrics of the patient/owner to generate a key management system to obtain the parameters involved in the proposed technique. The medical image is then encrypted employing PR-APBST, QR and singular value decomposition and is ready for secure transmission or storage. Finally, a reliable decryption process is employed to reconstruct the original medical image from the encrypted image. The validity and feasibility of the proposed framework have been demonstrated using an extensive experiments on various medical images and security analysis.

In Gaussian elimination and LU factorization we solve a linear system by transforming it to triangular form. These are not the only kind of transformations that can be used for such a task. Matrices with orthonormal columns, called unitary matrices can be used to reduce a square matrix to upper triangular form and more generally a rectangular matrix to upper triangular (also called upper trapezoidal) form. This lead to a decomposition of a rectangular matrix known as a QR decomposition and a reduced form which we refer to as a QR factorization. The QR decomposition and factorization will be used in later chapters to solve least squares- and eigenvalue problems.

Extract Although examples drive this book, we first provide a whirlwind survey of the general concepts of operator theory. We do not expect the student to master these topics now since they are covered in future chapters. A Hilbert space H is a complex vector space endowed with an inner product ⟨x,y⟩ that defines a norm ‖x‖=⟨x,x⟩ with respect to which H is (Cauchy) complete. The inner product on a Hilbert space satisfies the Cauchy–Schwarz inequality |⟨x,y⟩|⩽‖x‖‖y‖ for all x,y∈H⁠. Examples of Hilbert spaces include Cn (complex Euclidean space), ℓ2 (the space of square-summable complex sequences), and L2[0,1] (the Lebesgue space of square-integrable, complex-valued functions on [0,1]⁠). Vectors x,y in a Hilbert space H are orthogonal if ⟨x,y⟩=0⁠. The dimension of a Hilbert space H is the cardinality of a maximal set of nonzero orthogonal vectors. This book is almost exclusively concerned with Hilbert spaces of countable dimension. Every such Hilbert space has an orthonormal basis (un)n=1∞⁠, a (possibly finite) maximal orthogonal set such that ⟨um,un⟩=δmn for all m,n⩾1⁠. With respect to an orthonormal basis (un)n=1∞⁠, each x∈H enjoys a generalized Fourier expansion x=∑n=1∞⟨x,un⟩un that satisfies Parseval’s identity ‖x‖2=∑n=1∞|⟨x,un⟩|2. A subspace (a norm-closed linear submanifold) of H is itself a Hilbert space with the operations inherited from H⁠. If (wn)n=1∞ is a (possibly finite) orthonormal basis for a subspace M of H and x∈H⁠, then PMx=∑n=1∞⟨x,wn⟩wn belongs to M and satisfies ‖x−PMx‖⩽‖x−y‖ for every y∈M⁠. In short, PMx is the unique closest vector to x in M⁠. Furthermore, PM defines a linear transformation on H whose range is M⁠. It is called the orthogonal projection of H onto M and it satisfies PM2=PM and ⟨PMx,y⟩=⟨x,PMy⟩ for all x,y∈H⁠. Let H and K be Hilbert spaces. A linear transformation A:H→K is bounded if ‖A‖=sup{‖Ax‖K:‖x‖H=1} is finite. Let B(H,K) denote the set of bounded linear operators from H to K⁠. We write B(H) for B(H,H)⁠. The quantity ‖A‖ is the norm of A. Since B(H) is closed under addition and scalar multiplication, it is a vector space. Furthermore, since ‖A+B‖⩽‖A‖+‖B‖ and ‖cA‖=|c|‖A‖ for all A,B∈B(H) and c∈C⁠, it follows that B(H) is a normed vector space. Endowed with this norm, B(H) is complete and thus forms a Banach space. Moreover, the composition AB belongs to B(H) and ‖AB‖⩽‖A‖‖B‖ for all A,B∈B(H)⁠. Therefore, B(H) is a Banach algebra. For each A∈B(H,K)⁠, there is a unique A*∈B(K,H) such that ⟨Ax,y⟩K=⟨x,A*y⟩H for all x∈H and y∈K⁠. The operator A* is the adjoint of A; it is the analogue of the conjugate transpose of a matrix. One can show that A↦A* is conjugate linear, that A**=A⁠, ‖A‖=‖A*‖⁠, and ‖A*A‖=‖A‖2⁠. This additional structure upgrades B(H) from a Banach algebra to a C*-algebra. One can exploit adjoints to obtain information about the kernel and range of an operator. For most of the operators A∈B(H) covered in this book, we give the matrix representation [A]=[⟨Auj,ui⟩]i,j=1∞ with respect to an orthonormal basis (un)n=1∞ for H⁠. This matrix representation [A] defines a bounded operator x↦[A]x on the Hilbert space ℓ2 of square summable sequences that is structurally identical to A. Schur’s theorem helps us determine which infinite matrices define bounded operators on ℓ2⁠. Many of the operators covered in this book, such as the Cesàro operator, the Volterra operator, weighted shifts, Toeplitz operators, and Hankel operators, have fascinating structured-matrix representations. An important class of operators is the compact operators. These are the A∈B(H) such that (Axn)n=1∞ has a convergent subsequence whenever (xn)n=1∞ is a bounded sequence in H⁠. Equivalently, an operator is compact if it takes each bounded set to one whose closure is compact. Each finite-rank operator is compact and every compact operator is the norm limit of finite-rank operators. The compact operators form a norm-closed, *-closed ideal within B(H)⁠. Some operators have a particularly close relationship with their adjoint. For example, the operator M on L2[0,1] defined by (Mf)(x)=xf(x) satisfies M*=M⁠. Such operators are selfadjoint. If µ is a positive finite compactly supported Borel measure on C⁠, then the operator N on L2(μ) defined by (Nf)(z)=zf(z) satisfies (N*f)(z)=z―f(z)⁠, and thus N*N=NN*⁠. Such operators are normal. The operator (Uf)(eiθ)=eiθf(eiθ) on L2(T) satisfies U*U=UU*=I⁠. Such operators are unitary. Unitary operators preserve the ambient structure of Hilbert spaces and can serve as a vehicle to relate A∈B(H) with B∈B(K)⁠. We say that A is unitarily equivalent to B if there is a unitary U∈B(H,K) such that UAU*=B⁠. Unitary equivalence is often used to identify seemingly complicated operators with relatively simple ones.

Reproducibility is the ability to obtain bitwise identical results from different runs of the same program on the same input data, regardless of the available computing resources, or how they are scheduled. Recently, techniques have been proposed to attain reproducibility for BLAS operations, all of which rely on reproducibly computing the floating-point sum and dot product. Nonetheless, a reproducible BLAS library does not automatically translate into a reproducible higher-level linear algebra library, especially when communication is optimized. For instance, for the QR factorization, conventional algorithms such as Householder transformation or Gram-Schmidt process can be used to reproducibly factorize a floating-point matrix by fixing the high-level order of computation, for example column-by-column from left to right, and by using reproducible versions of level-1 BLAS operations such as dot product and 2-norm. In a massively parallel environment, those algorithms have high communication cost due to the need for synchronization after each step. The Tall-Skinny QR algorithm obtains much better performance in massively parallel environments by reducing the number of messages by a factor of n to O(log(P)) where P is the processor count, by reducing the number of reduction operations to O(1). Those reduction operations however are highly dependent on the network topology, in particular the number of computing nodes, and therefore are difficult to implement reproducibly and with reasonable performance. In this paper we present a new technique to reproducibly compute a QR factorization for a tall skinny matrix, which is based on the Cholesky QR algorithm to attain reproducibility as well as to improve communication cost, and the iterative refinement technique to guarantee the accuracy of the computed results. Our technique exhibits strong scalability in massively parallel environments, and at the same time can provide results of almost the same accuracy as the conventional Householder QR algorithm unless the matrix is extremely badly conditioned, in which case a warning can be given. Initial experimental results in Matlab show that for not too ill-conditioned matrices whose condition number is smaller than sqrt(1/e) where e is the machine epsilon, our technique runs less than 4 times slower than the built-in Matlab qr() function, and always computes numerically stable results in terms of column-wise relative error.

In noncommutative Galois theory, the problem arises2 as to whether the ring of all linear transformations on a vector space over a division ring is generated by its units (=the nonsingular linear transformations=the automorphisms of the space). Equivalently, is every singular linear transformation a sum of nonsingular ones? The answer is trivially yes when the space is finite-dimensional and the division ring is an infinite (commutative) field, for then it suffices to choose, for each linear transformation a, a nonzero scalar a not a characteristic root of a, and write3 a =at+(a-a). The proof is not much more difficult in the general finite-dimensional case (cf. Case I in the proof of the theorem below). In this note we show that in the absence of restrictions on the vector space, the answer is still yes and in fact that every linear transformation is still a sum of two nonsingular ones (with the trivial exception of the identity transformation on a space of two elements). We use the following notations: ,N for O is an isomorphism of the vector space M onto the vector space All isomorphisms will be isomorphisms onto. Mappings will be written on the right: xa for the image of x under the mapping a; and hence af3 for the mapping obtained by applying first a and then d. M=L OK will have its usual significance: M is the sum of its complementary subspaces L and K. If a and 3 are linear mappings of L and K into a space N, a GO will denote the unique linear mapping of M into N which induces a on L and 3 on K. Note that if a and 3 are isomorphisms onto complementary subspaces of N, then a e3 is an isomorphism of M onto N. Notation for direct sum of a collection of subspaces or mappings: EMi, Eai.

This paper presents a deep-pipelined FPGA implementation of real-time ellipse estimation for eye tracking. The system is constructed by the Starburst algorithm on a stream-oriented architecture and the RANSAC algorithm without any external memories. In particular, the paper presents comparative results between three different hypothesis generators for the RANSAC algorithm based on Cramer's rule, Gauss-Jordan elimination and LU decomposition. Comparison criteria include resource usage, throughput and energy consumption. The result shows that the three implementations have different characteristics and the optimal algorithm needs to be chosen depending on the amount of resources on FPGAs and required performance.

Similarity of signal processing effect between Hankel matrix-based SVD and wavelet transform and its mechanism analysis

The article investigates the use of linear algebra methods for query optimisation in modern database management systems. The relevance of the study is due to the growth of data volumes and the need for more efficient methods of their processing. The proposed approach is to formalise tabular data structures as numerical matrices, which allows us to interpret relational operations (selection, projection, join) as linear algebraic transformations, which opens up opportunities for the use of powerful mathematical tools that optimise computations. The main methods of linear algebra for query optimisation are considered: matrix factorisation (LU, QR decomposition) for structuring complex calculations; singular value decomposition (SVD) for reducing dimensionality and identifying hidden dependencies; matrix modelling of JOIN operations as an efficient block multiplication; systems of linear equations for formalising data filtering. It is noted that matrix factorisation helps to optimise the execution of complex analytical queries, SVD is effective in aggregation and dimensionality reduction tasks, the matrix representation of JOIN provides high performance when processing large amounts of data, and linear systems unify the filtering logic with numerical processing. To empirically verify the proposed methods, an experimental study was conducted on a synthetic dataset simulating a transactional table with a million records. The experiment was carried out using the NumPy and PyTorch libraries with the use of GPU acceleration. We evaluated such indicators as query execution time, RAM usage, and scalability. The results showed that JOIN matrix modelling was the fastest, which is especially important for parallel computing. LU/QR factorisation demonstrated stability and versatility, while SVD proved to be resource-intensive but valuable for data compression. Linear equation methods showed balanced performance. It is concluded that linear algebra is a powerful formalisation framework for next-generation DBMS architectures capable of providing mathematical consistency, computational efficiency and scalability.

We propose a way to unify the construction of Newton’s methods for matrix decompositions. This construction is based on a perturbation analysis of a suitable system associated with matrix decompositions. Then it appears that the resolution of a linear Sylvester equation permits defining Newton’s method. We give a general result to analyze the quadratic convergence of this Newton’s method. We apply it to classical matrix decompositions: LU decomposition, QR decomposition, eigenproblem in the diagonalizable case, singular value decomposition, and Schur decomposition. Finally, we propose for future work a generalization of this construction to approximate matrix decomposition by high-order methods.

The equivalence of semantic networks with spreading activation and vector spaces with dot product is investigated under ranked retrieval. Semantic networks are viewed as networks of concepts organized in terms of abstraction and packaging relations. It is shown that the two models can be effectively constructed from each other. A formal method is suggested to analyze the models in terms of their relative performance in the same universe of objects.

In this chapter we consider ways of ‘multiplying’ two vectors in a vector space V to give a scalar. Such a product is called an inner product, or scalar product, and the theory is based initially on a familiar example (sometimes called the dot product) on three-dimensional vectors in ℝ3. To begin with, we consider real vector spaces, but we will consider complex vector spaces later on. The scalar product, inner product, or dot product, of two vectors in ℝ2 or ℝ3 is rather well known. We start this chapter with its definition and description of its main properties. Throughout this section, llvll will denote the length of the vector v in ℝ2 or ℝ3

Chapter 9 - Numerical Techniques