A GENERALIZED REDUCED GRADIENT ALGORITHM FOR SOLVING LARGE-SCALE DISCRETE-TIME NONLINEAR OPTIMAL CONTROL PROBLEMS

A new recurrent neural-network predictive feedback control structure for a class of uncertain nonlinear dynamic time-delay systems in canonical form is developed and analyzed. The dynamic system has constant input and feedback time delays due to a communications channel. The proposed control structure consists of a linearized subsystem local to the controlled plant and a remote predictive controller located at the master command station. In the local linearized subsystem, a recurrent neural network with on-line weight tuning algorithm is employed to approximate the dynamics of the time-delay-free nonlinear plant. No linearity in the unknown parameters is required. No preliminary off-line weight learning is needed. The remote controller is a modified Smith predictor that provides prediction and maintains the desired tracking performance; an extra robustifying term is needed to guarantee stability. Rigorous stability proofs are given using Lyapunov analysis. The result is an adaptive neural net compensation scheme for unknown nonlinear systems with time delays. A simulation example is provided to demonstrate the effectiveness of the proposed control strategy.

Computer Solution of Large Linear Systems

Direct methods and iterative methods have been used in most analog circuit analysis tools for matrix solution. But neither can efficiently solve the large-scale sparse matrix or ill matrix because of the closely coupled relationship of execution time and matrix size for direct methods and the convergence problem for iterative methods. This paper presents two methods that are a combination of direct solution using complete LU factorization and iterative solution using conjugate gradient square (CGS) or conjugate residual (CR) method. The methods converge faster and more reliably than the LU factorization method and conjugate gradient (CG) method, and can achieve high resolution on a wide variety of circuits. A very efficient preconditioning method is used in our approach to accelerate convergence. Furthermore, we present an effective judgment method for re-factorization of matrix. Several numerical experiments results are included to compare their efficiencies.

In this talk we will present the SUPRENUM multiprocessor system and some implementations of parallelized sparse matrix algorithms. The SUPRENUM multiprocessor system was delivered late in 1989 for the first time. It is the result of a research project where German research institutes, universities and industrial companies worked together to built a 256 processor distributed memory machine. In parallel with the construction of the SUPRENUM a lot of time and man power was invested for the software support of the project. As an important application in scientific computation we parallelized the solution of systems of linear equations Ax=b. For realistic problems the large coefficient matrix A is sparse most of the time, i.e. a large number of its entries are zero. We show how direct algorithms based on Gauss Elimination and semi-iterative algorithms (Conjugate Gradient Methods) can be implemented on SUPRENUM. Especially the Conjugate Gradient Methods which are very well suited for parallelization and vectorization proved to be very efficient on multiprocessor architectures.

Frequency-domain elastic wave modeling relies on an efficient linear solver for the large, sparse, ill-conditioned linear system derived from the discretization of the elastic wave equation. Direct solvers are mostly based on LU decomposition. These methods are efficient for multiple right-hand sides problems, but they require significant memory resources. Conversely, iterative solvers benefit from the sparsity of the system, but they require sophisticated preconditioners to converge due to the system ill-conditioning. In this study, we investigate the performance of an iterative method named CARP-CG for frequency-domain elastic wave modeling. The CARP-CG method transforms the original system into a symmetric positive semi-definite system by cyclic row-projections. This system is efficiently solved with the conjugate gradient (CG) method. The cyclic row-projection transformation can be seen as a purely algebraic preconditioning technique which is easy to implement. The algorithm can be parallelized through a row-block decomposition combined with component-averaging operations. Numerical experiments on the 2D frequency-domain elastic problem with high Poisson's ratio exhibit a good scalability of CARP-CG. Comparisons for different frequencies between CARP-CG and standard Krylov iterative solvers (GMRES and CG on the normal equations) emphasize the robustness and the fast convergence of the method.

In the previous installment, we looked at some generalpurpose linear solvers—some approaches to solving systems of simultaneous linear algebraic equations for cases in which there are no distinctive features offered by the coefficient matrix used in the description of the system. We concentrated on Gaussian elimination and LU decomposition. The computational effort required to solve an n-square system via either of those techniques is O(n3), although, as was pointed out, solution via LU decomposition requires only O(n2) arithmetic operations after the factorization is complete. Thus, solving a system of linear equations for more than one right-hand side (i.e., for more than one forcing vector) enjoys significant computational economy when LU decomposition is employed instead of Gaussian elimination.

Preface. Conventions and Notations. 1 An Introduction To Maple. 1.1 The Commands . 1.2 Programming. 2 Linear Systems of Equations and Matrices. 2.1 Linear Systems of Equations. 2.2 Augmented Matrix of a Linear System and Row Operations. 2.3 Some Matrix Arithmetic. 3 Gauss-Jordan Elimination and Reduced Row Echelon Form. 3.1 Gauss-Jordan Elimination and rref. 3.2 Elementary Matrices. 3.3 Sensitivity of Solutions to Error in the Linear System. 4 Applications of Linear Systems and Matrices. 4.1 Applications of Linear Systems to Geometry. 4.2 Applications of Linear Systems to Curve Fitting. 4.3 Applications of Linear Systems to Economics. 4.4 Applications of Matrix Multiplication to Geometry. 4.5 An Application of Matrix Multiplication to Economics. 5 Determinants, Inverses and Cramer s Rule. 5.1 Determinants and Inverses from the Adjoint Formula. 5.2 Determinants by Expanding Along Any Row or Column . 5.3 Determinants Found by Triangularizing Matrices. 5.4 LU Factorization. 5.5 Inverses from rref. 5.6 Cramer s Rule. 6 Basic Linear Algebra Topics. 6.1 Vectors. 6.2 Dot Product. 6.3 Cross Product. 6.4 Vector Projection. 7 A Few Advanced Linear Algebra Topics. 7.1 Rotations in Space. 7.2 Rolling a Circle Along a Curve. 7.3 The TNB Frame. 8 Independence, Basis and Dimension for Subspaces of Rn. 8.1 Subspaces of Rn. 8.2 Independent and Dependent Sets of Vectors in Rn. 8.3 Basis and Dimension for Subspaces of Rn. 8.4 Vector Projection onto a Subspace of Rn. 8.5 The Gram-Schmidt Orthonormalization Process. 9 Linear Maps from Rn to Rm. 9.1 Basics About Linear Maps. 9.2 The Kernel and Image Subspaces of a Linear Map. 9.3 Composites of Two Linear maps and Inverses. 9.4 Change of Bases for the Matrix Representation of a Linear Map. 10 The Geometry of Linear and Affine Maps. 10.1 The Effect of a Linear Map on Area and Arclength in Two Dimensions. 10.2 The Decomposition of Linear Maps into Rotations, Reflections and Rescalings in R2. 10.3 The Effect of Linear Maps on Volume, Area and Arclength in R3. 10.4 Rotations, Reflections and Rescalings in Three Dimensions. 10.5 Affine Maps. 11 Least Squares Fits and Pseudoinverses. 11.1 Pseudoinverse to a Non-Square Matrix and Almost Solving an Overdetermined Linear System. 11.2 Fits and Pseudoinverses. 11.3 Least Squares Fits and Pseudoinverses. 12 Eigenvalues and Eigenvectors. 12.1 What Are Eigenvalues and Eigenvectors, and Why Do We Need Them? 12.2 Summary of Definitions and Methods for Computing Eigenvalues and Eigenvectors as well as the Exponential of a Matrix. 12.3 Applications of the Diagonalizability of Square Matrices. 12.4 Solving a Square First Order Linear. System of Differential Equations ... 12.5 Basic Facts About Eigenvalues and Eigenvectors, and Diagonalizability. 12.6 The Geometry of the Ellipse Using Eigenvalues and Eigenvectors. 12.7 A Maple Eigen-Procedure. Bibliography. Indices. Keyword Index. Index of Maple Commands and Packages.

Direct solvers for dense systems of linear equations commonly use partial pivoting to ensure numerical stability. However, pivoting can introduce significant performance overheads, such as synchronization and data movement, particularly on distributed systems. To improve the performance of these solvers, we present an alternative to pivoting in which numerical stability is obtained through additive updates. We implemented this approach using SLATE, a GPU-accelerated numerical linear algebra library, and evaluated it on the Summit supercomputer. Our approach provides better performance (up to 5-fold speedup) than Gaussian elimination with partial pivoting for comparable accuracy on most of the tested matrices. It also provides better accuracy (up to 15 more digits) than Gaussian elimination with no pivoting for comparable performance.

Stochastic Automata Networks (SANs) are widely used in modeling communication systems, manufacturing systems and computer systems. The SAN approach gives a more compact and efficient representation of the network when compared to the stochastic Petri nets approach. To find the steady state distribution of SANs, it requires solutions of linear systems involving the generator matrices of the SANs. Very often, direct methods such as the LU decomposition are inefficient because of the huge size of the generator matrices. An efficient algorithm should make use of the structure of the matrices. Iterative methods such as the conjugate gradient methods are possible choices. However, their convergence rates are slow in general and preconditioning is required. We note that the MILU and MINV based preconditioners are not appropriate because of their expensive construction cost. In this paper, we consider preconditioners obtained by circulant approximations of SANs. They have low construction cost and can be inverted efficiently. We prove that if only one of the automata is large in size compared to the others, then the preconditioned system of the normal equations will converge very fast. Numerical results for three different SANs solved by CGS are given to illustrate the fast convergence of our method.

SUMMARY Solving the frequency-domain elastic wave equations relies on an efficient linear solver for the large, sparse, indefinite and ill-conditioned linear system derived from the discretization of the elastic wave equation. Direct solvers, which are mostly based on LU decomposition, are efficient for multiple right-hand sides problems, but the memory requirement is huge due to the fill-in effects. On the contrary, iterative solvers fully benefit from the sparsity of the system, but they require problem-specific preconditioners to ensure the convergence because of the ill-conditioning of the system. In this study, we investigate the performance of a robust iterative method named CARP-CG for frequency-domain elastic wave modeling. CARP-CG method turns the original system into a symmetric positive semi-definite system by Kaczmarz row-projections. Such a system can be efficiently solved by the conjugate gradient (CG) method. The row-projections can be seen as a purely algebraic preconditioning technique which is general and is easy to implement. The parallelization is straightforward through a row-block decomposition combined with a component-averaging method. We discretize the 2D frequency-domain elastic wave equation through a 4th order finite difference scheme. Numerical experiments on the Marmousi2 model exhibit a good scalability of CARP-CG. Comparisons between CARP-CG and standard Krylov iterative solvers (GMRES and CGNR) further emphasize the robustness and the fast convergence of CARP-CG method.

A parallel preconditioned conjugate gradient package for solving sparse linear systems on a Cray Y-MP

This paper outlines the parallelisation and vectorisation methods we have used to port a LU decomposition library to the Xeon Phi co-processor. We ported a LU factorisation algorithm, which utilizes the Gaussian elimination method to perform the decomposition, using Intel LEO directives, OpenMP 4.0 directives, Intel's Cilk array notation, and vectorisation directives. We compare the performance achieved with these different methods, investigate the cost of data transfer on the overall time to solution, and analyse the impact of these optimization and parallelisation techniques on code running on the host processors as well. The results show that performance can be improved on the Xeon Phi by optimising the memory operations, and that Cilk array notation can benefit this benchmark on standard processors but do not have the same impact on the Xeon Phi co-processor. We have also demonstrated cases where the Xeon Phi will compute our implementations faster than we can run them on a node of a HPC system, and that our implementations are not as efficient as the LU factorisation implemented in the mkl library.

Power flow is the most fundamental computation in power system analysis. Traditionally, the linear solution in power flow is solved by a direct method like LU decomposition on a CPU platform. However, direct methods may suffer scalability issues in parallel computing when the scale of the system increases. In contrast, iterative solvers, as an alternative to direct solvers, are generally more scalable with better parallelism. This work presents a fast decouple power flow (FDPF) algorithm with a graphic processing unit (GPU)-based preconditioned conjugate gradient (CG) iterative solver. In addition, the inexact Newton method is integrated to further improve the GPU-based parallel computing performance for solving FDPF. The results show that the GPU-based FDPF maintains the same precision and convergence as the original CPU-based FDPF, while providing considerable performance improvement for several large-scale systems. The proposed GPU-based FDPF with inexact Newton method gives a speedup of 2.86 times for a system with over 10,000 buses if compared with traditional FDPF, both implemented based on Matlab. This demonstrates the promising potential of the proposed FDPF computation using a preconditioned iterative solver under GPU architecture.

<p style='text-indent:20px;'>The ensemble method has been developed for accelerating a sequence of numerical simulations of evolution problems. Its main idea is, by manipulating the time stepping and grouping discrete problems, to make all members in the same group share a common coefficient matrix. Thus, at each time step, instead of solving a sequence of linear systems each of which contains only one right-hand-side vector, the ensemble method simultaneously solves a single linear system with multiple right-hand-side vectors for each group. Such a system could be solved efficiently by using direct linear solvers when the problems are of small scale, as the same LU factorization would work for the entire group members. However, for large-scale problems, iterative linear solvers often have to be used and then this appealing advantage becomes not obvious. In this paper we systematically investigate numerical performance of the ensemble method with block iterative solvers for two typical evolution problems: the heat equation and the incompressible Navier-Stokes equations. In particular, the block conjugate gradient (CG) solver is considered for the former and the block generalized minimal residual (GMRES) solver for the latter. Our numerical results demonstrate the effectiveness and efficiency of the ensemble method when working together with these block iterative solvers.

The application of parallel computers to reservoir simulation is studied with the aid of two recently introduced Multiple Instruction, Multiple Data (MIMD) computers. The problems posed by reservoir simulators are shown to be particularly well suited for such parallel computers* Fundamental parallel programming techniques using Fortran are presented for the Hetrogeneous Element Processor (HEP) produced by Denelcor and the iPSC Hypercube produced by Intel. These techniques are then applied to two of the most time consuming tasks in reservoir simulation: forming the matrix coefficients and producing a sparse matrix solution. The problem of parallel formation of sparse matrix coefficients is addressed for the single and multiphase cases using both black oil and compositional fluid models. Parallel sparse matrix solution is addressed by considering D4 ordered Gaussian elimination, and the multigrid, conjugate gradient, and Successive Over-Relaxation (SOR) methods. Three parallel algorithms based on SOR are developed to illustrate the possible diversity of parallel algorithms. The red-black and multicolored SOR methods which are often used on vector machines are shown to be easily adaptable to parallel MIMD machine. Matrix partitioning and a new method of iteration pipelining are also presented as parallel SOR methods. The various SOR algorithms, in parallel form, are mapped onto the HEP to illustrate the speedups currently possible on an MIMD machine. Projections are made on the future importance of these types of machines to reservoir simulation.