Hiding global synchronization latency in the preconditioned Conjugate Gradient algorithm

Direct methods and iterative methods have been used in most analog circuit analysis tools for matrix solution. But neither can efficiently solve the large-scale sparse matrix or ill matrix because of the closely coupled relationship of execution time and matrix size for direct methods and the convergence problem for iterative methods. This paper presents two methods that are a combination of direct solution using complete LU factorization and iterative solution using conjugate gradient square (CGS) or conjugate residual (CR) method. The methods converge faster and more reliably than the LU factorization method and conjugate gradient (CG) method, and can achieve high resolution on a wide variety of circuits. A very efficient preconditioning method is used in our approach to accelerate convergence. Furthermore, we present an effective judgment method for re-factorization of matrix. Several numerical experiments results are included to compare their efficiencies.

In this report we present parallel solvers for large linear systems arising from the finite-element discretization of the three-dimensional steady-state groundwater flow problem. Our solvers are based on multigrid and Krylov subspace methods. The parallel implementation is based on a domain decomposition strategy with explicit message passing using NX and MPI libraries. We have tested our parallel implementations on the Intel Paragon XP/S 150 supercomputer using up to 1024 parallel processors and on other parallel platforms such as SGI/Power Challenge Array, Cray/SGI Origin 2000, Convex Exemplar SPP-1200, and IBM SP using up to 64 processors. We show that multigrid can be a scalable algorithm on distributed memory machines. We demonstrate the effectiveness of parallel multigrid based solvers by solving problems requiring more than 70 million nodes in less than a minute. This is more than 25 times faster than the diagonal preconditioned conjugate gradient method which is one of the more popular methods for large sparse linear systems. Our results also show that multigrid as a stand alone solver works best for problems with smooth coefficients, but for rough coefficients it is best used as a preconditioner for a Krylov subspace method such as the conjugate gradient method. We show that even for extremely heterogeneous systems the multigrid pre-conditioned conjugate gradient method is at least 10 times faster than the diagonally preconditioned conjugate gradient method.

In this study, we analyze the real-time solution of a series of augmented least squares problems, which are generated by adding information to an original least squares model repetitively. Instead of solving the least squares problems directly, we transform them into a batch of saddle point linear systems and subsequently solve the linear systems using restrictively preconditioned conjugate gradient (RPCG) methods. Approximation of the new Schur complement is generated effectively based on a previously approximated Schur complement. Owing to the variations of the preconditioned conjugate gradient method, the proposed methods generate convergence results similar to the conjugate gradient method and achieve a very fast convergent iterative sequence when the coefficient matrix is well preconditioned. Numerical tests show that the new methods are more effective than some standard Krylov subspace methods. Updated RPCG methods meet the requirement of real-time computing successfully for multifactor models.

The focus of this study is on the efficient solution for three-dimensional (3D) time-harmonic eddy current problems discretised by the finite-element method (FEM) with thin elements. The systems of equations arising from the finite-element formulations are solved by the geometric multigrid preconditioned conjugate gradient (MGCG) method. Numerical examples show that the MGCG method has stable convergence, which does not deteriorate for elements with high aspect ratios and its computation time is much less than that of the conjugate gradient method with incomplete Cholesky factorisation as the preconditioner (ICCG).

A novel sine transform based preconditioned conjugate gradient (PCG) method for signal restoration is presented. If the bandwidth 2? +1 in the model matrix is a constant, then the asymptotic complexity for the new algorithm is O(N) per iterative step, where Nis the order of the model. This complexity is reduced by an order of magnitude in comparison with the known fast Fourier transform (FFT) technique proposed recently in the literature. Moreover, if ? has a size similar to N, then the new PCG method is accomplished by the fast sine transform (FST) and the fast Hartley transform (FHT). The computational and storage cost per preconditioning step for the new PCG method is reduced by 50% as compared to the FFT approach. To stabilize the computation, a special Tikhonov regularization is introduced. Numerical experimentations show that the new PCG method is of a speedy convergence rate. The new PCG methods maintain less computational complexity per iterative step and have a better convergence rate than the other known PCG methods.

Abstract: The conjugate gradient (CG) method is a one of the common approaches for solving unconstrained optimization problems, notably known for its suitability for large scale problems. Many recent studies show that this method is also useful for problems of smaller scale. One of the methods used for improving the performance of CG method is hybrid approach, where a CG method is combined with another method. In this study, the ARM CG method is combined with the SMR CG method and tested under exact line search. The resulting hybrid algorithm is globally convergent under exact line search and shown to perform well numerically in comparison to other tested CG methods.

Virtual earthquake engineering laboratory with physics-based degrading materials on parallel computers

We consider the bilinear optimal control of an advection-reaction-diffusion system, where the control arises as the velocity field in the advection term. Such a problem is generally challenging from both theoretical analysis and algorithmic design perspectives, mainly because the state variable depends nonlinearly on the control variable and, an additional divergence-free constraint on the control is coupled together with the state equation. Mathematically, the proof of the existence of optimal solutions is delicate, and, up to now, only some results have been known for a few special cases where additional restrictions are imposed on the space dimension and the regularity of the control. We prove the existence of optimal controls and derive the first-order optimality conditions in general settings without any extra assumptions. Computationally, the well-known conjugate gradient (CG) method can be applied conceptually. However, due to the additional divergence-free constraint on the control variable and the nonlinear relation between the state and control variables, it is challenging to compute the gradient and the optimal stepsize at each CG iteration, and thus nontrivial to implement the CG method. To address these issues, we advocate a fast inner preconditioned CG method to ensure the divergence-free constraint and an efficient inexactness strategy to determine an appropriate stepsize. An easily implementable nested CG method is thus proposed for solving such a complicated problem. For the numerical discretization, we combine finite difference methods for the time discretization and finite element methods for the space discretization. Efficiency of the proposed nested CG method is promisingly validated by the results of some preliminary numerical experiments.

The left-preconditioned communication avoiding conjugate gradient (LP-CA-CG) method is applied to the pressure Poisson equation in the multiphase CFD code JUPITER. The arithmetic intensity of the LP-CA-CG method is analyzed, and is dramatically improved by loop splitting for inner product operations and for three term recurrence operations. Two LPCA-CG solvers with block Jacobi preconditioning and with underlap preconditioning are developed. The former is developed based on a hybrid CA approach, in which CA is applied only to global collective communications for inner product operations. The latter is a full CA approach, in which CA is applied also to local point-to-point communications in sparse matrix-vector (SpMV) operations and preconditioning. CA-SpMV requires additional computation for overlapping regions. CA-preconditiong is enabled by underlap preconditioning, which approximates preconditioning for overlapping regions by point Jacobi preconditioning. It is shown that on the K computer, the former is faster, because the performance of local point-to-point communications scales well, and the convergence property becomes worse with underlap preconditioning. The LP-CA-CG solver shows good strong scaling up to 30,000 nodes, where the LP-CA-CG solver achieved higher performance than the original CG solver by reducing the cost of global collective communications by 69 percent.

Self-scaled conjugate gradient training algorithms

The conjugate orthogonal conjugate gradient (COCG) method has been considered an attractive part of the Lanczos-type Krylov subspace method for solving complex symmetric linear systems. However, it is often faced with apparently irregular convergence behaviors in practical electromagnetic simulations. To avoid such a problem, the symmetric quasi-minimal residual (QMR) method has been developed. On the other hand, the conjugate A-orthogonal conjugate residual (COCR) method, which can be regarded as an extension of the conjugate residual method, also had been established. It shows that the COCR often gives smoother convergence behavior than the COCG method. The purpose of this paper is to apply the QMR approaches to the COCG and COCR to derive two new methods (including their preconditioned versions), and to report the benefits of the modified methods by some practical examples arising in electromagnetic simulations.

PurposeThe purpose of this paper is to apply the conjugate gradient (CG) method, together with the adjoint operator (AO) to the pulsating heat pipe problem, including some quite interesting experimental results. The CG method, together with the AO, was able to estimate the unknown functions more efficiently than the other techniques presented in this paper. The estimation of local heat transfer coefficients, rather than the global ones, in pulsating heat pipes is a relatively new subject and presenting a robust, efficient and self-regularized inverse tool to estimate it, supported also by some experimental results, is the main purpose of this paper. To also increase the visibility and the general use of the paper to the heat transfer community, the authors include, as supplemental material, all numerical and experimental data used in this paper.Design/methodology/approachThe approach was established on the solution of the inverse heat conduction problem in the wall by using as starting data the temperature measurements on the outer surface. The procedure is based on the CG method with AO. The here proposed approach was first verified adopting synthetic data and then it was validated with real cases regarding pulsating heat pipes.FindingsAn original fast methodology to estimate local convective heat flux is proposed. The procedure has been validated both numerically and experimentally. The procedure has been compared to other classical methods presenting some peculiar benefits.Practical implicationsThe approach is suitable for pulsating heat pipes performance evaluation because these devices present a local heat flux distribution characterized by an important variation both in time and in space as a result of the complex flow patterns that are generated in this type of devices.Originality/valueThe procedure here proposed shows these benefits: it affords a general model of the heat conduction problem that is effortlessly customized for the particular case, it can be applied also to large datasets and it presents reduced computational expense.

At present, the conjugate gradient (CG) method of Hager and Zhang (Hager and Zhang, SIAM Journal on Optimization, 16(2005)) is regarded as one of the most effective CG methods for optimization problems. In order to further study the CG method, we develop the Hager and Zhang's CG method and present two modified CG formulas, where the given formulas possess the value information of not only the gradient but also the function. Moreover, the sufficient descent condition will be holden without any line search. The global convergence is established for nonconvex function under suitable conditions. Numerical results show that the proposed methods are competitive to the normal conjugate gradient method.

The application of parallel computers to reservoir simulation is studied with the aid of two recently introduced Multiple Instruction, Multiple Data (MIMD) computers. The problems posed by reservoir simulators are shown to be particularly well suited for such parallel computers* Fundamental parallel programming techniques using Fortran are presented for the Hetrogeneous Element Processor (HEP) produced by Denelcor and the iPSC Hypercube produced by Intel. These techniques are then applied to two of the most time consuming tasks in reservoir simulation: forming the matrix coefficients and producing a sparse matrix solution. The problem of parallel formation of sparse matrix coefficients is addressed for the single and multiphase cases using both black oil and compositional fluid models. Parallel sparse matrix solution is addressed by considering D4 ordered Gaussian elimination, and the multigrid, conjugate gradient, and Successive Over-Relaxation (SOR) methods. Three parallel algorithms based on SOR are developed to illustrate the possible diversity of parallel algorithms. The red-black and multicolored SOR methods which are often used on vector machines are shown to be easily adaptable to parallel MIMD machine. Matrix partitioning and a new method of iteration pipelining are also presented as parallel SOR methods. The various SOR algorithms, in parallel form, are mapped onto the HEP to illustrate the speedups currently possible on an MIMD machine. Projections are made on the future importance of these types of machines to reservoir simulation.

Presently, a preferred minimization for strong‐constraint four‐dimensional variational (4D‐Var) assimilation uses a Lanczos‐based conjugate gradient (CG) algorithm. This requires the availability of a square‐root of the background‐error covariance matrix (B). In the context of weak‐constraint 4D‐Var, this requirement might be too restrictive for the formulations of the model error term. It might therefore be desirable to avoid a square‐root decomposition of the augmented background term. An appealing minimization scheme is the double CG minimization employed, for example, in the grid‐point statistical interpolation (GSI) analysis. Realizing the double CG algorithm is a special case of the more general bi‐conjugate gradient (BiCG) method for solving non‐symmetric problems, the present work introduces a Lanczos‐based preconditioning strategy when B, instead of its square‐root, is used initially. Implementation of the scheme is done in the context of the GSI analysis system, and preliminary experiments are presented using its 3D‐Var version. Comparison of the Lanczos‐based CG and the BiCG shows that the algorithms converge at the same rate and to the same solution. Despite the additional computational cost, the importance of the re‐orthogonalization step is also shown to be fundamental to any of these CG algorithms. Furthermore, when using the Hessian eigenvectors for preconditioning, the BiCG behaviour is shown to be comparable to that of the Lanczos‐CG algorithm. Both schemes construct the same approximation of the Hessian with the same number of eigenvectors, and benefit in the same way from the reduction of the condition number. The efficiency, computational cost, and stability of the three algorithms are discussed. Copyright © 2012 Royal Meteorological Society