Predict-and-Recompute Conjugate Gradient Variants

Hiding global synchronization latency in the preconditioned Conjugate Gradient algorithm

A new computational technique for synthesizing the optimal control policy for problems having singular arcs is developed. Through a transformation of the control variable, a new, non-singular gradient method is developed which shows improved convergence properties over the standard gradient and conjugate gradient methods. A number of examples are worked to illustrate the convergence properties of the algorithm.

An approximate inverse preconditioner and its implementation for conjugate gradient method

Metaheuristic algorithms are used to solve many optimization problems. Firefly algorithm, particle swarm improvement, harmonic search, and bat algorithm are used as search algorithms to find the optimal solution to the problem field. In this paper, we have investigated and analyzed a new scaled conjugate gradient algorithm and its implementation, based on the exact Wolfe line search conditions and the restart Powell criterion. The new spectral conjugate gradient algorithm is a modification of the Birgin and Martínez method, a manner to overcome the lack of positive definiteness of the matrix defining the search direction. The preliminary computational results for a set of 30 unconstrained optimization test problems show that this new spectral conjugate gradient outperforms a standard conjugate gradient in this field and we have applied the newly proposed spectral conjugate gradient algorithm in bat algorithm to reach the lowest possible goal of bat algorithm. The newly proposed approach, namely, the directional bat algorithm (CG-BAT), has been then tested using several standard and nonstandard benchmarks from the CEC’2005 benchmark suite with five other algorithms and has been then tested using nonparametric statistical tests and the statistical test results show the superiority of the directional bat algorithm, and also we have adopted the performance profiles given by Dolan and More which show the superiority of the new algorithm (CG-BAT).

A nonlinear reconstruction scheme based on the Distorted Born Iterative Method (DBIM) is presented to solve two-dimensional inverse scattering problems with metallic scatterers. Half-quadratic regularization alleviates the inherently ill-posed problem of nonlinear inverse scattering while simultaneously preserving edges. Reconstruction results are found by minimizing a cost function. A bistatic experimental set-up is studied with angular and frequency diversity. A Multilevel Fast Multipole Algorithm (MLFMA) combined with a conjugate gradient (CG) scheme is introduced to solve the forward as well as the inverse scattering problem involved in the DBIM. The computational complexity per CG iteration using the MLFMA is of order O(N log N) compared to O(N2) in a standard implementation without the MLFMA. Numerical reconstruction results obtained from synthetic scattering data are presented.

Multimillion-atom molecular dynamics (MD) simulations have been performed to study the flat InAs overlayers with self-limiting thickness on GaAs square nanomesas. The in-plane lattice constant of InAs layers parallel to the InAs/GaAs(001) interface starts to exceed the InAs bulk value at 12th monolayer (ML) and the hydrostatic stresses in InAs layers become tensile above ∼12 th ML. As a result, it is not favorable to have InAs overlayers thicker than 12 ML. This may explain the experimental findings of the growth of flat InAs overlayers with self-limiting thickness of ∼11 ML on GaAs nanomesas. We have also examined the lateral size effects on the stress distribution and morphology of InAs/GaAs square nanomesas using parallel molecular dynamics. Two mesas with the same vertical size but different lateral sizes are simulated. For the smaller mesa, a single stress domain is observed in the InAs overlayer, whereas two stress domains are found in the larger mesa. This indicates the existence of a critical lateral size for domain formation in accordance with recent experimental findings. The InAs overlayer in the larger mesa is laterally constrained to the GaAs bulk lattice constant but vertically relaxed to the InAs bulk lattice constant, consistent with the Poisson effect. Moreover, we have calculated surface energies of GaAs and InAs for the (100), (110), and (111) orientations. Both MD and the conjugate gradient method are used and the results are in excellent agreement. Surface reconstructions on GaAs(100) and InAs(100) are studied via the conjugate gradient method. We have developed a new model for GaAs(100) and InAs(100) surface atoms. Not only this model reproduces well surface energies for the (100) orientation, it also yields (1 x 2) dimer lengths in accordance with Ab initio calculations. Finally, a series of molecular dynamics simulations are performed to investigate the behavior under load of several 〈001〉 and 〈011〉 symmetrical tilt grain boundaries (GBs) in diamond. These MD simulations are based on the bond-order analytic potential. Crack propagation in polycrystalline diamond samples under an applied load is simulated, and found to be predominantly transgranular rather than intergranular.

Use of preconditioned bi-conjugate gradient method in modeling two-phase fluid flow in porous media

A new low-memory modification of the density fitting approximation based on a combination of a continuous fast multipole method (CFMM) and a preconditioned conjugate gradient solver is presented. Iterative conjugate gradient solver uses preconditioners formed from blocks of the Coulomb metric matrix that decrease the number of iterations needed for convergence by up to one order of magnitude. The matrix-vector products needed within the iterative algorithm are calculated using CFMM, which evaluates them with the linear scaling memory requirements only. Compared with the standard density fitting implementation, up to 15-fold reduction of the memory requirements is achieved for the most efficient preconditioner at a cost of only 25% increase in computational time. The potential of the method is demonstrated by performing density functional theory calculations for zeolite fragment with 2592 atoms and 121,248 auxiliary basis functions on a single 12-core CPU workstation.

We aimed to develop a conjugate gradient method by reformulating parameters so that iterative optimization techniques perform more effectively. Instead of using a set of basic conjugate gradient formulas, the methodology introduces a new parameter that produces better convergence. The method is applied in Fortran to test how many iterations and how many evaluations of the function are needed, as listed in table 1. The behavior of convergence and the results used for comparison are created with Matplotlib on Python and Ggplot2 on R programming for the chart. We like to compare our method to the proven LS (Liu-Storey) method when checking how effective our proposed method is. We found that the technique provides better results with lower iteration numbers and better convergence speeds for many test cases, proving it can challenge conventional methods in optimizing problems without constraints.

Conjugate Gradient (CG) method is an iterative linear solver which is used by many scientific and engineering applications to solve a linear system of algebraic equations. CG generates a heavy load of computation and therefore it slows the performance of the applications using it. Parallelizing CG is considered as a way to increase its performance. However, CG suffers from communication dependencies among its divisible loads. Most of the studies to parallelize CG concentrate on parallelizing its matrix-vector multiplication. In this paper, we answer the following questions: 1) what are the divisible loads in the CG, and 2) where is communication involved in the parallel CG. To answer 1), we highlight the different divisible data blocks in CG. To answer 2), we introduce a dependency graph among the different data blocks. We conduct experiments on a parallel CG implementation and evaluate communication cost.

In order to solve large mathematical formulations of real-life electromagnetic problems, we must use advances in both solution algorithms and computer hardware. The fast multipole method (FMM) and its multilevel version, the multilevel fast multipole algorithm (MLFMA), are two of the preferred choices for the algorithm due to their reduced computational complexities and memory requirements. A parallel architecture is preferred for the hardware due to its increased computing power, with a consequent parallel implementation of the MLFMA. Some of the high-level choices that need to be made to implement the MLFMA are as follows: integral-equation (IE) formulation - EFIE, MFIE, or combined-field IE (CFIE); iterative solver - Krylov-subspace methods such as different conjugate gradient methods or generalized minimal residual; preconditioner - near-field (NF), filtered NF, block-diagonal or diagonal preconditioners, or no preconditioner; initial guess. These parameters are extensively investigated. For this purpose, a series of scattering problems of various sizes (at different frequencies) containing different numbers of unknowns are used as a testbed.

Parallel preconditioned conjugate gradient algorithm on GPU

This paper presents the fusion of artificial intelligence (AI) learning algorithms that are genetic algorithms (GA) and conjugate gradient (CG) methods. Both methods are used to find the optimum weights for the hidden and output layers of feedforward artificial neural network (ANN) model. Each algorithm is presented in separate module and we proposed three different types of Dynamic Connection Strategies (DyConS) for combining both algorithms to improve the recognition performance of spoken Malay speech recognition. Two different GA techniques are used in this research: a mutated GA technique is proposed and compared with the standard GA technique. One hundred experiments with 5000 words are conducted using the proposed DyConS. Owing to previous facts, GA combined with ANN proved to attain certain advantages with sufficient recognition performance. Thus, from the results, it was observed that the performance of mutated GA algorithm when combined with CG is better than standard GA and CG models. Integrating the GA with feed-forward network improved mean square error (MSE) performance and with good connection strategy by this two stage training scheme, the recognition rate is increased up to 99%.

Fully coupled Newton's method is combined with conjugate gradient-like iterative algorithms to form inexact Newton-Krylov algorithms for solving the steady, incompressible, Navier-Stokes and energy equations in primitive variables. Finite volume differencing is employed using the power law convection-diffusion scheme on a uniform but staggered mesh. The well-known model problem of natural correction in an enclosed cavity is solved. Three conjugate gradient-like algorithms are selected from a clans of algorithms based upon the Lanczos biorthogonalization procedure; namely, the conjugate gradients squared algorithm, the transpose-free quasi-minimal-residual algorithm, and a more smoothly convergent version of the biconjugate gradients algorithm. A fourth algorithm is based upon the Arnoldi process, namely the popular generalized minimal residual algorithm (GMRES). The performance of a standard inexact Newton's method implementation is compared with a matrix-free implementation. Results indicate that the performance of the matrix-free implementation is strongly dependent upon grid size (number of unknowns) and the selection of the conjugate gradient-like method. GMRES appeared to be superior to the Lanczos based algorithms within the context of a matrix-free implementation

Solving systems of linear algebraic equations (SLAE) is a problem often encountered in fields like engineering, physics, computer science and economics. As the number of unknowns in the linear system grows, the runtime and the memory requirement of solving SLAE increases dramatically. To manage this, the execution of the solver should be parallelizable and be performed in distributed environments like cloud. However, to fully take the advantage of cloud infrastructure, one should adapt the SLAE to frameworks that can successfully exploit the cloud resources like the MapReduce framework, which provides automatic parallelism, scalability and fault tolerance. With this goal, in our previous work we have adapted a SLAE algorithm Conjugate Gradient (CG) to Hadoop MapReduce framework. However, the relative complexity and the iterative structure of the CG algorithm makes it unsuited for Hadoop, which is designed for embarrassingly parallel data intensive tasks. One of the most widely used types of embarrassingly parallel algorithms are algorithms based on the Monte Carlo method. This paper presents a Monte Carlo based linear system solver that is adapted to the MapReduce model, and compares the resulting parallel efficiency and scalability to the CG implementation. The detailed analysis shows that the algorithm performs better than the Hadoop CG implementation, however loses to Twister, an alternative MapReduce implementation.