Implementation of a distributed parallel in time scheme using PETSc for a Parabolic Optimal Control Problem

Conjugate Gradient (CG) method is an iterative linear solver which is used by many scientific and engineering applications to solve a linear system of algebraic equations. CG generates a heavy load of computation and therefore it slows the performance of the applications using it. Parallelizing CG is considered as a way to increase its performance. However, CG suffers from communication dependencies among its divisible loads. Most of the studies to parallelize CG concentrate on parallelizing its matrix-vector multiplication. In this paper, we answer the following questions: 1) what are the divisible loads in the CG, and 2) where is communication involved in the parallel CG. To answer 1), we highlight the different divisible data blocks in CG. To answer 2), we introduce a dependency graph among the different data blocks. We conduct experiments on a parallel CG implementation and evaluate communication cost.

The mathematical modeling of numerous real-world applications results in hierarchical optimization problems with two decision makers where at least one of them has to solve an optimal control problem of ordinary or partial differential equations. Such models are referred to as bilevel optimal control problems. Here, we first review some different features of bilevel optimal control including important applications, existence results, solution approaches, and optimality conditions. Afterwards, we focus on a specific problem class where parameters appearing in the objective functional of an optimal control problem of partial differential equations have to be reconstructed. After verifying the existence of solutions, necessary optimality conditions are derived by exploiting the optimal value function of the underlying parametric optimal control problem in the context of a relaxation approach.

Abstract: The conjugate gradient (CG) method is a one of the common approaches for solving unconstrained optimization problems, notably known for its suitability for large scale problems. Many recent studies show that this method is also useful for problems of smaller scale. One of the methods used for improving the performance of CG method is hybrid approach, where a CG method is combined with another method. In this study, the ARM CG method is combined with the SMR CG method and tested under exact line search. The resulting hybrid algorithm is globally convergent under exact line search and shown to perform well numerically in comparison to other tested CG methods.

Fast parallel solvers for fourth-order boundary value problems

With the wider use of vector-processing computers, much attention has come to be paid to the conjugate gradient method to solve a linear system of algebraic equations. However, in the case of finite element methods, it is rather difficult to tune up the computer program of the conjugate gradient method so as to make a big reduction in CPU time, because a coefficient matrix appearing in the finite element analysis is sparse and the nonzero components are irregularly located. In this paper, we present a vectorization technique for the conjugate gradient method by using element matrices only. The loop length attained by the present technique is as long as the number of elements and the rate of acceleration is very high.

Ferroelectrics, due a number of characteristics, behave as hereditary materials with fractal structure. To model mathematically the systems with so-called memory effects one can use the fractional time-derivatives. The pyro-electric properties of ferroelectrics arouse interest in developing the fractional-differential approach to simulating heat conductivity process.The present study deals with development and numerical implementation of fractal heat conductivity model for hereditary materials using the concepts of fractional-differential calculus applied to the simulation of intensive heating processes in ferroelectrics.The paper proposes a mathematical model governed through mixed initial-boundary value problem for partial differential equation containing a fractional time-derivative as well as nonlinear temperature dependence on the heat capacity. To solve the problem the computational algorithm was designed which is based on an analog of the Crank – Nicolson finite difference scheme combining with the Grunwald – Letnikov formula for fractional time-derivative approximation. The approximation of Neumann boundary condition is included into the finite difference problem statement using scheme of fictitious mesh points. The total system of linear algebraic equations is solved by sweep method.The designed application program allows one to perform the computer simulation of heat conductivity process in hereditary materials. The model verification was performed for numerical solving test problem with known analytical solution. The results of computational experiments are demonstrated for the example of estimating heat distribution in a typical ferroelectric crystal of TGS (triglycine sulfate) near the temperature of phase transition. The fractional derivative order was approximately evaluated to be ~0.7 at variation of this parameter. We applied the comparison of fractal model implementation results with experimental data related to the time when the ferroelectric crystal is heated to Curie temperature. These findings demonstrate that one needs to use the modified models at the analysis of the field effects arising in hereditary materials.

The left-preconditioned communication avoiding conjugate gradient (LP-CA-CG) method is applied to the pressure Poisson equation in the multiphase CFD code JUPITER. The arithmetic intensity of the LP-CA-CG method is analyzed, and is dramatically improved by loop splitting for inner product operations and for three term recurrence operations. Two LPCA-CG solvers with block Jacobi preconditioning and with underlap preconditioning are developed. The former is developed based on a hybrid CA approach, in which CA is applied only to global collective communications for inner product operations. The latter is a full CA approach, in which CA is applied also to local point-to-point communications in sparse matrix-vector (SpMV) operations and preconditioning. CA-SpMV requires additional computation for overlapping regions. CA-preconditiong is enabled by underlap preconditioning, which approximates preconditioning for overlapping regions by point Jacobi preconditioning. It is shown that on the K computer, the former is faster, because the performance of local point-to-point communications scales well, and the convergence property becomes worse with underlap preconditioning. The LP-CA-CG solver shows good strong scaling up to 30,000 nodes, where the LP-CA-CG solver achieved higher performance than the original CG solver by reducing the cost of global collective communications by 69 percent.

Self-scaled conjugate gradient training algorithms

PurposeThe purpose of this paper is to apply the conjugate gradient (CG) method, together with the adjoint operator (AO) to the pulsating heat pipe problem, including some quite interesting experimental results. The CG method, together with the AO, was able to estimate the unknown functions more efficiently than the other techniques presented in this paper. The estimation of local heat transfer coefficients, rather than the global ones, in pulsating heat pipes is a relatively new subject and presenting a robust, efficient and self-regularized inverse tool to estimate it, supported also by some experimental results, is the main purpose of this paper. To also increase the visibility and the general use of the paper to the heat transfer community, the authors include, as supplemental material, all numerical and experimental data used in this paper.Design/methodology/approachThe approach was established on the solution of the inverse heat conduction problem in the wall by using as starting data the temperature measurements on the outer surface. The procedure is based on the CG method with AO. The here proposed approach was first verified adopting synthetic data and then it was validated with real cases regarding pulsating heat pipes.FindingsAn original fast methodology to estimate local convective heat flux is proposed. The procedure has been validated both numerically and experimentally. The procedure has been compared to other classical methods presenting some peculiar benefits.Practical implicationsThe approach is suitable for pulsating heat pipes performance evaluation because these devices present a local heat flux distribution characterized by an important variation both in time and in space as a result of the complex flow patterns that are generated in this type of devices.Originality/valueThe procedure here proposed shows these benefits: it affords a general model of the heat conduction problem that is effortlessly customized for the particular case, it can be applied also to large datasets and it presents reduced computational expense.

At present, the conjugate gradient (CG) method of Hager and Zhang (Hager and Zhang, SIAM Journal on Optimization, 16(2005)) is regarded as one of the most effective CG methods for optimization problems. In order to further study the CG method, we develop the Hager and Zhang's CG method and present two modified CG formulas, where the given formulas possess the value information of not only the gradient but also the function. Moreover, the sufficient descent condition will be holden without any line search. The global convergence is established for nonconvex function under suitable conditions. Numerical results show that the proposed methods are competitive to the normal conjugate gradient method.

Presently, a preferred minimization for strong‐constraint four‐dimensional variational (4D‐Var) assimilation uses a Lanczos‐based conjugate gradient (CG) algorithm. This requires the availability of a square‐root of the background‐error covariance matrix (B). In the context of weak‐constraint 4D‐Var, this requirement might be too restrictive for the formulations of the model error term. It might therefore be desirable to avoid a square‐root decomposition of the augmented background term. An appealing minimization scheme is the double CG minimization employed, for example, in the grid‐point statistical interpolation (GSI) analysis. Realizing the double CG algorithm is a special case of the more general bi‐conjugate gradient (BiCG) method for solving non‐symmetric problems, the present work introduces a Lanczos‐based preconditioning strategy when B, instead of its square‐root, is used initially. Implementation of the scheme is done in the context of the GSI analysis system, and preliminary experiments are presented using its 3D‐Var version. Comparison of the Lanczos‐based CG and the BiCG shows that the algorithms converge at the same rate and to the same solution. Despite the additional computational cost, the importance of the re‐orthogonalization step is also shown to be fundamental to any of these CG algorithms. Furthermore, when using the Hessian eigenvectors for preconditioning, the BiCG behaviour is shown to be comparable to that of the Lanczos‐CG algorithm. Both schemes construct the same approximation of the Hessian with the same number of eigenvectors, and benefit in the same way from the reduction of the condition number. The efficiency, computational cost, and stability of the three algorithms are discussed. Copyright © 2012 Royal Meteorological Society

We consider the bilinear optimal control of an advection-reaction-diffusion system, where the control arises as the velocity field in the advection term. Such a problem is generally challenging from both theoretical analysis and algorithmic design perspectives, mainly because the state variable depends nonlinearly on the control variable and, an additional divergence-free constraint on the control is coupled together with the state equation. Mathematically, the proof of the existence of optimal solutions is delicate, and, up to now, only some results have been known for a few special cases where additional restrictions are imposed on the space dimension and the regularity of the control. We prove the existence of optimal controls and derive the first-order optimality conditions in general settings without any extra assumptions. Computationally, the well-known conjugate gradient (CG) method can be applied conceptually. However, due to the additional divergence-free constraint on the control variable and the nonlinear relation between the state and control variables, it is challenging to compute the gradient and the optimal stepsize at each CG iteration, and thus nontrivial to implement the CG method. To address these issues, we advocate a fast inner preconditioned CG method to ensure the divergence-free constraint and an efficient inexactness strategy to determine an appropriate stepsize. An easily implementable nested CG method is thus proposed for solving such a complicated problem. For the numerical discretization, we combine finite difference methods for the time discretization and finite element methods for the space discretization. Efficiency of the proposed nested CG method is promisingly validated by the results of some preliminary numerical experiments.

We survey preconditioned iterative methods with the emphasis on solving large sparse systems such as arise by discretization of boundary value problems for partial differential equations. We discuss shortly various acceleration methods but the main emphasis is on efficient preconditioning techniques. Numerical simulations on practical problems have indicated that an efficient preconditioner is the most important part of an iterative algorithm. We report in particular on the state of the art of preconditioning methods for vectorizable and/or parallel computers.

Solving systems of linear algebraic equations (SLAE) is a problem often encountered in fields like engineering, physics, computer science and economics. As the number of unknowns in the linear system grows, the runtime and the memory requirement of solving SLAE increases dramatically. To manage this, the execution of the solver should be parallelizable and be performed in distributed environments like cloud. However, to fully take the advantage of cloud infrastructure, one should adapt the SLAE to frameworks that can successfully exploit the cloud resources like the MapReduce framework, which provides automatic parallelism, scalability and fault tolerance. With this goal, in our previous work we have adapted a SLAE algorithm Conjugate Gradient (CG) to Hadoop MapReduce framework. However, the relative complexity and the iterative structure of the CG algorithm makes it unsuited for Hadoop, which is designed for embarrassingly parallel data intensive tasks. One of the most widely used types of embarrassingly parallel algorithms are algorithms based on the Monte Carlo method. This paper presents a Monte Carlo based linear system solver that is adapted to the MapReduce model, and compares the resulting parallel efficiency and scalability to the CG implementation. The detailed analysis shows that the algorithm performs better than the Hadoop CG implementation, however loses to Twister, an alternative MapReduce implementation.

Although numerical calculations of heat transfer and fluid flow can provide detailed insights into welding processes and welded materials, these calculations are complex and unsuitable in situations where rapid calculations are needed. A recourse is to train and validate a neural network, using results from a well tested heat and fluid flow model to significantly expedite calculations and ensure that the computed results conform to the basic laws of conservation of mass, momentum and energy. Seven feedforward neural networks were developed for gas metal arc (GMA) fillet welding, one each for predicting penetration, leg length, throat, weld pool length, cooling time between 800°C and 500°C, maximum velocity and peak temperature in the weld pool. Each model considered 22 inputs that included all the welding variables, such as current, voltage, welding speed, wire radius, wire feed rate, arc efficiency, arc radius, power distribution, and material properties such as thermal conductivity, specific heat and temperature coefficient of surface tension. The weights in the neural network models were calculated using the conjugate gradient (CG) method and by a hybrid optimisation scheme involving the CG method and a genetic algorithm (GA). The neural network produced by the hybrid optimisation model produced better results than the networks based on the CG method with various sets of randomised initial weights. The CG method alone was unable to find the best optimal weights for achieving low errors. The hybrid optimisation scheme helped in finding optimal weights through a global search, as evidenced by good agreement between all the outputs from the neural networks and the corresponding results from the heat and fluid flow model.