Robust additive block triangular preconditioners for block two-by-two linear systems

SummaryThe Chebyshev accelerated preconditioned modified Hermitian and skew‐Hermitian splitting (CAPMHSS) iteration method is presented for solving the linear systems of equations, which have two‐by‐two block coefficient matrices. We derive an iteration error bound to show that the new method is convergent as long as the eigenvalue bounds are not underestimated. Even when the spectral information is lacking, the CAPMHSS iteration method could be considered as an exponentially converging iterative scheme for certain choices of the method parameters. In this case, the convergence rate is independent of the parameters. Besides, the linear subsystems in each iteration can be solved inexactly, which leads to the inexact CAPMHSS iteration method. The iteration error bound of the inexact method is derived also. We discuss in detail the implementation of CAPMHSS for solving two models arising from the Galerkin finite‐element discretizations of distributed control problems and complex symmetric linear systems. The numerical results show the robustness and the efficiency of the new methods.

In this paper, we introduce and analyze an accelerated preconditioning modification of the Hermitian and skew-Hermitian splitting (APMHSS) iteration method for solving a broad class of complex symmetric linear systems. This accelerated PMHSS algorithm involves two iteration parameters ź,β and two preconditioned matrices whose special choices can recover the known PMHSS (preconditioned modification of the Hermitian and skew-Hermitian splitting) iteration method which includes the MHSS method, as well as yield new ones. The convergence theory of this class of APMHSS iteration methods is established under suitable conditions. Each iteration of this method requires the solution of two linear systems with real symmetric positive definite coefficient matrices. Theoretical analyses show that the upper bound ź1(ź,β) of the asymptotic convergence rate of the APMHSS method is smaller than that of the PMHSS iteration method. This implies that the APMHSS method may converge faster than the PMHSS method. Numerical experiments on a few model problems are presented to illustrate the theoretical results and examine the numerical effectiveness of the new method.

The Modified Hermitian and skew-Hermitian splitting (MHSS) iteration method was presented and studied by Bai, Benzi and Chen (Computing, 87(2010), 93-111) for solving a class of complex symmetric linear systems. In this paper, using the properties of Toeplitz matrix, we propose a class of structured MHSS iteration methods for solving the complex Toeplitz linear system. Theoretical analysis shows that the structured MHSS iteration method is unconditionally convergent to the exact solution. When the MHSS iteration method is used directly to complex symmetric Toeplitz linear systems, the computational costs can be considerately reduced by use of Toeplitz structure. Finally, numerical experiments show that the structured MHSS iteration method and the structured MHSS preconditioner are efficient for solving the complex Toeplitz linear system.

A generalization of preconditioned MHSS iteration method for complex symmetric indefinite linear systems

Computer Solution of Large Linear Systems

A new splitting preconditioner for the iterative solution of complex symmetric indefinite linear systems

Two variants of the PMHSS iteration method for a class of complex symmetric indefinite linear systems

In this paper, a lopsided modified Euler-extrapolated Hermitian and skew-Hermitian splitting (LME-HS) iteration method is introduced for solving the complex symmetric linear systems. Under a loose restriction on parameter $\\theta$, we demonstrate that LME-HS iteration method is convergent. Moreover, we present the optimal parameter ${\\theta}^{*}$ of the LME-HS method and discuss the spectral properties of corresponding preconditioned matrix. Finally, the numerical experiments are used to verify the effectiveness of the proposed method.

This work investigates the iterative solution of complex symmetric linear systems with indefinite matrix term. Based on a technical equivalent reformulation of the original indefinite systems, an efficient stationary splitting iteration method is established by combining with its real and imaginary parts. In the view of the structure, the new method can be viewed as a modification of the combination method of the real and imaginary parts (CRI) in [Wang & Lu, J Comput Appl Math. 2017;325:188–197], which is originally designed for solving complex symmetric linear systems with positive semi-definite coefficient matrices. Despite that it is devoted to more difficult indefinite problems, the new method keeps the merits of unconditional convergence and parameter free implementation of the original method. Moreover, Chebyshev polynomial acceleration is considered to further improve the convergence rate of the new method. Numerical experiments on a set of model problems are tested to illustrate the efficiency of the proposed methods compared with some existing ones.

In this paper, we propose a relaxed block splitting preconditioner for a class of complex symmetric indefinite linear systems to accelerate the convergence rate of the Krylov subspace iteration method and the relaxed preconditioner is much closer to the original block two-by-two coefficient matrix. We study the spectral properties and the eigenvector distributions of the corresponding preconditioned matrix. In addition, the degree of the minimal polynomial of the preconditioned matrix is also derived. Finally, some numerical experiments are presented to illustrate the effectiveness of the relaxed splitting preconditioner.

In this dissertation we develop numerical algorithms and software tools to facilitate parallel solutions of nonlinear programming (NLP) problems. In particular, we address large-scale, block-structured problems with an intrinsic decomposable configuration. These problems arise in a great number of engineering applications, including parameter estimation, optimal control, network optimization, and stochastic programming. The structure of these problems can be leveraged by optimization solvers to accelerate solutions and overcome memory limitations, and we propose variants to two classes of optimization algorithms: augmented Lagrangian (AL) schemes and Schur-complement interior-point methods. The convergence properties of augmented Lagrangian decomposition schemes like the alternating direction method of multipliers (ADMM) and progressive hedging (PH) are well established for convex optimization but convergence guarantees in non-convex settings are still poorly understood. In practice, however, ADMM and PH often perform satisfactorily in complex non-convex NLPs. In this work, we study connections between the method of multipliers (MM), ADMM, and PH to derive benchmarking metrics that explain why PH and ADMM work in practice. We illustrate the concepts using challenging dynamic optimization problems. Our exposition seeks to establish more formalism in benchmarking ADMM, PH, and AL schemes and to motivate algorithmic improvements. The effectiveness of nonlinear interior-point solvers for solving large-scale problems relies quite heavily on the solution of the underlying linear algebra systems. The schur-complement decomposition is very effective for parallelizing the solution of linear systems with modest coupling. However, for systems with large number of coupling variables the schur-complement method does not scale favorably. We implement an approach that uses a Krylov solver (GMRES) preconditioned with ADMM to solve block-structured linear systems that arise in the interior-point method. We show that this ADMM-GMRES approach overcomes the well-known scalability issues of Schur decomposition. One important drawback of using decomposition approaches like ADMM and PH is their convergence rate. Unlike Schur-complement interior-point algorithms that have super-linear convergence, augmented Lagrangian approaches typically exhibit linear and sublinear rates. We exploit connections between ADMM and the Schur-complement decomposition to derive an accelerated version of ADMM. Specifically, we study the effectiveness of performing a Newton-Raphson algorithm to compute multiplier estimates for augmented Lagrangian methods. We demonstrate using two-stage stochastic programming problems that our multiplier update achieves convergence in fewer iterations for MM on general nonlinear problems. In the case of ADMM, the newton update significantly reduces the number of subproblem solves for convex quadratic programs (QPs). Moreover, we show that using newton multiplier updates makes the method robust to the selection of the penalty parameter. Traditionally, state-of-the-art optimization solvers are implemented in low-level programming languages. In our experience, the development of decomposition algorithms in these frameworks is challenging. They present a steep learning curve and can slow the development and testing of new numerical algorithms. To mitigate these challenges, we developed PyNumero, a new open source framework implemented in Python and C++. The package seeks to facilitate development of optimization algorithms for large-scale optimization within a high-level programming environment while at the same time minimizing the computational burden of using Python. The efficiency of PyNumero is illustrated by implementing algorithms for problems arising in stochastic programming and optimal control. Timing results are presented for both serial and parallel implementations. Our computational studies demonstrate that with the appropriate balance between compiled code and Python, efficient implementations of optimization algorithms are achievable in these high-level languages.

The conjugate orthogonal conjugate gradient (COCG) method has been considered an attractive part of the Lanczos-type Krylov subspace method for solving complex symmetric linear systems. However, it is often faced with apparently irregular convergence behaviors in practical electromagnetic simulations. To avoid such a problem, the symmetric quasi-minimal residual (QMR) method has been developed. On the other hand, the conjugate A-orthogonal conjugate residual (COCR) method, which can be regarded as an extension of the conjugate residual method, also had been established. It shows that the COCR often gives smoother convergence behavior than the COCG method. The purpose of this paper is to apply the QMR approaches to the COCG and COCR to derive two new methods (including their preconditioned versions), and to report the benefits of the modified methods by some practical examples arising in electromagnetic simulations.

The solution of systems of linear equations Ax=b with complex symmetric coefficient matrix A of size N, typically appearing in quantum-reactive scattering problems, is discussed. The quasiminimal residual (QMR) method is introduced to solve the complex symmetric linear system and is compared to the generalized minimal residual (GMRES) method. The methods are applied to two different chemical problems: the initial state-selected reaction probability for the H2+OH→H +H2O reaction, and the cumulative reaction probability for the isomerization of ketene, both with N≳104. It is shown that the QMR method behaves more favorably, i.e., converges faster, than the GMRES for large N, especially when high accuracy is needed.

New condition numbers for matrices and linear systems are proposed, based on the dependence of the relative errors in the result upon the relative errors of the data.

In this paper we describe, analyse and implement a parallel iterative method for the solution of the steady-state drift diffusion equations governing the behaviour of a semiconductor device in two space dimensions. The unknowns in our model are the electrostatic potential and the electron and hole quasi-Fermi potentials. Our discretisation consists of a finite element method with mass lumping for the electrostatic potential equation and a hybrid finite element with local current conservation properties for the continuity equations. A version of Gummel's decoupling algorithm which only requires the solution of positive definite symmetric linear systems is used to solve the resulting nonlinear equations. We show that this method has an overall rate of convergence which only degrades logarithmically as the mesh is refined. Indeed the (inner) nonlinear solves of the electrostatic potential equation converge quadratically, with a mesh independent asymptotic constant. We also describe an implementation on a MasPar MP-1 data parallel machine, where the required linear systems are solved by the preconditioned conjugate gradient method. Domain decomposition methods are used to parallelise the required matrix-vector multiplications and to build preconditioners for these very poorly-conditioned systems. Our preconditioned linear solves also have a rate of convergence which degrades logarithmically as the grid is refined relative to subdomain size, and their performance is resilient to the severe layers which arise in the coefficients of the underlying elliptic operators. Parallel experiments are given.