Convergence analysis of superoptimal PCG algorithm for Toeplitz systems with a Fisher–Hartwig singularity

A novel sine transform based preconditioned conjugate gradient (PCG) method for signal restoration is presented. If the bandwidth 2? +1 in the model matrix is a constant, then the asymptotic complexity for the new algorithm is O(N) per iterative step, where Nis the order of the model. This complexity is reduced by an order of magnitude in comparison with the known fast Fourier transform (FFT) technique proposed recently in the literature. Moreover, if ? has a size similar to N, then the new PCG method is accomplished by the fast sine transform (FST) and the fast Hartley transform (FHT). The computational and storage cost per preconditioning step for the new PCG method is reduced by 50% as compared to the FFT approach. To stabilize the computation, a special Tikhonov regularization is introduced. Numerical experimentations show that the new PCG method is of a speedy convergence rate. The new PCG methods maintain less computational complexity per iterative step and have a better convergence rate than the other known PCG methods.

The generalized superoptimal preconditioner

Atmospheric tomography, i.e., the reconstruction of the turbulence profile in the atmosphere, is a challenging task for adaptive optics (AO) systems for the next generation of extremely large telescopes. Within the AO community, the solver of first choice is the so-called Matrix Vector Multiplication (MVM) method, which directly applies the (regularized) generalized inverse of the system operator to the data. For small telescopes this approach is feasible, however, for larger systems such as the European Extremely Large Telescope (ELT), the atmospheric tomography problem is considerably more complex, and the computational efficiency becomes an issue. Iterative methods such as the Finite Element Wavelet Hybrid Algorithm (FEWHA) are a promising alternative. FEWHA is a wavelet-based reconstructor that uses the well-known iterative preconditioned conjugate gradient (PCG) method as a solver. The number of floating point operations and the memory usage are decreased significantly by using a matrix-free representation of the forward operator. A crucial indicator for the real-time performance are the number of PCG iterations. In this paper, we propose an augmented version of FEWHA, where the number of iterations is decreased by 50% using an augmented Krylov subspace method. We demonstrate that a parallel implementation of augmented FEWHA allows the fulfilment of the real-time requirements of the ELT.

High-resolution global ocean general circulation models (OGCMs) play a key role in accurate ocean forecasting. However, the models of the operational forecasting systems are still not in high resolution due to the subsequent high demand for large computation, as well as the low parallel efficiency barrier. Good scalability is an important index of parallel efficiency and is still a challenge for OGCMs. We found that the communication cost in a barotropic solver, namely, the preconditioned conjugate gradient (PCG) method, is the key bottleneck for scalability due to the high frequency of the global reductions. In this work, we developed a new algorithm—a communication-avoiding Krylov subspace method with a PCG (CA-PCG)—to improve scalability and then applied it to the Nucleus for European Modelling of the Ocean (NEMO) as an example. For PCG, inner product operations with global communication were needed in every iteration, while for CA-PCG, inner product operations were only needed every eight iterations. Therefore, the global communication cost decreased from more than 94.5% of the total execution time with PCG to less than 63.4% with CA-PCG. As a result, the execution time of the barotropic modes decreased from more than 17,000 s with PCG to less than 6000 s with CA-PCG, and the total execution time decreased from more than 18,000 s with PCG to less than 6200 s with CA-PCG. Besides, the ratio of the speedup can also be increased from 3.7 to 4.6. In summary, the high process count scalability when using CA-PCG was effectively improved from that using the PCG method, providing a highly effective solution for accurate ocean simulation.

Circulant preconditioners for pricing options

The Legendre spectral Galerkin method of self-adjoint second order elliptic equations usually results in a linear system with a dense and ill-conditioned coefficient matrix. In this paper, the linear system is solved by a preconditioned conjugate gradient (PCG) method where the preconditioner M is constructed by approximating the variable coefficients with a (T+1)-term Legendre series in each direction to a desired accuracy. A feature of the proposed PCG method is that the iteration step increases slightly with the size of the resulting matrix when reaching a certain approximation accuracy. The efficiency of the method lies in that the system with the preconditioner M is approximately solved by a one-step method based on the incomplete LU factorization technique with no fill-in, denoted by ILU(0). The ILU(0) factorization of \(M\in {\mathbb {R}}^{(N-1)^d\times (N-1)^d}\) can be computed using \({\mathcal {O}}(T^{2d} N^d)\) operations, and the number of nonzeros in the factorization factors is of \({\mathcal {O}}(T^{d} N^d)\), \(d=1,2,3\). A conclusion of the algorithm is to fast solve the resulting system from the Legendre Galerkin spectral method for Poisson equations with Dirichlet boundary conditions, which has a complexity of \({\mathcal {O}}(N^d)\). To further speed up the PCG method, an algorithm is developed for fast matrix-vector multiplications by the resulting matrix of Legendre-Galerkin spectral discretization, without the need to explicitly form it. The complexity of the fast matrix-vector multiplications is of \({\mathcal {O}}(N^d (\log _2 N)^{2})\). In view that T is independent of N in one dimension and is set to be of order \({\mathcal {O}}(\log _2 N)\) in two and three dimensions, the PCG method has a \({\mathcal {O}}(N^d (\log _2 N)^{2d})\) quasi-optimal complexity for a d dimensional domain with \((N-1)^d\) unknows, \(d=1,2,3\). In addition, a fast direct solver for the three-dimensional Poisson equation is developed, which is of \({\mathcal {O}}(N^{3} (\log _2 N)^{2})\) and improves the existing results on the computational complexity. Numerical examples are given to demonstrate the efficiency of proposed preconditioners and the algorithm for fast matrix-vector multiplications.

In this paper we present new preconditioning techniques for the solution by the preconditioned conjugate gradient (PCG) method of Hermitian Toeplitz systems with real and non- definite generating functions: actually we extend some results of Chan [IMA J. Numer. Anal., 11 (1991), pp. 333–345] and Di Benedetto, Fiorentino, and Serra [Comput. Math. Appl., 25 (1993), pp. 33–45] proved for positive definite Toeplitz systems. Moreover we demonstrate some density properties of the spectra of the preconditioned matrices. Finally, we show that the convergence speed of this PCG method is independent of the dimension of the involved matrices.

Quadarant diagonal partitioning (QDP) method is an iterative method for solving linear system proposed by D. J. Evans et al. (see [1],[2],[3]) appropriate for paralell implementation. Here we show that in case of second order elliptic problem, the classical, color and multicolor ordering technique [4] are applicable for further paralellization of QDP iterative methods. Moreover we give a direct algorithm with arithmetical complexity O(n) for solving a special quadrant tridiagonal matrix and using these results we propose a quadrant tridiagonal preconditioned (QTP) conjugate gradient method for solving second order elliptic problems. The numerical experiments presented here shows that this method in several cases are more effective than other PCG algorithm.

Preface 1. Introduction 2. Circulant preconditioners 3. Unified treatment from kernels 4. Ill-conditioned Toeplitz systems 5. Block Toeplitz systems A. M-files used in the book Bibliography.

This study presents an efficacy comparison of iterative solution methods for solving the compressible-fluid Reynolds equation in modeling air- or gas-lubricated bearings. A direct fixed-point iterative (DFI) method and Newton’s method are employed to transform the Reynolds equation in a form that can be solved iteratively. The iterative solution methods examined are the Gauss–Seidel method, the successive over-relaxation (SOR) method, the preconditioned conjugate gradient (PCG) method, and the multigrid method. The overall solution time is affected by both the transformation method and the iterative method applied. In this study, Newton’s method shows its effectiveness over the straightforward DFI method when the same iterative method is used. It is demonstrated that the use of an optimal relaxation factor is of vital importance for the efficiency of the SOR method. The multigrid method is an order faster than the PCG and optimal SOR methods. Also, the multigrid and PCG methods involve an extended coding work and are less flexible in dealing with gridwork and boundary conditions. Consequently, a compromise has to be made in terms of ease of use as well as programming effort for the solution of the compressible-fluid Reynolds equation.

High-speed downlink packet access (HSDPA) is an interface of UMTS systems for data communication. It combines 16-QAM modulation with low spreading factors to achieve high data rates. Generalized RAKE (G-RAKE) receiver is an advanced receiver, which can be employed in HSDPA to obtain reliable communication. In this work, we investigate iterative methods, which reduce overall complexity requirements of the G-RAKE receiver, without sacrificing its performance. The names of the methods are The Gauss-Seidel (GS), Successive Over-Relaxation (SOR), Conjugate Gradient (CG), and Preconditioned Conjugate Gradient (PCG). Our simulation results show that the PCG method provides the best trade-off between the complexity and required conditions.

Band Toeplitz preconditioners for non-symmetric real Toeplitz systems by preconditioned GMRES method

In this study, we analyze the real-time solution of a series of augmented least squares problems, which are generated by adding information to an original least squares model repetitively. Instead of solving the least squares problems directly, we transform them into a batch of saddle point linear systems and subsequently solve the linear systems using restrictively preconditioned conjugate gradient (RPCG) methods. Approximation of the new Schur complement is generated effectively based on a previously approximated Schur complement. Owing to the variations of the preconditioned conjugate gradient method, the proposed methods generate convergence results similar to the conjugate gradient method and achieve a very fast convergent iterative sequence when the coefficient matrix is well preconditioned. Numerical tests show that the new methods are more effective than some standard Krylov subspace methods. Updated RPCG methods meet the requirement of real-time computing successfully for multifactor models.

SummaryWe propose an adaptive scheme to reduce communication overhead caused by data movement by selectively storing the diagonal blocks of a block‐Jacobi preconditioner in different precision formats (half, single, or double). This specialized preconditioner can then be combined with any Krylov subspace method for the solution of sparse linear systems to perform all arithmetic in double precision. We assess the effects of the adaptive precision preconditioner on the iteration count and data transfer cost of a preconditioned conjugate gradient solver. A preconditioned conjugate gradient method is, in general, a memory bandwidth‐bound algorithm, and therefore its execution time and energy consumption are largely dominated by the costs of accessing the problem's data in memory. Given this observation, we propose a model that quantifies the time and energy savings of our approach based on the assumption that these two costs depend linearly on the bit length of a floating point number. Furthermore, we use a number of test problems from the SuiteSparse matrix collection to estimate the potential benefits of the adaptive block‐Jacobi preconditioning scheme.

In order to solve Toeplitz linear systems An(f)x=b generated by a nonnegative integrable function f, through use of the preconditioned conjugate gradient (PCG) method, several authors have proposed An(g) as preconditioner in the case where g is a trigonometric polynomial [10, 14, 27, 12, 28]. In preceding works, we studied the distribution and the extremal properties of the spectrum of the preconditioned matrix G=An−1(g) An(f). In this paper we prove that the union of the spectra of all the Gn is dense on the essential range of f/g, i.e.,ER(f/g) and we obtain asymptotic information about the rate of convergence of the smallest eigenvalue λln of Gn to r (and of λnn to R). As a consequence of this second order result, it is possible to handle the case where f has zeros of any order θ, through the PCG methods proposed in [10, 14]. This is a noteworthy extension since the techniques developed in [10, 14, 27, 12, 28] are shown to be effective only when f has zeros of even orders. The cost of this procedure is O(n1+c(θ) log n) arithmetic operations (ops) where the quantity c(θ) belongs to interval [0,2−1] and takes the maximum value 2−1 when f has a zero of odd order. Finally, for the special case of zeros of odd orders, we propose a further algorithm which makes use of the PCG techniques proposed in [10, 14, 27, 12, 28] for theeven order case, reducing the cost to O(n long n) ops.