Data Flow Analysis of Orthogonal Properties on the Conjugate Gradient and the Lanczos Algorithm

Computer Solution of Large Linear Systems

Many of the fast methods for factoring integers and computing discrete logarithms require the solution of large sparse linear systems of equations over finite fields. This paper presents the results of implementations of several linear algebra algorithms. It shows that very large sparse systems can be solved efficiently by using combinations of structured Gaussian elimination and the conjugate gradient, Lanczos, and Wiedemann methods.

This paper presents a novel hardware solution for solving sparse matrices using emulation technology. The proposed solution introduces a bio-inspired technique, which is Genetic Algorithm (GA) to solve the sparse system of linear equations (SLEs). The proposed solution can be used with any physical system that can be modeled as SLEs. Computations involved in Finite Element Method (FEM) consume too much time, which affects the final time-to-market value. Profiling shows that the most time-consuming part in the simulation process is the solver part which is responsible for solving the resultant SLEs generated from the FEM. The total number of equations may reach thousands or millions of linear equations. Hardware implementation of GA and making a good use of parallelism in implementing the architecture accelerate the time taken by the solver part. Solving the system by classical numerical methods, such as conjugate gradient (CG) implemented in software is performed sequentially, here the architecture allows the parallelism in performing GA operators, such as selection, crossover, and mutation. The parallelism in the proposed architecture is the superiority point which we rely on. The results show that our proposed method to solve the SLEs outperforms the classical CG software implementation on run time and number of equations.

The Multi-Stage Nested Wiener Filter (MSNWF) and the Conjugate Gradient (CG) method yield the solution of the Wiener-Hopf equation in the Krylov subspace of the covariance matrix of the observation and the crosscorrelation vector between the observation and the desired signal. Using the Lanczos algorithm instead of the Arnoldi algorithm for the MSNWF simplifies the computation of the Krylov subspace basis. In this paper, we show the relationship between the CG method and the Lanczos based MSNWF and finally derive that the MSNWF may be mathematically transformed into the CG algorithm. Consequently, we present a new implementation of the MSNWF where the weight vector and the Mean Square Error (MSE) is directly updated as each new stage is added. The new algorithm is applied to an Enhanced Data rates for GSM Evolution (EDGE) system where it linearily equalizes the received signal. Simulation results demonstrate the ability of the MSNWF to reduce the receiver complexity while maintaining the same level of system performance.

A projection method for solving nonsymmetric linear systems on multiprocessors

Abstract: The conjugate gradient (CG) method is a one of the common approaches for solving unconstrained optimization problems, notably known for its suitability for large scale problems. Many recent studies show that this method is also useful for problems of smaller scale. One of the methods used for improving the performance of CG method is hybrid approach, where a CG method is combined with another method. In this study, the ARM CG method is combined with the SMR CG method and tested under exact line search. The resulting hybrid algorithm is globally convergent under exact line search and shown to perform well numerically in comparison to other tested CG methods.

Iterative solvers allow for a trade-off between speed and accuracy. We propose an iterative method for the estimation of the internal states of a given discrete-time linear state-space model from a series of noisy measurements. In particular we identify the MAP estimate of those states as being the solution of a sparse system of linear equations and derive an iterative solver based on the conjugate gradient method. We derive convergence results to quantify the trade-off between speed and accuracy and finally apply the method to channel estimation where it is shown to outperform Kalman smoothing complexity-wise.

We present a geometric framework for surface denoising using graph signal processing, which is an emerging field that aims to develop new tools for processing and analyzing graph-structured data. The proposed approach is formulated as a constrained optimization problem whose objective function consists of a fidelity term specified by a noise model and a regularization term associated with prior data. Both terms are weighted by a normalized mesh Laplacian, which is defined in terms of a data-adaptive kernel similarity matrix in conjunction with matrix balancing. Minimizing the objective function reduces it to iteratively solve a sparse system of linear equations via the conjugate gradient method. Extensive experiments on noisy carpal bone surfaces demonstrate the effectiveness of our approach in comparison with existing methods. We perform both qualitative and quantitative comparisons using various evaluation metrics.

This paper investigates the performance of a class of methods based on the Preconditioned Conjugate Gradient (PCG) method for the solution of large and sparse systems of linear equations which arise from the p‐version of the adaptive finite element method. The hierarchical type of preconditioning used in the past is extended to the SSOR and incomplete factorization type preconditioning and applied with a global handling of the stiffness matrix. The paper also presents some new ideas on Domain Decomposition (DD) matrix‐handling techniques, whereby the explicit formulation of the resulting Schur complement is avoided by utilizing the PCG method with efficient preconditioners and by processing separately on the coarse and the fine mesh. The suitability of the methods to the p‐version of the adaptive finite element method is demonstrated through extensive numerical tests in two dimensions.

The left-preconditioned communication avoiding conjugate gradient (LP-CA-CG) method is applied to the pressure Poisson equation in the multiphase CFD code JUPITER. The arithmetic intensity of the LP-CA-CG method is analyzed, and is dramatically improved by loop splitting for inner product operations and for three term recurrence operations. Two LPCA-CG solvers with block Jacobi preconditioning and with underlap preconditioning are developed. The former is developed based on a hybrid CA approach, in which CA is applied only to global collective communications for inner product operations. The latter is a full CA approach, in which CA is applied also to local point-to-point communications in sparse matrix-vector (SpMV) operations and preconditioning. CA-SpMV requires additional computation for overlapping regions. CA-preconditiong is enabled by underlap preconditioning, which approximates preconditioning for overlapping regions by point Jacobi preconditioning. It is shown that on the K computer, the former is faster, because the performance of local point-to-point communications scales well, and the convergence property becomes worse with underlap preconditioning. The LP-CA-CG solver shows good strong scaling up to 30,000 nodes, where the LP-CA-CG solver achieved higher performance than the original CG solver by reducing the cost of global collective communications by 69 percent.

Self-scaled conjugate gradient training algorithms

Presently, a preferred minimization for strong‐constraint four‐dimensional variational (4D‐Var) assimilation uses a Lanczos‐based conjugate gradient (CG) algorithm. This requires the availability of a square‐root of the background‐error covariance matrix (B). In the context of weak‐constraint 4D‐Var, this requirement might be too restrictive for the formulations of the model error term. It might therefore be desirable to avoid a square‐root decomposition of the augmented background term. An appealing minimization scheme is the double CG minimization employed, for example, in the grid‐point statistical interpolation (GSI) analysis. Realizing the double CG algorithm is a special case of the more general bi‐conjugate gradient (BiCG) method for solving non‐symmetric problems, the present work introduces a Lanczos‐based preconditioning strategy when B, instead of its square‐root, is used initially. Implementation of the scheme is done in the context of the GSI analysis system, and preliminary experiments are presented using its 3D‐Var version. Comparison of the Lanczos‐based CG and the BiCG shows that the algorithms converge at the same rate and to the same solution. Despite the additional computational cost, the importance of the re‐orthogonalization step is also shown to be fundamental to any of these CG algorithms. Furthermore, when using the Hessian eigenvectors for preconditioning, the BiCG behaviour is shown to be comparable to that of the Lanczos‐CG algorithm. Both schemes construct the same approximation of the Hessian with the same number of eigenvectors, and benefit in the same way from the reduction of the condition number. The efficiency, computational cost, and stability of the three algorithms are discussed. Copyright © 2012 Royal Meteorological Society

The Lanczos algorithm is among the most frequently used iterative techniques for computing a few dominant eigenvalues of a large sparse non-symmetric matrix. At the same time, it serves as a building block within biconjugate gradient (BiCG) and quasi-minimal residual (QMR) methods for solving large sparse non-symmetric systems of linear equations. It is well known that, when implemented on distributed-memory computers with a huge number of processes, the synchronization time spent on computing dot products increasingly limits the parallel scalability. Therefore, we propose synchronization-reducing variants of the Lanczos, as well as BiCG and QMR methods, in an attempt to mitigate these negative performance effects. These so-called s-step algorithms are based on grouping dot products for joint execution and replacing time-consuming matrix operations by efficient vector recurrences. The purpose of this paper is to provide a rigorous derivation of the recurrences for the s-step Lanczos algorithm, introduce s-step BiCG and QMR variants, and compare the parallel performance of these new s-step versions with previous algorithms.

PurposeThe purpose of this paper is to apply the conjugate gradient (CG) method, together with the adjoint operator (AO) to the pulsating heat pipe problem, including some quite interesting experimental results. The CG method, together with the AO, was able to estimate the unknown functions more efficiently than the other techniques presented in this paper. The estimation of local heat transfer coefficients, rather than the global ones, in pulsating heat pipes is a relatively new subject and presenting a robust, efficient and self-regularized inverse tool to estimate it, supported also by some experimental results, is the main purpose of this paper. To also increase the visibility and the general use of the paper to the heat transfer community, the authors include, as supplemental material, all numerical and experimental data used in this paper.Design/methodology/approachThe approach was established on the solution of the inverse heat conduction problem in the wall by using as starting data the temperature measurements on the outer surface. The procedure is based on the CG method with AO. The here proposed approach was first verified adopting synthetic data and then it was validated with real cases regarding pulsating heat pipes.FindingsAn original fast methodology to estimate local convective heat flux is proposed. The procedure has been validated both numerically and experimentally. The procedure has been compared to other classical methods presenting some peculiar benefits.Practical implicationsThe approach is suitable for pulsating heat pipes performance evaluation because these devices present a local heat flux distribution characterized by an important variation both in time and in space as a result of the complex flow patterns that are generated in this type of devices.Originality/valueThe procedure here proposed shows these benefits: it affords a general model of the heat conduction problem that is effortlessly customized for the particular case, it can be applied also to large datasets and it presents reduced computational expense.

At present, the conjugate gradient (CG) method of Hager and Zhang (Hager and Zhang, SIAM Journal on Optimization, 16(2005)) is regarded as one of the most effective CG methods for optimization problems. In order to further study the CG method, we develop the Hager and Zhang's CG method and present two modified CG formulas, where the given formulas possess the value information of not only the gradient but also the function. Moreover, the sufficient descent condition will be holden without any line search. The global convergence is established for nonconvex function under suitable conditions. Numerical results show that the proposed methods are competitive to the normal conjugate gradient method.