Comparison of different solving strategies in multiple-trait mixed model equations with iterative algorithms

Utility of the preconditioned conjugate gradient algorithm with a diagonal preconditioner for solving mixed-model equations in animal breeding applications was evaluated with 16 test problems. The problems included single- and multiple-trait analyses, with data on beef, dairy, and swine ranging from small examples to national data sets. Multiple-trait models considered low and high genetic correlations. Convergence was based on relative differences between left- and right-hand sides. The ordering of equations was fixed effects followed by random effects, with no special ordering within random effects. The preconditioned conjugate gradient program implemented with double precision converged for all models. However, when implemented in single precision, the preconditioned conjugate gradient algorithm did not converge for seven large models. The preconditioned conjugate gradient and successive overrelaxation algorithms were subsequently compared for 13 of the test problems. The preconditioned conjugate gradient algorithm was easy to implement with the iteration on data for general models. However, successive overrelaxation requires specific programming for each set of models. On average, the preconditioned conjugate gradient algorithm converged in three times fewer rounds of iteration than successive overrelaxation. With straightforward implementations, programs using the preconditioned conjugate gradient algorithm may be two or more times faster than those using successive overrelaxation. However, programs using the preconditioned conjugate gradient algorithm would use more memory than would comparable implementations using successive overrelaxation. Extensive optimization of either algorithm can influence rankings. The preconditioned conjugate gradient implemented with iteration on data, a diagonal preconditioner, and in double precision may be the algorithm of choice for solving mixed-model equations when sufficient memory is available and ease of implementation is essential.

The stationary point of optimization problems can be obtained via conjugate gradient (CG) methods without the second derivative. Many researchers have used this method to solve applications in various fields, such as neural networks and image restoration. In this study, we construct a three-term CG method that fulfills convergence analysis and a descent property. Next, in the second term, we employ a Hestenses-Stiefel CG formula with some restrictions to be positive. The third term includes a negative gradient used as a search direction multiplied by an accelerating expression. We also provide some numerical results collected using a strong Wolfe line search with different sigma values over 166 optimization functions from the CUTEr library. The result shows the proposed approach is far more efficient than alternative prevalent CG methods regarding central processing unit (CPU) time, number of iterations, number of function evaluations, and gradient evaluations. Moreover, we present some applications for the proposed three-term search direction in image restoration, and we compare the results with well-known CG methods with respect to the number of iterations, CPU time, as well as root-mean-square error (RMSE). Finally, we present three applications in regression analysis, image restoration, and electrical engineering.

This study presents an efficacy comparison of iterative solution methods for solving the compressible-fluid Reynolds equation in modeling air- or gas-lubricated bearings. A direct fixed-point iterative (DFI) method and Newton’s method are employed to transform the Reynolds equation in a form that can be solved iteratively. The iterative solution methods examined are the Gauss–Seidel method, the successive over-relaxation (SOR) method, the preconditioned conjugate gradient (PCG) method, and the multigrid method. The overall solution time is affected by both the transformation method and the iterative method applied. In this study, Newton’s method shows its effectiveness over the straightforward DFI method when the same iterative method is used. It is demonstrated that the use of an optimal relaxation factor is of vital importance for the efficiency of the SOR method. The multigrid method is an order faster than the PCG and optimal SOR methods. Also, the multigrid and PCG methods involve an extended coding work and are less flexible in dealing with gridwork and boundary conditions. Consequently, a compromise has to be made in terms of ease of use as well as programming effort for the solution of the compressible-fluid Reynolds equation.

Data included 90,242,799 test day records from first, second and third parities of 5,402,484 Holstein cows and 9,326,754 animals in the pedigree. Additionally, daily temperature humidity indexes (THI) from 202 weather stations were available. The fixed effects included herd test day, age at calving, milking frequency and days in milk classes (DIM). Random effects were additive genetic, permanent environment and herd-year and were fit as random regressions. Covariates included linear splines with four knots at 5, 50, 200 and 305 DIM and a function of THI. Mixed model equations were solved using an iteration on data program with a preconditioned conjugate gradient algorithm. Preconditioners used were diagonal (D), block diagonal due to traits (BT) and block diagonal due to traits and correlated effects (BTCORR). One run included BT with a 'diagonalized' model in which the random effects were reparameterized for diagonal (co)variance matrices among traits (BTDIAG). Memory requirements were 8.7 Gb for D, 10.4 Gb for BT and BTDIAG, and 24.3 Gb for BTCORR. Computing times (rounds) were 14 days (952) for D, 10.7 days (706) for BT, 7.7 days (494) for BTDIAG and 4.6 days (289) for BTCORR. The convergence pattern was strongly influenced by the choice of fixed effects. When sufficient memory is available, the option BTCORR is the fastest and simplest to implement; the next efficient method, BTDIAG, requires additional steps for diagonalization and back-diagonalization.

The application of parallel computers to reservoir simulation is studied with the aid of two recently introduced Multiple Instruction, Multiple Data (MIMD) computers. The problems posed by reservoir simulators are shown to be particularly well suited for such parallel computers* Fundamental parallel programming techniques using Fortran are presented for the Hetrogeneous Element Processor (HEP) produced by Denelcor and the iPSC Hypercube produced by Intel. These techniques are then applied to two of the most time consuming tasks in reservoir simulation: forming the matrix coefficients and producing a sparse matrix solution. The problem of parallel formation of sparse matrix coefficients is addressed for the single and multiphase cases using both black oil and compositional fluid models. Parallel sparse matrix solution is addressed by considering D4 ordered Gaussian elimination, and the multigrid, conjugate gradient, and Successive Over-Relaxation (SOR) methods. Three parallel algorithms based on SOR are developed to illustrate the possible diversity of parallel algorithms. The red-black and multicolored SOR methods which are often used on vector machines are shown to be easily adaptable to parallel MIMD machine. Matrix partitioning and a new method of iteration pipelining are also presented as parallel SOR methods. The various SOR algorithms, in parallel form, are mapped onto the HEP to illustrate the speedups currently possible on an MIMD machine. Projections are made on the future importance of these types of machines to reservoir simulation.

A program generator for the incomplete cholesky conjugate gradient (ICCG) method with a symmetrizing preprocessor

The preconditioned conjugate gradient (PCG) algorithm is a well-known iterative method for solving sparse linear systems in scientific computations. GPU-accelerated PCG algorithms for large-sized problems have attracted considerable attention recently. However, on a specific multi-GPU platform, producing a highly parallel PCG implementation for any large-sized problem requires significant time because several manual steps are involved in adjusting the related parameters and selecting an appropriate storage format for the matrix block that is assigned to each GPU. This motivates us to propose adaptive optimization modeling of PCG on multi-GPUs, which mainly involves the following parts: (1) an optimization multi-GPU parallel framework of PCG and (2) the profile-based optimization modeling for each one of the main components of the PCG algorithm, including vector operation, inner product, and sparse matrix-vector multiplication (SpMV). Our model does not construct a new storage format or kernel but automatically and rapidly generates an optimal parallel PCG algorithm for any problem on a specific multi-GPU platform by integrating existing storage formats and kernels. We take a vector operation kernel, an inner-product kernel, and five popular SpMV kernels for an example to present the idea of constructing the model. Given that our model is general, independent of the problems, and dependent on the resources of devices, this model is constructed only once for each type of GPU. The experiments validate the high efficiency of our proposed model.

For transonic flow calculations, a combination of the successive line over-relaxation (SLOR) and the preconditioned conjugate gradient (CG) method has been suggested by Wong and Hafez (1981). This paper studies the method of minimal residual (MR) which avoids a combined iteration. This method is closely related to the CG method, may be regarded as a first-order gradient method, and is applicable to symmetric and nonsymmetric matrices. The problem is formulated as a nonlinear mixed elliptic-hyperbolic partial differential equation which includes an artificial viscosity and a switching function which is zero in subsonic regions and nonzero in supersonic regions. Alternatives to the SLOR method which provide faster convergence rates are introduced. The preconditioned MR algorithm is developed, and transonic potential flows around NACA 0012 airfoil are calculated for different Mach numbers and angles of attack. Preliminary results are presented, demonstrating that the MR algorithm requires no parameter estimation and rapidly converges for subsonic flows.

The objective of this paper is to investigate the efficiency of hybrid solution methods when incorporated into large‐scale optimization problems solved by evolution strategies (ESs) and to demonstrate their influence on the overall performance of these optimization algorithms. ESs imitate biological evolution and combine the concept of artificial survival of the fittest with evolutionary operators to form a robust search mechanism. In this paper modified multi‐membered evolution strategies with discrete variables are adopted. Two solution methods are implemented based on the preconditioned conjugate gradient (PCG) algorithm. The first method is a PCG algorithm with a preconditioner resulted from a complete Cholesky factorization, and the second is a PCG algorithm in which a truncated Neumann series expansion is used as a preconditioner. The numerical tests presented demonstrate the computational advantages of the proposed methods, which become more pronounced in large‐scale optimization problems and in a parallel computing environment.

Parallel preconditioned conjugate gradient algorithm on GPU

Iterative methods applied to the normal equationsA T Ax=A T b are sometimes used for solving large sparse linear least squares problems. However, when the matrix is rank-deficient many methods, although convergent, fail to produce the unique solution of minimal Euclidean norm. Examples of such methods are the Jacobi and SOR methods as well as the preconditioned conjugate gradient algorithm. We analyze here an iterative scheme that overcomes this difficulty for the case of stationary iterative methods. The scheme combines two stationary iterative methods. The first method produces any least squares solution whereas the second produces the minimum norm solution to a consistent system.

High-speed downlink packet access (HSDPA) is an interface of UMTS systems for data communication. It combines 16-QAM modulation with low spreading factors to achieve high data rates. Generalized RAKE (G-RAKE) receiver is an advanced receiver, which can be employed in HSDPA to obtain reliable communication. In this work, we investigate iterative methods, which reduce overall complexity requirements of the G-RAKE receiver, without sacrificing its performance. The names of the methods are The Gauss-Seidel (GS), Successive Over-Relaxation (SOR), Conjugate Gradient (CG), and Preconditioned Conjugate Gradient (PCG). Our simulation results show that the PCG method provides the best trade-off between the complexity and required conditions.

The A-phi formulation, which is widely used in electromagnetic analysis, leads to a redundant linear system of equations that includes a substantial number of redundant degrees of freedom (DOF). We can derive a redundancy-reduced linear system of equations by eliminating the redundant DOF, thereby decreasing the computation costs per iteration for iterative solvers, such as the incomplete Cholesky conjugate gradient (ICCG) solver. This does not, however, result in a reduction in total computation time, due to significant convergence deterioration. In this paper, we present a solution to this problem in the form of folded preconditioners. First, the theorem presented reveals that, for any preconditioned Krylov subspace method for the original redundant linear systems, we can derive the equivalent Krylov subspace method for the redundancy-reduced linear systems by using the corresponding folded preconditioner. As an uncomplicated example, the standard ICCG solver for the original redundant systems has exactly the same convergence property as the CG solver for the redundancy-reduced systems using the folded variant of the IC preconditioner (the folded IC preconditioner). Furthermore, we discuss efficient computational procedures for the folded preconditioners and the design of Krylov subspace algorithms using the preconditioners. A sample full-wave analysis demonstrates the good performance of a newly developed solver, the conjugate orthogonal conjugate gradient (COCG) method with the folded IC preconditioner. The new solver not only lowers the computation costs per iteration by reducing the number of DOF, but also completely avoids the convergence deterioration.

This chapter discusses the construction of rational cubic Ball (cubic/quadratic) which consists of three parameters with C1 continuity. These three parameters can be used to refine the interpolating curve as well as to obtain the best approximation in error estimation. This scheme was tested by interpolate data from the true function. To improve the accuracy, a new method to estimate the first derivative is proposed as it is significant in curve interpolation. Based on error measurement such as Maximum Absolute Error (\(e_{max}\)), Coefficient of Determination (\(R^2 )\), Root Mean Square Error (\(RMSE\)) and Central Processing Unit (CPU) time in second, the proposed scheme is better than existing schemes. Notably, we achieved \(e_{max} =\) 0.01331105, \(R^2 =\) 0.91398405, \(RMSE =\) 0.00448758 and CPU time = 0.00009958 by using the new derivative estimation. Thus, the proposed scheme with construction of a new derivative estimation has improved the existing rational interpolant.KeywordsRational interpolationCurve interpolationFree parameters \(e_{max}\) R 2 RMSECPU time

The effectiveness of renumbering for the incomplete Cholesky conjugate gradient (ICCG) solver, which is usually applied to direct solvers, is examined quantitatively by analyzing 3D standard benchmark models. On an acceleration factor which is introduced to obtain convergence quickly, indices for determining the optimum value of the acceleration factor, which minimizes the number of iterations, are discussed. It is found that the renumbering is effective to use with the ICCG solver, and the solver using the acceleration factor gives a good convergence characteristic even in the case when the conventional solver fails to provide convergent solutions.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>