Parallel PCG Algorithm On Distributed Network By PVM

The preconditioned conjugate gradient (PCG) algorithm is a well-known iterative method for solving sparse linear systems in scientific computations. GPU-accelerated PCG algorithms for large-sized problems have attracted considerable attention recently. However, on a specific multi-GPU platform, producing a highly parallel PCG implementation for any large-sized problem requires significant time because several manual steps are involved in adjusting the related parameters and selecting an appropriate storage format for the matrix block that is assigned to each GPU. This motivates us to propose adaptive optimization modeling of PCG on multi-GPUs, which mainly involves the following parts: (1) an optimization multi-GPU parallel framework of PCG and (2) the profile-based optimization modeling for each one of the main components of the PCG algorithm, including vector operation, inner product, and sparse matrix-vector multiplication (SpMV). Our model does not construct a new storage format or kernel but automatically and rapidly generates an optimal parallel PCG algorithm for any problem on a specific multi-GPU platform by integrating existing storage formats and kernels. We take a vector operation kernel, an inner-product kernel, and five popular SpMV kernels for an example to present the idea of constructing the model. Given that our model is general, independent of the problems, and dependent on the resources of devices, this model is constructed only once for each type of GPU. The experiments validate the high efficiency of our proposed model.

A multi-GPU parallel optimization model for the preconditioned conjugate gradient algorithm

The efficiency of parallel preconditioned conjugate gradient PCG algorithm for solving large sparse linear systems arising from application of interior point methods to conic optimisation problems in the context of nonlinear finite element limit analysis FELA for computational geomechanics is studied. For large 3D problems, the use of direct solvers in general becomes prohibitively expensive owing to exponentially growing memory requirements and computational time. And the so-called saddle-point systems resulting from use of optimisation framework is not an exemption. On the other hand, although preconditioned iterative methods have moderate storage requirements and therefore can be applied to much larger problems than direct methods, they usually exhibit high number of iterations to reach convergence. In the present paper, we show that this problem can be effectively tackled using efficient variants of sparse approximate inverse preconditioners along with an elaborate parallel implementation on multicore CPUs and significant improvements can be achieved by parallel implementation on graphic processing unit GPU. Furthermore, the efficiency of our proposed implementation is verified by the presented numerical results.

Virtual earthquake engineering laboratory with physics-based degrading materials on parallel computers

A parallel numerical acoustic simulation on a GPU using an edge-based smoothed finite element method

A new class of normalized explicit approximate inverse matrix techniques, based on normalized approximate factorization procedures, for solving sparse linear systems resulting from the finite difference discretization of partial differential equations in three space variables are introduced. A new parallel normalized explicit preconditioned conjugate gradient square method in conjunction with normalized approximate inverse matrix techniques for solving efficiently sparse linear systems on distributed memory systems, using Message Passing Interface (MPI) communication library, is also presented along with theoretical estimates on speedups and efficiency. The implementation and performance on a distributed memory MIMD machine, using Message Passing Interface (MPI) is also investigated. Applications on characteristic initial/boundary value problems in three dimensions are discussed and numerical results are given.

Groundwater flow simulation has become one of the top international issues in new generation of environmental applications. When managing large-scale groundwater flow problems, the intensive computational ability and large amounts of memory space required for modeling are the main bottlenecks for researchers. In order to solve three-dimensional large-scale groundwater flow problems more rapidly, the Open MP was adopted to parallelize the preconditioned conjugate gradient (PCG) algorithm in this paper. And this paper carried out a numerical experiment of the three-dimensional groundwater flow model on a computer with four cores. Based on the numerical experiment, it is found that the execution time of the original serial PCG program is about 1.74 to 2.86 times of the parallel PCG program executed with different number of threads. The experimental results also demonstrate that the PCG solver based on Open MP is an effective way for solving large-scale three-dimensional groundwater flow problem.

Utility of the preconditioned conjugate gradient algorithm with a diagonal preconditioner for solving mixed-model equations in animal breeding applications was evaluated with 16 test problems. The problems included single- and multiple-trait analyses, with data on beef, dairy, and swine ranging from small examples to national data sets. Multiple-trait models considered low and high genetic correlations. Convergence was based on relative differences between left- and right-hand sides. The ordering of equations was fixed effects followed by random effects, with no special ordering within random effects. The preconditioned conjugate gradient program implemented with double precision converged for all models. However, when implemented in single precision, the preconditioned conjugate gradient algorithm did not converge for seven large models. The preconditioned conjugate gradient and successive overrelaxation algorithms were subsequently compared for 13 of the test problems. The preconditioned conjugate gradient algorithm was easy to implement with the iteration on data for general models. However, successive overrelaxation requires specific programming for each set of models. On average, the preconditioned conjugate gradient algorithm converged in three times fewer rounds of iteration than successive overrelaxation. With straightforward implementations, programs using the preconditioned conjugate gradient algorithm may be two or more times faster than those using successive overrelaxation. However, programs using the preconditioned conjugate gradient algorithm would use more memory than would comparable implementations using successive overrelaxation. Extensive optimization of either algorithm can influence rankings. The preconditioned conjugate gradient implemented with iteration on data, a diagonal preconditioner, and in double precision may be the algorithm of choice for solving mixed-model equations when sufficient memory is available and ease of implementation is essential.

The linear quadtree is a useful data structure for representing an image for the sake of the storage saving and further image manipulations. In this paper we propose a linear quadtree coding scheme and implement this algorithm on the parallel virtual machine (PVM). Our goal is to demonstrate the applicability of using the PVM in combining the computing power of computers in a network to solve this kind of image processing problems. The processors in the PVM are organized as a master-slave paradigm and various numbers of processors are applied for different PVM’s to compare their performances. Experimental results show that the speedup of solving this image encoding problem in parallel is quite satisfactory. With such a PVM environment which is easily accessible in the public domain, high performance computing is truly possible without additional hardware cost.

For many large scientific applications, computing on a cluster is a viable, economical alternative to a dedicated parallel machine. Application performance on a cluster is largely determined by the speed of the underlying communication network. The authors use a parallel network approach to improve the communication network performance. More specifically, they use multiple networks based on Ethernet to improve the performance. The authors modified the parallel virtual machine (PVM) environment to take advantage of multiple Ethernets by using round robin scheduling strategy. The modified PVM, referred as PPVM, showed an improvement of 35% over PVM under no background load, and an improvement of 100% under background load on two channels. >

MATLAB Distributed Computing Server (MDCS) and Parallel Virtual Machine (PVM) software are two types of distributed computing environments. MDCS is recently used in selecting the best network training algorithm and assessing the effect of parallelization. Since it is practical and user friendly, it gives PVM the opportunity to become the communication paradigm of choice. The PVM provides a powerful set of process control and dynamic resource management features. In the distributed parallel computing (DPC), however, both solutions have different strengths and limitations. Based on these concerns, this paper compares the numerical analysis for mathematical modeling of large sparse 2D and 3D of second order parabolic partial differential equations (PDE) on MDCS and PVM based on parallel performance indicators (PPI). The PDE geometry is discretized into a sparse grid structure using the FDM method. In using a method with the highest accuracy, Parallel Alternating Group Explicit (PAGE) scheme was chosen. The parallel strategies focus on the PAGE's convergence speed and various domain de-composition techniques, as well as a block iterative scheme and load balance using fine granular techniques. Furthermore, comparison of distributed computing environments also relies on multiple processors running on Unix-like operating systems with Fedora installed to support large-scale simulations. The analysis and validation of PPI for both communication software is also investigated in this paper. For a sequential algorithm, accuracy, estimation of error, and stability are employed as indicators. As a conclusion, based on the PPI and numerical analysis, the tables and graphs show that compared to MDCS, PVM is a better environment for solving multidimensional parabolic PDE modeling.KeywordsDistributed computingParabolic PDEsMDCSPVM

The implementation of a distributed digital logic simulation algorithm on a network of workstations is presented. The simulation of digital circuits is done using a demand driven approach. The simulation is performed on a network of workstations using the Parallel Virtual Machine (PVM) software. The use of the PVM as a platform for parallel and distributed applications is assessed in the light of the test results. The feasibility of running parallel logical simulation algorithms on such platform is considered.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Parallel preconditioned conjugate gradient algorithm on GPU

The objective of this paper is to investigate the efficiency of hybrid solution methods when incorporated into large‐scale optimization problems solved by evolution strategies (ESs) and to demonstrate their influence on the overall performance of these optimization algorithms. ESs imitate biological evolution and combine the concept of artificial survival of the fittest with evolutionary operators to form a robust search mechanism. In this paper modified multi‐membered evolution strategies with discrete variables are adopted. Two solution methods are implemented based on the preconditioned conjugate gradient (PCG) algorithm. The first method is a PCG algorithm with a preconditioner resulted from a complete Cholesky factorization, and the second is a PCG algorithm in which a truncated Neumann series expansion is used as a preconditioner. The numerical tests presented demonstrate the computational advantages of the proposed methods, which become more pronounced in large‐scale optimization problems and in a parallel computing environment.

Parallel workstation clusters and MPI for sparse systems in computational science