Fast and highly scalable parallel computations for fundamental matrix problems on distributed memory systems

We present fast and scalable parallel computations for a number of important and fundamental matrix problems on distributed memory systems (DMS). These problems include computing the powers, the inverse, the characteristic polynomial, the determinant, the rank, the Krylov matrix, and an LU- and a QR-factorization of a matrix, and solving linear systems of equations. These parallel computations are based on efficient implementations of the fastest sequential matrix multiplication algorithm on DMS. We show that compared with the best known time complexities on PRAM, our parallel matrix computations achieve the same speeds on distributed memory parallel computers (DMPC), and have an extra polylog factor in the time complexities on DMS with hypercubic networks. Furthermore, our parallel matrix computations are fully scalable on DMPC and highly scalable over a wide range of system size on DMS with hypercubic networks. Such fast and scalable parallel matrix computations were not seen before on any distributed memory systems.

The complex nature of the ocean environment requires advanced computational acoustic models to gain insights into the dominating physical factors controlling the underwater sound propagation and scattering in the ocean. In this study, we explore the “ab-initio” approach to solve wave equations with numerical algorithms that can be implemented in distributed memory parallel computers. The goal is to improve the calculation speed so long-range global scale hydroacoustic wave propagation can be studied more efficiently and effectively. Two major algorithms: the finite difference time domain (FDTD) method and the Parabolic Equation (PE) method are investigated. Two PE-based numerical models are considered. One is the Split-Step-Fourier Parabolic Equation (SSFPE) model using split-step Fourier schemes in three-dimensional (3D) environments. The second PE model is a multi-frequency implementation of the Range-dependent Acoustic Model (RAM) that computes two-dimensional (2D) sound pressure fields. One of the primary challenges in global-scale hydroacoustic propagation modeling with the “ab-initio” approach is the demand for significant computational resources on both memories and computational speeds. To address this, we employed parallel computing techniques using directive-based programming languages, such as OpenACC and XscalableACC, to leverage the power of multiple Graphics Processing Unit (GPU) systems and Central Processing Unit (CPU) cluster computers. Our performance evaluation revealed substantial speedup gains. For the FDTD method, we achieved approximately 3.5-fold and 4-fold speedups with four GPUs and four CPU cluster nodes, respectively. With the 3D SSFPE model, we obtained a speedup of 3.7-fold using four CPU cluster nodes. A significant result was observed with the 2D broadband multi-frequency RAM model, where we achieved a 110-fold speedup compared to one core original implementation. These results demonstrate the promising potential of massively parallel computers for ocean acoustic propagation modeling and highlight the significant performance benefits of utilizing parallel computing techniques. Our findings emphasize the importance of efficient computational strategies.

An efficient parallel-computing method for modeling nonisothermal multiphase flow and multicomponent transport in porous and fractured media

Ray tracing is a well known technique to generate life-like images. Unfortunately, ray tracing complex scenes can require large amounts of CPU time and memory storage. Distributed memory parallel computers with large memory capacities and high processing speeds are ideal candidates to perform ray tracing. However, the computational cost of rendering pixels and patterns of data access cannot be predicted until runtime. To parallelize such an application efficiently on distributed memory parallel computers, the issues of database distribution, dynamic data management and dynamic load balancing must be addressed. In this paper, we present a parallel implementation of a ray tracing algorithm on the Intel Delta parallel computer. In our database distribution, a small fraction of database is duplicated on each processor, while the remaining part is evenly distributed among groups of processors. In the system, there are multiple copies of the entire database in the memory of groups of processors. Dynamic data management is acheived by an ALRU cache scheme which can exploit image coherence to reduce data movements in ray tracing consecutive pixels. We balance load among processors by distributing subimages to processors in a global fashion based on previous workload requests. The success of our implementation depends crucially on a number of parameters which are experimentally evaluated. © 1997 John Wiley & Sons, Ltd.

Problem statement: The parallelization of multiple progressive alignment algorithms is a difficult task. All known methods have strong bottlenecks resulting from synchronization delays. This is even more constraining in distributed memory systems, where message passing also delays the interprocess communication. Despite these drawbacks, parallel computing is becoming increasingly necessary to perform multiple sequence alignment. Approach: In this study, it is introduced a solution for parallelizing multiple progressive alignments in distributed memory systems that overcomes such delays. Results: The proposed approach uses threads to separate actual alignment from synchronization and communication. It also uses a different approach to schedule independent tasks. Conclusion/Recommendations: The approach was intensively tested, producing a performance remarkably better than a largely used algorithm. It is suggested that it can be applied to improve the performance of some multiple alignment tools, as CLUSTALW and MUSCLE.

PurposeA computational fluid dynamics code for the calculation of laminar hypersonic multi‐species gas flows in chemical non‐equilibrium in axisymmetric or two‐dimensional configuration on shared and distributed memory parallel computers is presented and validated. The code is designed to work efficiently in combination with an automatic domain decompositioning method developed to facilitate efficient parallel computations of various flow problems.Design/methodology/approachThe baseline implicit numerical method developed is the lower‐upper symmetric Gauss‐Seidel scheme, which is combined with a sub‐iteration scheme to achieve time‐accuracy up to third‐order. The spatial discretisation is based on Roe's flux‐difference splitting and various non‐linear flux limiters maintaining total‐variation diminishing properties and up to third‐order spatial accuracy in continuous regions of flow. The domain subdivision procedure is designed to work for single‐ and multi‐block domains without being constrained by the block boundaries, and an arbitrary number of processors used for the computation.FindingsThe code developed reproduces accurately various types of flows, e.g. flow over a flat plate, diffusive mixing and oscillating shock induced combustion around a projectile fired into premixed gas, and demonstrates close to linear scalability within limits of load imbalance.Research limitations/implicationsThe cases considered are axisymmetric or two‐dimensional, and assume laminar flow. An extension to three‐dimensional turbulent flows is left for future work.Originality/valueResults of a parallel computation, utilising a newly developed automatic domain subdivision procedure, for oscillating shock‐induced combustion around a projectile and various other cases are presented. The influence of entropy correction in Roe's flux‐difference splitting algorithm on diffusive mixing of multi‐species flows was examined.

Measurement and modelling of distributions of data communication times is commonly done for telecommunication networks, but this has not previously been done for message passing communications on parallel computers. We have used the MPIBench program to measure distributions of point-to-point MPI communication times for two different parallel computers, with a low-end Ethernet network and a high-end Quadrics network respectively. Here we present and discuss the results of efforts to fit the measured distributions with standard probability distribution functions such as exponential, lognormal, Erlang, gamma, Pearson 5 and Weibull distributions.KeywordsWeibull DistributionParallel ComputerParallel ProgramInterarrival TimeCommunication TimeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The production of realistic image generated by computer requires a huge amount of computation and a large memory capacity. The use of highly parallel computers allows this process to be performed faster. Distributed memory parallel computers (DMPCs), such as hypercubes or transputer-based machines, offer an attractive performance/cost ratio when the load balancing has been balance and the partition of the data domain has been performed. This paper presents a parallel ray tracing algorithm for DMPC using a Shared Virtual Memory (SVM) which solves these two classical problems. This algorithm has been implemented on a hypercube iPSC/2 and results are given.

Mapping Coriolis-coupled quantum dynamics onto parallel computer architectures

Parallel computing has become fundamental in processing the massive data volumes generated in modern science and industry. This paper presents a comprehensive survey and practical review of parallel computing in data processing, examining key frameworks (MPI, Open MP, CUDA, Hadoop MapReduce, Apache Spark, etc.), implementations, and real- world case studies. We discuss the architectures underlying shared-memory and distributed-memory systems and illustrate how parallelism on CPUs and GPUs is exploited for high- performance computing (HPC) and big data analytics. We review theoretical foundations (including Amdahl’s law for speedup limits) and compare programming models in terms of design, performance, and fault tolerance. A two-column IEEE-style for- mat is used to present critical insights from scholarly literature, with diagrams illustrating parallel architectures and workflows, and tabular summaries of tools and results. Case studies—from scientific HPC simulations to large-scale data analytics—highlight practical parallelism on multicore CPUs, GPUs, and computing clusters. We conclude with a discussion on the convergence of HPC and big data paradigms, the challenges of heterogeneous computing, and emerging trends. This survey serves as a reference for PhD researchers and practitioners seeking to understand frameworks and real-world practices in parallel computing for data processing.

The tiny encryption algorithm (TEA) is widely used when performing dissipative particle dynamics (DPD) calculations in parallel, usually on distributed memory systems. In this research, we reduced the computational cost of the TEA hash function and investigated the influence of the quality of the random numbers generated on the results of DPD calculations. It has already been established that the randomness, or quality, of the random numbers depend on the number of processes from internal functions such as SHIFT, XOR and ADD, which are commonly referred to as "rounds". Surprisingly, if we choose seed numbers from high entropy sources, with a minimum number of rounds, the quality of the random numbers generated is sufficient to successfully perform accurate DPD simulations. Although it is well known that using a minimal number of rounds is insufficient for generating high-quality random numbers, the combination of selecting good seed numbers and the robustness of DPD simulations means that we can reduce the random number generation cost without reducing the accuracy of the simulation results.

Parallel matrix computations in air pollution modelling

A parallel algorithm for the simulation of steady‐state, two‐dimensional flows of integral viscoelastic fluids on Multiple Instruction Multiple DATA (MIMD) computers with distributed memory is proposed. The algorithm has been implemented within the commercial finite element package polyflow developed at Louvain‐la‐Neuve and widely used in polymer processing applications. Parallel performance results obtained on the Intel iPSC/860 hypercube are discussed for the flow of a K.B.K.Z. fluid through an abrupt contraction. It is found that satisfactory levels of parallel speed‐up can be obtained by using a rather simple allocation scheme to schedule workload to the available processors. The optimal use of the parallel computer requires, however, a dynamic load allocation scheme. An adaptive strategy that reallocates the workload to the processors has been proposed on the basis of the history of the nonlinear iterative process. The proposed adaptive allocation scheme is found to carry only marginal compute and communication overheads.

A mesh-vertex finite volume scheme for solving the Euler equations on triangular unstructured meshes is implemented on a multiple-instruction/multiple-data stream parallel computer. An explicit four-stage Runge-Kutta scheme is used to solve two-dimensional flow problems. A family of implicit schemes is also developed to solve these problems, where the linear system that arises at each time step is solved by a preconditioned GMRES algorithm. Two partitioning strategies are employed: one that partitions triangles and the other that partitions vertices. The choice of the preconditioner in a distributed memory setting is discussed. All of the methods are compared both in terms of elapsed times and convergence rates. It is shown that the implicit schemes offer adequate parallelism at the expense of minimal sequential overhead. The use of a global coarse grid to further minimize this overhead is also investigated. The schemes are implemented on a distributed memory parallel computer, the Intel iPSC/860.

Three-dimensional unstructured mesh ocean modelling