Fast and Scalable Parallel Matrix Computations on Distributed Memory Systems

We present fast and highly scalable parallel computations for a number of important and fundamental matrix problems on distributed memory systems (DMS). These problems include matrix multiplication, matrix chain product, and computing the powers, the inverse, the characteristic polynomial, the determinant, the rank, the Krylov matrix, and an LU- and a QR-factorization of a matrix, and solving linear systems of equations. Our highly scalable parallel computations for these problems are based on a highly scalable implementation of the fastest sequential matrix multiplication algorithm on DMS. We show that compared with the best known parallel time complexities on parallel random access machines (PRAM), the most powerful but unrealistic shared memory model of parallel computing, our parallel matrix computations achieve the same speeds on distributed memory parallel computers (DMPC), and have an extra polylog factor in the time complexities on DMS with hypercubic networks. Furthermore, our parallel matrix computations are fully scalable on DMPC and highly scalable over a wide range of system size on DMS with hypercubic networks. Such fast (in terms of parallel time complexity) and highly scalable (in terms of our definition of scalability) parallel matrix computations were rarely seen before on any distributed memory systems.

A strategy is proposed for fast and efficient parallel computations using a cluster of workstations to simulate flood flows. We implement ANUGA software to conduct flood simulations. ANUGA solves the two-dimensional shallow water equations using a finite volume method. ANUGA can be run in either sequential or parallel. We focus our work on parallel computations. Our computation scenarios are implemented on a cluster of workstations. Parallel computation results are assessed based on execution time and efficiency. The strategy that we propose gives a significant advantage when we want to get fast and efficient computations using a cluster of workstations.

The complex nature of the ocean environment requires advanced computational acoustic models to gain insights into the dominating physical factors controlling the underwater sound propagation and scattering in the ocean. In this study, we explore the “ab-initio” approach to solve wave equations with numerical algorithms that can be implemented in distributed memory parallel computers. The goal is to improve the calculation speed so long-range global scale hydroacoustic wave propagation can be studied more efficiently and effectively. Two major algorithms: the finite difference time domain (FDTD) method and the Parabolic Equation (PE) method are investigated. Two PE-based numerical models are considered. One is the Split-Step-Fourier Parabolic Equation (SSFPE) model using split-step Fourier schemes in three-dimensional (3D) environments. The second PE model is a multi-frequency implementation of the Range-dependent Acoustic Model (RAM) that computes two-dimensional (2D) sound pressure fields. One of the primary challenges in global-scale hydroacoustic propagation modeling with the “ab-initio” approach is the demand for significant computational resources on both memories and computational speeds. To address this, we employed parallel computing techniques using directive-based programming languages, such as OpenACC and XscalableACC, to leverage the power of multiple Graphics Processing Unit (GPU) systems and Central Processing Unit (CPU) cluster computers. Our performance evaluation revealed substantial speedup gains. For the FDTD method, we achieved approximately 3.5-fold and 4-fold speedups with four GPUs and four CPU cluster nodes, respectively. With the 3D SSFPE model, we obtained a speedup of 3.7-fold using four CPU cluster nodes. A significant result was observed with the 2D broadband multi-frequency RAM model, where we achieved a 110-fold speedup compared to one core original implementation. These results demonstrate the promising potential of massively parallel computers for ocean acoustic propagation modeling and highlight the significant performance benefits of utilizing parallel computing techniques. Our findings emphasize the importance of efficient computational strategies.

An efficient parallel-computing method for modeling nonisothermal multiphase flow and multicomponent transport in porous and fractured media

With the rapid development of national economy, our demand for power resources has also increased rapidly. The traditional power grid can not be achieved operational status of equipment for effective monitoring and control, we will not be able to ensure the quality and quality of electricity service. Cloudbased build smart grid information platform enables gathering the parameter of grid operation state, greatly increasing the economic efficiency of power enterprises. Firstly, cloud computing and smart grid briefly described, and then pointed out that cloud computing applications in the smart grid information platform, and specialized of cloud-based smart grid information platform, in order for the power grid enterprises to enhance their level of service and provide effective reference. KEYWORD: cloud computing; smart power grids; information platform; Application 4th International Conference on Mechanical Materials and Manufacturing Engineering (MMME 2016) © 2016. The authors Published by Atlantis Press 202 3 THE MAIN PROBLEM IS STILL THE SMART GRID INFORMATION PLATFORM FACING The transition to a cloud computing data center. Our data center because of the different regions and the degree of information presents different characteristics, some data centers already have a basic cloud computing capabilities, but most data centers still remain in the initial level, to ensure the normal operation of the existing data center power under the premise, gradual transition to cloud computing data center in order to meet future requirements of the development of smart grid. Restricting data transmission problem. State Grid Corporation between provincial branches and there are a lot of data needs to be transmitted, will bring massive data transmission efficiency decreases, taking into account the load balancing device between the cloud node, thus building a wide area network channels to resolve data transmission problem. Security of data transmission. Data Center for Cloud Computing power companies to build internal private cloud, capable of self-maintenance and management, system wide area network data transmission services, according to the third security zone defined security partition, and install proprietary equipment production area and external public network isolation . 4 CLOUD COMPUTING APPLICATIONS IN THE SMART GRID INFORMATION POLICY PLATFORM Integrate and optimize the management of heterogeneous resources. Among the smart grid systems, data resources, information systems between various platforms and mutually dispersed, severe heterogeneity, thus leading to the integration of the quality of information sharing to increase difficulty, unable to effectively improve network management efficiency for the power distribution system electrical work quality management and marketing operations system maintenance upgrade will not be able to effectively meet demand (Jie Yang, 2014). Cloud computing technology enables the power system to effectively integrate the various business data and computing resources on the basis of information for business collaboration platform built on effective, able to satisfy the huge demand for smart grid information resource sharing. Cloud computing network virtualization and application virtualization technology, the server virtualization technology and other resources to achieve each service abstraction, and then realize the difference between the system and the software effectively eliminated by the appropriate method to achieve the effective integration of information resources, and then to optimize the management, greatly improving the efficiency of heterogeneous resources management applications. Distributed storage and management of large volume of data. After application of smart grid information platform construction, customer service requests varied, huge data volume information brought it to the storage and management of data provides a great challenge. Cloud computing technology in general face amount of data each smart grid system and unit collected using forms distributed storage of data to achieve the effective storage, greatly to ensure the security and reliability of data storage. GFS effective use of technology with each node, the master server is responsible for the effective management of the whole file system, the file system and complete the work of the metadata stored in the data block is the main server for data storage block sliced, and by saving three or more redundant backup storage. At the same time, client access and complete access to the main block server information. Fast Power System Parallel Computation and Treatment. Smart grids and information platform for the large volume of data to provide stable operation of the power grid data base, but more critical is the analysis and design of each system operating data, planning and decision-making, and even failures and transient stability, state estimation and intelligent a variety of computing and content analysis and decision-making process, its huge computing needs, general computing system can not reach its goal (Wang, Mengxue, 2015). Cloud computing technology through parallel computing technology has greatly enhanced parallel processing capability of the power system, which provides parallel programming model makes more simple and efficient parallel algorithms. MapReduce as Google made an important parallel programming model, and its role in large data sets in parallel computing plays a very prominent. Parallel processing job results by the client to achieve effective user submitted, via decomposition automated master role, the corresponding task scheduling to working nodes and nodes work completed by the specific calculations and perform the appropriate task. 5 SMART GRID INFORMATION PLATFORM CLOUD COMPUTING ANALYSIS Cloud Computing Smart Grid Information Platform Architecture. Smart grid technology information platform architecture cloud computing include the following several levels and aspects (Yan Cui, 2015): First, that the platform infrastructure layer is the integration of the various hardware resources and related management capabilities of virtualized using virtual technology to achieve the corresponding hardware devices such as computers and network storage devices abstraction, and then complete the automation and optimization of resource data and load management; secondly, the main platform for the integration of the various layers of software

Ray tracing is a well known technique to generate life-like images. Unfortunately, ray tracing complex scenes can require large amounts of CPU time and memory storage. Distributed memory parallel computers with large memory capacities and high processing speeds are ideal candidates to perform ray tracing. However, the computational cost of rendering pixels and patterns of data access cannot be predicted until runtime. To parallelize such an application efficiently on distributed memory parallel computers, the issues of database distribution, dynamic data management and dynamic load balancing must be addressed. In this paper, we present a parallel implementation of a ray tracing algorithm on the Intel Delta parallel computer. In our database distribution, a small fraction of database is duplicated on each processor, while the remaining part is evenly distributed among groups of processors. In the system, there are multiple copies of the entire database in the memory of groups of processors. Dynamic data management is acheived by an ALRU cache scheme which can exploit image coherence to reduce data movements in ray tracing consecutive pixels. We balance load among processors by distributing subimages to processors in a global fashion based on previous workload requests. The success of our implementation depends crucially on a number of parameters which are experimentally evaluated. © 1997 John Wiley & Sons, Ltd.

The application of parallel computers to reservoir simulation is studied with the aid of two recently introduced Multiple Instruction, Multiple Data (MIMD) computers. The problems posed by reservoir simulators are shown to be particularly well suited for such parallel computers* Fundamental parallel programming techniques using Fortran are presented for the Hetrogeneous Element Processor (HEP) produced by Denelcor and the iPSC Hypercube produced by Intel. These techniques are then applied to two of the most time consuming tasks in reservoir simulation: forming the matrix coefficients and producing a sparse matrix solution. The problem of parallel formation of sparse matrix coefficients is addressed for the single and multiphase cases using both black oil and compositional fluid models. Parallel sparse matrix solution is addressed by considering D4 ordered Gaussian elimination, and the multigrid, conjugate gradient, and Successive Over-Relaxation (SOR) methods. Three parallel algorithms based on SOR are developed to illustrate the possible diversity of parallel algorithms. The red-black and multicolored SOR methods which are often used on vector machines are shown to be easily adaptable to parallel MIMD machine. Matrix partitioning and a new method of iteration pipelining are also presented as parallel SOR methods. The various SOR algorithms, in parallel form, are mapped onto the HEP to illustrate the speedups currently possible on an MIMD machine. Projections are made on the future importance of these types of machines to reservoir simulation.

Problem statement: The parallelization of multiple progressive alignment algorithms is a difficult task. All known methods have strong bottlenecks resulting from synchronization delays. This is even more constraining in distributed memory systems, where message passing also delays the interprocess communication. Despite these drawbacks, parallel computing is becoming increasingly necessary to perform multiple sequence alignment. Approach: In this study, it is introduced a solution for parallelizing multiple progressive alignments in distributed memory systems that overcomes such delays. Results: The proposed approach uses threads to separate actual alignment from synchronization and communication. It also uses a different approach to schedule independent tasks. Conclusion/Recommendations: The approach was intensively tested, producing a performance remarkably better than a largely used algorithm. It is suggested that it can be applied to improve the performance of some multiple alignment tools, as CLUSTALW and MUSCLE.

PurposeA computational fluid dynamics code for the calculation of laminar hypersonic multi‐species gas flows in chemical non‐equilibrium in axisymmetric or two‐dimensional configuration on shared and distributed memory parallel computers is presented and validated. The code is designed to work efficiently in combination with an automatic domain decompositioning method developed to facilitate efficient parallel computations of various flow problems.Design/methodology/approachThe baseline implicit numerical method developed is the lower‐upper symmetric Gauss‐Seidel scheme, which is combined with a sub‐iteration scheme to achieve time‐accuracy up to third‐order. The spatial discretisation is based on Roe's flux‐difference splitting and various non‐linear flux limiters maintaining total‐variation diminishing properties and up to third‐order spatial accuracy in continuous regions of flow. The domain subdivision procedure is designed to work for single‐ and multi‐block domains without being constrained by the block boundaries, and an arbitrary number of processors used for the computation.FindingsThe code developed reproduces accurately various types of flows, e.g. flow over a flat plate, diffusive mixing and oscillating shock induced combustion around a projectile fired into premixed gas, and demonstrates close to linear scalability within limits of load imbalance.Research limitations/implicationsThe cases considered are axisymmetric or two‐dimensional, and assume laminar flow. An extension to three‐dimensional turbulent flows is left for future work.Originality/valueResults of a parallel computation, utilising a newly developed automatic domain subdivision procedure, for oscillating shock‐induced combustion around a projectile and various other cases are presented. The influence of entropy correction in Roe's flux‐difference splitting algorithm on diffusive mixing of multi‐species flows was examined.

Measurement and modelling of distributions of data communication times is commonly done for telecommunication networks, but this has not previously been done for message passing communications on parallel computers. We have used the MPIBench program to measure distributions of point-to-point MPI communication times for two different parallel computers, with a low-end Ethernet network and a high-end Quadrics network respectively. Here we present and discuss the results of efforts to fit the measured distributions with standard probability distribution functions such as exponential, lognormal, Erlang, gamma, Pearson 5 and Weibull distributions.KeywordsWeibull DistributionParallel ComputerParallel ProgramInterarrival TimeCommunication TimeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The production of realistic image generated by computer requires a huge amount of computation and a large memory capacity. The use of highly parallel computers allows this process to be performed faster. Distributed memory parallel computers (DMPCs), such as hypercubes or transputer-based machines, offer an attractive performance/cost ratio when the load balancing has been balance and the partition of the data domain has been performed. This paper presents a parallel ray tracing algorithm for DMPC using a Shared Virtual Memory (SVM) which solves these two classical problems. This algorithm has been implemented on a hypercube iPSC/2 and results are given.

Mapping Coriolis-coupled quantum dynamics onto parallel computer architectures

Parallel computing has become fundamental in processing the massive data volumes generated in modern science and industry. This paper presents a comprehensive survey and practical review of parallel computing in data processing, examining key frameworks (MPI, Open MP, CUDA, Hadoop MapReduce, Apache Spark, etc.), implementations, and real- world case studies. We discuss the architectures underlying shared-memory and distributed-memory systems and illustrate how parallelism on CPUs and GPUs is exploited for high- performance computing (HPC) and big data analytics. We review theoretical foundations (including Amdahl’s law for speedup limits) and compare programming models in terms of design, performance, and fault tolerance. A two-column IEEE-style for- mat is used to present critical insights from scholarly literature, with diagrams illustrating parallel architectures and workflows, and tabular summaries of tools and results. Case studies—from scientific HPC simulations to large-scale data analytics—highlight practical parallelism on multicore CPUs, GPUs, and computing clusters. We conclude with a discussion on the convergence of HPC and big data paradigms, the challenges of heterogeneous computing, and emerging trends. This survey serves as a reference for PhD researchers and practitioners seeking to understand frameworks and real-world practices in parallel computing for data processing.

The tiny encryption algorithm (TEA) is widely used when performing dissipative particle dynamics (DPD) calculations in parallel, usually on distributed memory systems. In this research, we reduced the computational cost of the TEA hash function and investigated the influence of the quality of the random numbers generated on the results of DPD calculations. It has already been established that the randomness, or quality, of the random numbers depend on the number of processes from internal functions such as SHIFT, XOR and ADD, which are commonly referred to as "rounds". Surprisingly, if we choose seed numbers from high entropy sources, with a minimum number of rounds, the quality of the random numbers generated is sufficient to successfully perform accurate DPD simulations. Although it is well known that using a minimal number of rounds is insufficient for generating high-quality random numbers, the combination of selecting good seed numbers and the robustness of DPD simulations means that we can reduce the random number generation cost without reducing the accuracy of the simulation results.

Parallel matrix computations in air pollution modelling