Current trends in high performance parallel and distributed computing

This edited book aims to present the state of the art in research and development of the convergence of high-performance computing and parallel programming for various engineering and scientific applications. The book has consolidated algorithms, techniques, and methodologies to bridge the gap between the theoretical foundations of academia and implementation for research, which might be used in business and other real-time applications in the future.The book outlines techniques and tools used for emergent areas and domains, which include acceleration of large-scale electronic structure simulations with heterogeneous parallel computing, characterizing power and energy efficiency of a data-centric high-performance computing runtime and applications, security applications of GPUs, parallel implementation of multiprocessors on MPI using FDTD, particle-based fused rendering, design and implementation of particle systems for mesh-free methods with high performance, and evolving topics on heterogeneous computing. In the coming days the need to converge HPC, IoT, cloud-based applications will be felt and this volume tries to bridge that gap.

Water quality assessment and prediction of Lake Michigan are becoming major challenges in Northwest Indiana, USA. Traditionally, mechanistic simulation models are employed for water quality modeling and prediction. However, given the complicate nature of Lake Michigan in Northwestern Indiana, the detailed simulation model is extremely simple in comparison and, at some point, additional detail exceeds our ability to simulate and predict with reasonable error levels. In this regard, my project applied data mining technologies, as an innovative alternative, to develop an easy and more accurate approach for water quality assessment and prediction. The drawback of the data mining modeling is that the execution takes quite long time, especially when we employ a better accuracy but more time consuming algorithm in clustering. Therefore, we applied the High Performance Computing System of the Northwest Indiana Computational Grid to deal with this problem. Up to now, the pilot experiments have achieved very promising preliminary results. The visualized water quality assessment and prediction obtained from this project would be published in an interactive website so that the public and the environmental managers could use the information for their decision making.

An architecture of high performance computer,called Hyper Parallel Processing(HPP),is proposed to satisfy the requirements of both High Performance Computing,and Utility Computing which will be the application model of data centers.HPP combines the benefits of the scalability of MPP,the communication efficiency of DSM,as well as the commodity of cluster.Comparisons of current main-stream high performance computer architectures show that none of them can satisfy both HPC and utility computing.The main features of HPP are Global Address Space(GAS) and Hyper Node with single Operating System image.HPP supports the distributed global address space including both memory and I/O,but without hardware cache coherence.A Hyper Node consists of a set of application specific CPUs and a(or more) OS specific CPU.The OS CPU maintains the single system image,while the application CPUs run only lightweight run-time software.Besides the GAS interconnect network for applications,a standard SAN connects all OS CPUs and I/O devices providing system management and storage service.HPP is able to provide many opportunities of innovative research in High Performance and Utility Computing areas,including communication,synchronization,programming model,node operating system,utility computing,fault isolation,CPU and system etc.According to HPP architecture,a prototype system of Dawning5000 HPC is implemented and the feasibility of HPP is proved.

Recently, more and more researches and applications exploit grid computing systems to deal with high performance computing. However, the mass data transmissions across different grid sites affect overall computing performance. Therefore, grid systems start to integrate with the P2P technology to support the high performance distributed computing. The new distributed computing system is named the P2P Grid computing system. Although the P2P Grid computing system combines the advantages of the grid computing system and the P2P technology, some issues are still needed to be solved. For example, the highly variable resource usage and the heterogeneity of resources could intensely affect the P2P Grid system performance. In this case, the computing performance depends on the resource management policy. Therefore, this study proposes a distributed dynamic load balance policy to manage resources more effectively and to further improve the resource utilization. The prototype is implemented on the sites of the Taiwan Uni Grid, and the P2P grid sites exchange information by JXTA advertisements. Experimental results show that the proposed algorithm could efficiently distribute the workload for execution, that is, it not only can minimize the job execution time, but also maximize the resource utilization.

For decades, HPC has established itself as an essential tool for discoveries, innovations and new insights in science, research and development, engineering and business across a wide range of application areas in academia and industry. Today High-Performance Computing is also well recognized to be of strategic and economic value – HPC matters and is transforming industries. This article will discuss new emerging technologies that are being developed for all areas of HPC: compute/processing, memory and storage, interconnect fabric, I/O and software to address the ongoing challenges in HPC such as balanced architecture, energy efficient high-performance, density, reliability, sustainability, and last but not least ease-of-use. Of specific interest are the challenges and opportunities for the next frontier in HPC envisioned around the 2020 timeframe: ExaFlops computing. We will also outline the new and emerging area of High Performance Data Analytics, Big Data Analytics using HPC, and discuss the emerging new delivery mechanism for HPC - HPC in the Cloud.

The theoretical concept of technological performance has emerged only recently with the publication of Jon McKenzie's Perform or Else: From Discipline to Performance in 2001. McKenzie develops a general theory of performance based around the development of three performance paradigms: cultural performance, organizational performance (or performance management), and technological performance. In his examination of the techno-performance paradigm, he focuses primarily on late twentieth and early twenty-first century 'high performance' technologies such as computers, guided missiles and space shuttles. While he acknowledges that the concept of performance does not apply only to technologies in this period, his analysis implicitly suggests that high performance technologies are a unique invention of the modern age. This essay confronts McKenzie's restriction of techno-performance to the post-WWII period by demonstrating how technologies performed and were seen to perform in the late nineteenth century through a techno-performance reading of Bram Stoker's Dracula (1897). The late Victorian period saw startling innovations in information and communication technology (such as the overseas telegraph, the typewriter, and the gramophone), which were marketed as high performance technologies, though not in those words. To contextualise my reading of Dracula , I examine contemporary Victorian advertisements for communication technologies to demonstrate how such technologies were viewed as high performance products by Victorian advertisers and consumers. Technology in Dracula has usually been read as a metaphor. I employ McKenzie's concept of techno-performance to examine the performative functions of technology in Dracula that have not yet been explored by Victorianist scholarship. McKenzie notes two challenges posed by techno-performance: first, the challenge posed by a developer to his/her technological product, to perform or be classified as obsolete; and, second, the challenge posed by technology to its user to perform or be regarded as outmoded. I argue that Stoker's Dracula takes up both of these challenges. Emergent technologies sometimes perform in unexpected and potentially disruptive ways, much like the space shuttle Challenger cited by McKenzie; at the same time, such technologies oblige their users to perform in unexpected and disruptive ways. This essay examines emergent technologies in Dracula to highlight the relationship between individual and technological performance in the late nineteenth century.

Grid computing originated as paradigm for high performance computing and massive parallel processing. Currently, Grid systems have become an important paradigm for efficiently solving large scale complex problems from many fields. On the other hand, P2P paradigm originated from file sharing but each time more is being used for the development of large scale distributed platforms. Grid and P2P systems have thus followed different trajectories pushed by different motivations, needs and research communities. Yet, both paradigms are evolving in a way that each time more they are sharing common characteristics and are mutually benefiting from their best features. Among these characteristics, we could distinguish the cooperative model for solving complex problems by exploiting the large computing capacity offered by the nodes of the system altogether. As such, Grid and P2P systems have achieved notable success, in particular, for e-Science applications, a family of complex applications arising in science and engineering that need considerable computation power.

Parallelizing serial software systems in order to run in a High Performance Computing (HPC) environment presents many challenges to developers. In particular, the extant literature suggests the task of decomposing large-scale data applications is particularly complex and time-consuming. In order to take stock of the state of practice of data decomposition in HPC, we conducted a two-phased study. Firstly, using focus group methodology we conducted an exploratory study at a software laboratory with an established track record in HPC. Based on the findings of this first phase, we designed a survey to assess the state of practice among experts in this field around the world. Our study shows that approximately 75% of parallelized applications use some form of data decomposition. Furthermore, data decomposition was found to be the most challenging phase in the parallelization process, consuming approximately 40% of the total time. A key finding of our study is that experts do not use any of the available tools and formal representations, and in fact, are not aware of them. We discuss why existing tools have not been adopted in industry and based on our findings, provide a number of recommendations for future tool support.

The Service Oriented Architecture (SOA) is embraced in distributed and grid computing to produce high performance results for long time. SOA is likened by application programmers for its trademark characteristics of programmability, efficiency in heterogeneous conditions and fault-tolerance. It has worked well for high performance financial applications. but not for scientific applications which are too fine grained and communication intensive to be efficient on distributed environments. This paper argues that to make SOA model work well for those scientific applications, we need to reduce overhead costs associated with smaller task loads arising from finer granularity and increased communications in those applications. This paper proposes a data service to be used along with the existing compute services in SOA middlewares to enable inter-communication of finer tasks with out loosing SOA properties of programmability and efficiency under heterogeneity. This data service shall better enable high performance scientific computing of medium to fine grained scientific applications.

R is a robust open-source programming language mainly used for statistical computing . Many areas of statistical research are experiencing rapid growth in the size of data sets. Methodological advances drive increased use of simulations. A common approach is to use parallel/concurrent computing. This paper presents an overview of techniques for parallel computing with R on ACI (a PSU Infrastructure) and benchmark it with C/C++. We review the scalabilty concern of R, and look at the simplicity of using R as a primary language in Coding for HPC. We will look at the various R packages for HPC like Rmpi, Rcpp, snow and snowfall. We utilize a series of algorithms to benchmark and will illustrate each benchmark with a representative graph for ease of understanding. The paper concludes with a better understanding of which language to use when in high performance computing .

The transition towards net-zero emissions is inevitable for humanity’s future. Of all the sectors, electrical energy systems emit the most emissions. This urgently requires the witnessed accelerating technological landscape to transition towards an emission-free smart grid. It involves massive integration of intermittent wind and solar-powered resources into future power grids. Additionally, new paradigms such as large-scale integration of distributed resources into the grid, proliferation of Internet of Things (IoT) technologies, and electrification of different sectors are envisioned as essential enablers for a net-zero future. However, these changes will lead to unprecedented size, complexity and data of the planning and operation problems of future grids. It is thus important to discuss and consider High Performance Computing (HPC), parallel computing, and cloud computing prospects in any future electrical energy studies. This article recounts the dawn of parallel computation in power system studies, providing a thorough history and paradigm background for the reader, leading to the most impactful recent contributions. The reviews are split into Central Processing Unit (CPU) based, Graphical Processing Unit (GPU) based, and Cloud-based studies and smart grid applications. The state-of-the-art is also discussed, highlighting the issue of standardization and the future of the field. The reviewed papers are predominantly focused on classical imperishable electrical system problems. This indicates the need for further research on parallel and HPC approaches applied to future smarter grid challenges, particularly to the integration of renewable energy into the smart grid.

Dominant trends in parallel and distributed computing (parallel computers, parallel algorithms) aim to connect a number of powerful workstations (WS) based on single personal computers (PC) or symmetrical multiprocessor system (SMP) to solve complex problems in a parallel way. Typical representatives of these trends are network of workstation (NOW) and Grid as a high integrated network of NOW modules. In relation to it there is a bad need of unified modelling in traditionally evolved parallel computing and distributed computing as two separate research disciplines. Likewise current trends in high performance computing (HPC) aim to use NOW modules based on SMP parallel computers as cheaper alternatives to the traditionally used massive parallel multiprocessors or supercomputers and to profit from unifying modelling in parallel and distributed computing too. To exploit the parallel processing capability of such cluster, any application program must be paralleled. A way how to do it (decomposition model) belongs to a most important step in developing process of effective parallel algorithm. This article at first discusses the development of effective (optimised) iterative parallel algorithms (IPA) to solve partial differential equations (PDE). By carefully choosing an illustrative typical problem of PDE (Laplace equation) the proposed research has demonstrated the most important complexities in the process of complete modelling of substantial complexities and their optimisation in order to develop an effective PA. Such approach could be used as illustrative example for a complete modelling and effective solution of even other more complex PDE´s in a parallel way.

We present a novel methodology for integration of high-angle/horizontal (HA/HZ) well data into 3D geomodels as a natural extension to well placement workflows. Log interpretation, typically done in 2D cross-sections, is based on 1-D automated inversion, yielding near-wellbore reservoir structure and properties. 2D cross-section, updated by inversion and further refined using 2D/3D modeling, is subsequently retrofitted into the geomodel so as to minimally perturb the original topology. Changes in positions/dips/azimuths of boundaries and faults, cell properties, and further local grid refinements are applied automatically. The updated geomodel honors high-resolution logs and low-resolution seismic/nearby wells data. We demonstrate this on a typical real-time well placement scenario. Electromagnetic log interpretation codes are integrated as a high performance computing (HPC) Web service into a geosteering/model update workflow. As the initial model, we use 3D geomodel constructed from seismic/vertical well data. A "curtain" cross-section is extracted, edited based on 1D inversion, and further refined to match HA/HZ logs through 2D and 3D forward modeling, by changing properties/dips/layer thicknesses /fault positions, while preserving the original topology. Then, updated node coordinates and cell properties of the affected pillar grid region are calculated to optimally retrofit the changed 2D cross-section into the grid. These changes are then automatically applied to the 3D model. For quality control, we recompute the 2D cross-section from the refined geomodel. Ultimately, we arrive at the geomodel that honors both seismic and resistivity well-log data. The combination, in a single workflow, of physics-based log modeling codes, Services-Oriented Architecture, HPC framework, and the solver to optimally retrofit 2D cross-sections into 3D models, creates a qualitatively new opportunity for well placement engineers. This integrated workflow (1) maximizes the value of deep directional resistivity well-logs and real-time well placement interpretation by incorporating them into the source of data for building geomodels; (2) radically speeds up the model refinement loop by automatically calculating and applying the modifications to 3D reservoir model; (3) enables geoscientists to directly refine geomodels while geosteering. The latter has not been a standard practice, hindered by challenges of scale difference between geomodels and well-logs and lack of availability of efficient modeling codes.

In the fast-evolving software market, the demand for rapid development of digital solutions, especially prototypes and MVPs, is increasing. Creating a functional backend requires skilled developers, adding costs and extending timelines, even for early-stage projects. This underscores the need for automated backend deployment tools, especially for startups and smaller teams. Building RESTful APIs, essential for most software products, demands expertise in server architecture, database management, and security. Current technology offers partial solutions through tools like Node.js, RethinkDB, Nginx, and JWT for secure data exchange, yet integrating these into a cohesive, low-maintenance system that adapts dynamically to client-side demands remains challenging. This research proposes an automated backend software solution that enables rapid, low-cost server deployment tailored for MVPs and scalable applications. The system allows users to specify only database and table names; fields are dynamically created and indexed upon receiving POST requests, ensuring high performance. An integrated React-based administrative panel facilitates user management and secure, multi-client access, enabling flexible, multi-tenant usage on a single infrastructure. This solution minimizes backend development costs, optimizes database performance, and lays the groundwork for future automated backend solutions that accelerate product deployment. Formulation of the problem. Modern development demands efficient creation of RESTful backend services with minimal time and resource investment, especially for prototypes and MVPs. Companies and teams often need to hire skilled developers even for basic server logic, increasing development costs. Automating backend infrastructure deployment has become crucial to expedite product release and optimize expenses. Target. To create automated software for backend server development that enables users to easily deploy REST API services. The system should include adaptive database and table structure creation, automatic data indexing for high performance, and an administrative panel for managing requests and databases. Results. A server based on Node.js and RethinkDB was developed, supporting JWT authentication and integrated with an administrative panel built with React and React-Admin. Users only need to specify the database and table names, and the server then automatically adapts and indexes fields upon receiving POST requests, ensuring high query processing performance. With automatic indexing and flexible table handling, this system is suitable for both MVPs and high-load products. Practical significance. This automation system significantly reduces the time and costs required to establish backend infrastructure, making it valuable for small teams and startups. Its high performance and flexibility provide an effective solution for long-term operation under high loads, giving users a universal and powerful tool for backend application deployment.

Although grid computing has become popular in scientific research and the enterprise domain, there is no way a personal user who does not belong to any organization can construct or access the grid system for solving her problems. The main goal is to propose a lightweight personal grid that is self-organized without complicated configuration and infrastructural support like central servers and self-managed with the cooperation of participants. The key component of our system is the supernode network designed for autonomously and efficiently constructing the grid system. Based on the supernode network, our personal grid system forms a virtual organization (VO) with nodes having physical proximity. Therefore, communication latency between nodes in a VO can be minimized. Moreover, hierarchical scheduling is adopted using the two-level structure of the supernode network and new service architecture is proposed to improve the reusability of applications.