User Transparent Task Parallel Multimedia Content Analysis

This paper reports on the integration of parallel image processing in the ITK library and on improvements to the state-of-the-art of user transparency. In our approach, image processing tasks are wrapped into objects which are passed to the parallel engine. The engine is able to exploit data and task parallelism when executing the tasks on multicores, clusters and/or GPUs. All features necessary for efficient parallel processing are specified by the task objects. The engine can figure out most of the features itself, and is able to check the correctness of the features provided by the user. Interoperation optimization is attained by efficient scheduling of the tasks. The task dependency graph is automatically created at runtime. This is possible by delaying the execution of the tasks and by the intrinsic ITK pipeline updating mechanism. The low-level functions are also made available for the user, as well as a library-independent version.

The US Department of Energy (DOE) Exascale Computing Project (ECP) has fostered and strengthened the use of modern software engineering practices for developing applications and libraries, and this effort has resulted in the coordinated and interoperable E4S1 and xSDK2 ecosystems. Although this approach is cost-effective, it relies on robust programming systems and tools (PST) as the underlying foundation for our HPC software. At present, our primary PST stack consists of traditional high-performance computing (HPC) languages, namely Fortran, C, C++, and the popular Python language for data analysis and AI workflows. These languages support various programming frameworks and run-time abstractions that enable parallelism and concurrency across multiple node architectures and thousands of nodes through a variety of interconnect systems. However, to accommodate users’ diverse needs, certain aspects of the HPC ecosystem are delegated to vendor-specific or third-party implementations that extend beyond a particular scientific domain. This broader scope results in a multitude of specifications and variations, which leads to a complex orchestration of many-ecosystems. Unfortunately, this complexity in the ecosystem imposes additional overhead costs on consumers during the latter stages of the development cycle. In addition to the software ecosystem challenge, the upcoming conclusion of the ECP by December 2023 has raised significant concerns within the HPC programming systems community, from both the economic and social perspectives. The ECP has implemented a management structure for software development and funding decisions across all ECP participants by following a conventional hierarchical and centralized approach. However, this structure has prompted certain considerations within the community, particularly in anticipation of the Software Sustainability initiative by the DOE’s Advanced Scientific Computing Research Program (ASCR). For the success of this new initiative, it is of utmost importance to secure consistent funding and foster close engagement with researchers and core developers of existing programming-system products. This collaboration is vital to maintaining the critical capabilities of the current software during the transition phase while proactively adapting to future technology and workforce trends. The community recognizes the significance of adapting to emerging trends and is aware of the inherent fragility of the HPC software ecosystem, particularly in relation to programming systems that cater to all users. The ability to adapt and evolve is essential to staying relevant and effectively addressing these technical, economic, and social challenges. The S4PST team, which represents one of the six ASCR Software Sustainability seedling projects, is dedicated to tackling these challenges through community-based approaches that go beyond the scope of the DOE. This involves collaboration between national laboratories with academia, non-DOE institutions, hardware and system vendors, and international partners. By fostering these partnerships, we aim to create a robust and sustainable HPC software ecosystem that can effectively meet the needs of the community. This new community effort, driven by the eight DOE labs, will take on the responsibility of guiding funding decisions for programming-systems development and maintenance with transparency and consistency across all decisions. Additionally, the team will offer common technical services to the programming systems community, irrespective of their funding situations, and facilitate community-wide incubation to proactively nurture the software ecosystem. By actively engaging with stakeholders and employing a collaborative approach, we can collectively shape the future of programming systems and ensure a robust and thriving HPC software landscape. On May 11–12, 2023, the S4PST team conducted its inaugural kick-off workshop at the Innovative Computing Laboratory (ICL) in the University of Tennessee, Knoxville, hosted by Hartwig Anzt. The workshop encompassed various sessions dedicated to presentations and discussions, with the aim of comprehending the team members’ perspectives on the vision of software sustainability. Additionally, the workshop aimed to identify the technical, economic, and social requirements for sustaining the programming-systems community in the field of HPC. This report provides a summary of the S4PST effort by highlighting five major thrust areas discussed during the workshop: (i) community, (ii) technical support, (iii) training and diversity, (iv) verification, validation and correctness, and (v) emerging technologies. It also encompasses an overview of the presentations and discussions held throughout the event, our views and potential synergies with other seedling efforts, along with the outcomes and key takeaways from our initial discussions.

Solutions for Time-Critical Remote Sensing Applications The recent use of latest-generation sensors in airborne and satellite platforms is producing a nearly continual stream of high-dimensional data, which, in turn, is creating new processing challenges. To address the computational requirements of time-critical applications, researchers have begun incorporating high performance computing (HPC) models in remote sensing missions. High Performance Computing in Remote Sensing is one of the first volumes to explore state-of-the-art HPC techniques in the context of remote sensing problems. It focuses on the computational complexity of algorithms that are designed for parallel computing and processing. A Diverse Collection of Parallel Computing Techniques and Architectures The book first addresses key computing concepts and developments in remote sensing. It also covers application areas not necessarily related to remote sensing, such as multimedia andvideo processing. Each subsequent chapter illustrate s a specific parallel computing paradigm, including multiprocessor (cluster-based) systems, large-scale and heterogeneous networks of computers, grid computing platforms, and specialized hardware architectures for remotely sensed data analysis and interpretation. An Interdisciplinary Forum to Encourage Novel Ideas The extensive reviews of current and future developments combined with thoughtful perspectives on the potential challenges of adapting HPC paradigms to remote sensing problems will undoubtedly foster collaboration and development among many fields.

The field of high performance computing is undergoing a lot of changes. At the low end, supercomputers are becoming ever more affordable while, at the high end, it is becoming increasingly difficult to exploit the full potential of the most advanced hardware, given the sheer scale of the software challenges involved. There are also signs that the boundary between the worlds of the supercomputer and the desktop - traditionally very distinct - is starting to blur. Supercomputers continue to scale, with performance advancing at a rate that exceeds the rate of chip-level improvements, as characterized by Moore's law. These advances largely stem from the dramatic growth in cluster technology, which allows many processors to be closely coupled to solve large computational problems.

This paper presents the design and implementation of an energy saving daemon for clusters called cherub. The design of the cherub daemon is modular and extensible. In the field of High Performance Computing (HPC) well known Resource Management Systems (RMSs) like Portable Batch System (PBS) [8]or its open source derivative, the TORQUE resource manager [11]are used to manage clusters and work queues. Thus, cherub is able to interact with different RMSs to make them energy aware. cherub is also suited for load-balancing clusters managed by dispatchers like Linux Virtual Server (LVS) [17] since the only requirement is a central approach for resource management.

This article delves into the crucial roles of parallelism and asynchrony in the development of high-performance software programs. It provides an insightful exploration into how these methodologies enhance computing systems' efficiency and performance. The discussion begins with an overview of high-performance programs and their importance in modern computing, followed by a detailed analysis of parallelism, including its hardware and software aspects, various programming models, and diverse applications in fields like computational tasks, data processing, and machine learning. The article then shifts focus to asynchrony, elaborating on its implementation through callback functions, promises, and the async/await syntax. It highlights the significant benefits of asynchrony in improving program responsiveness and efficiency, especially in I/O-bound and network-intensive operations. A critical section of the article examines the combined application of parallelism and asynchrony in different domains, notably in Graphics Processing Unit (GPU) computations, distributed systems, and web development. This combination is shown to be particularly effective in handling Big Data, where rapid and efficient data processing is essential. The article concludes by emphasizing the indispensable nature of these approaches in modern software development. Parallelism and asynchrony are not merely performance enhancement tools but are fundamental in building innovative, resilient, and scalable software solutions. This comprehensive exploration underscores the ongoing relevance of these methodologies and their potential to drive further advancements in the field of high-performance computing.

Nonlinear numerical control is rightfully considered to be one of the most difficult engineering problems to tackle, in terms of both practical implementation and time-to-solution. It requires time-stepping numerical models for simulating the trajectory of the system, adjoint models for sensitivity analysis and matrix-free iterative solvers to produce the solution field of the inverse problem. The field of high-performance computing (HPC) provides computational tools and practices which enable the deployment of numerical applications on computational clusters, supercomputers and cloud computing facilities. This article presents a set of practical methods aimed at accelerating and parallelizing computation for sensitivity analysis in a large-scale 4D-Var data assimilation setting.

This research addresses the pressing need to optimize Python-based social science applications for high-performance computing (HPC)systems, emphasizing the combined use of task and data parallelism techniques. The paper delves into a substantial body of research,recognizing Python’s interpreted nature as a challenge for efficient social science data processing. The paper introduces a Pythonprogram that exemplifies the proposed methodology. This program uses task parallelism with multi-processing and data parallelismwith dask to optimize data processing workflows. It showcases how researchers can effectively manage large datasets and intricatecomputations on HPC systems. The research offers a comprehensive framework for optimizing Python-based social science applicationson HPC systems. It addresses the challenges of Python’s performance limitations, data-intensive processing, and memory efficiency.Incorporating insights from a rich literature survey, it equips researchers with valuable tools and strategies for enhancing the efficiencyof their social science applications in HPC environments.

In the scientific domain, extremely large amounts of data are generated by large-scale high performance computing (HPC) simulations. Storing and sending such vast volumes of data poses serious scalability and performance issues, which can be considerably mitigated by data compression techniques which significantly reduced storage size and data movement burdens. Since scientific data are being shared by scientists more and more frequently, data security methods that ensure the confidentiality, integrity, and availability of data itself are becoming increasingly important. As such, combing compression and encryption is critical to storing large-scale datasets securely. In this work, we explore how to integrate data compression and cryptography techniques as efficiently as possible for big scientific datasets in the HPC field. We perform thorough experiments using different scientific datasets with the state-of-the-art error-bounded lossy compressor - SZ - on a real-world supercomputing environment. Experiments verify that performing encryption before lossy compression (a.k.a., encr-cmpr method) may invalidate the advantage of compression algorithms. By contrast, executing encryption after lossy compression (a.k.a., cmpr-encr method) keeps not only high compression ratios but high overall execution speed. Experiments also verify that the encryption overhead under the cmpr-encr method decreases with increasing compression ratios, which means very good scalability.

This work focuses on possibilities of practical using public cloud in comparison with high-performance cluster for difficult calculations. A task dealing with problem of coloring the edges of a large set of cubic graphs and examining relations between these graphs was used as a criterion. In general, we dealt with couple of modifications and levels of difficulty of a given task. Assessing the convenience of usage nowadays, not only computer resources, is significant factor and phenomenon an environmental friendliness. In the world of informational technology and in the field of high-performance computing, is established the idea of green computing. Algorithms for effective usage of available computing resources are used to reach the aim within the high-performance computing that is hidden under the green computing. In the connection with service providers of public cloud computing is solving of securing datacenter power from renewable sources an added factor.

During the last decade, Heterogeneous systems are emerging for high performance computing [1]. In order to achieve high performance computing (HPC), existing technologies and programming models aims to see rapid growth toward intra-node parallelism [2]. The current high computational system and applications demand for a massive level of computation power. In last few years, Graphical processing unit (GPU) has been introduced an alternative of conventional CPU for highly parallel computing applications both for general purpose and graphic processing. Rather than using the traditional way of coding algorithms in serial by single CPU, many multithreading programming models has been introduced such as CUDA, OpenMP, and MPI to make parallel processing by using multicores. These parallel programming models are supportive to data driven multithreading (DDM) principle [3]. In this paper, we have presented performance based preliminary evaluation of these programming models and compared with the conventional single CPU serial processing system. We have implemented a massive computational operation for performance evaluation such as complex matrix multiplication operation. We used data driven multithreaded HPC system for performance evaluation and presented the results with a comprehensive analysis of these parallel programming models for HPC parallelism.

With the global trend of pursuing clean energy and decarbonization, power systems have been evolving in a fast pace that we have never seen in the history of electrification. This evolution makes the power system more dynamic and more distributed, with higher uncertainty. These new power system behaviors bring significant challenges in power system modeling and simulation as more data need to be analyzed for larger systems and more complex models to be solved in a shorter time period. The conventional computing approaches will not be sufficient for future power systems. This paper provides a historical review of computing for power system operation and planning, discusses technology advancements in high performance computing (HPC), and describes the drivers for employing HPC techniques. Some high performance computing application examples with different HPC techniques, including the latest quantum computing, are also presented to show how HPC techniques can help us be well prepared to meet the requirements of power system computing in a clean energy future.

Since the first general-purpose computing machines came up in the middle of the twentieth century, computer science's popularity has been growing steadily until our time. The first computers represented a significant leap forward in automating calculations, so that several theoretical methods could be taken from paper into practice. The continuous need for increased computing capacity made computers evolve and become more and more powerful. Nowadays, high-performance computing (HPC) is a crucial component of scientific and technological advancement. This book chapter introduces the field of HPC, covering key concepts and essential terminology to understand this complex and rapidly evolving area. The chapter begins with an overview of what HPC is and how it differs from conventional computing. It then explores the various components and configurations of supercomputers, including shared memory systems, distributed memory systems, and hybrid systems and the different programming models used in HPC, including message passing, shared memory, and data parallelism. Finally, the chapter discusses significant challenges and future directions in supercomputing. Overall, this chapter provides a comprehensive introduction to the world of HPC and is an essential resource for anyone interested in this fascinating field.

The transition towards net-zero emissions is inevitable for humanity’s future. Of all the sectors, electrical energy systems emit the most emissions. This urgently requires the witnessed accelerating technological landscape to transition towards an emission-free smart grid. It involves massive integration of intermittent wind and solar-powered resources into future power grids. Additionally, new paradigms such as large-scale integration of distributed resources into the grid, proliferation of Internet of Things (IoT) technologies, and electrification of different sectors are envisioned as essential enablers for a net-zero future. However, these changes will lead to unprecedented size, complexity and data of the planning and operation problems of future grids. It is thus important to discuss and consider High Performance Computing (HPC), parallel computing, and cloud computing prospects in any future electrical energy studies. This article recounts the dawn of parallel computation in power system studies, providing a thorough history and paradigm background for the reader, leading to the most impactful recent contributions. The reviews are split into Central Processing Unit (CPU) based, Graphical Processing Unit (GPU) based, and Cloud-based studies and smart grid applications. The state-of-the-art is also discussed, highlighting the issue of standardization and the future of the field. The reviewed papers are predominantly focused on classical imperishable electrical system problems. This indicates the need for further research on parallel and HPC approaches applied to future smarter grid challenges, particularly to the integration of renewable energy into the smart grid.

Recently, General Purpose Graphical Processing Units (GP-GPUs) have been identified as an intriguing technology to accelerate numerous data-parallel algorithms. Several GPU architectures and programming models are beginning to emerge and establish their niche in the High-Performance Computing (HPC) community. New massively parallel architectures such as the Nvidia’s Fermi and AMD/ATi’s Radeon pack tremendous computing power in their large number of multiprocessors. Their performance is unleashed using one of the two GP-GPU programming models: Compute Unified Device Architecture (CUDA) and Open Computing Language (OpenCL). Both of them offer constructs and features that have direct bearing on the application runtime performance. In this paper, we compare the two GP-GPU architectures and the two programming models using a two-level character recognition network. The two-level network is developed using four different Spiking Neural Network (SNN) models, each with different ratios of computation-to-communication requirements. To compare the architectures, we have chosen the two extremes of the SNN models for implementation of the aforementioned two-level network. An architectural performance comparison of the SNN application running on Nvidia’s Fermi and AMD/ATi’s Radeon is done using the OpenCL programming model exhausting all of the optimization strategies plausible for the two architectures. To compare the programming models, we implement the two-level network on Nvidia’s Tesla C2050 based on the Fermi architecture. We present a hierarchy of implementations, where we successively add optimization techniques associated with the two programming models. We then compare the two programming models at these different levels of implementation and also present the effect of the network size (problem size) on the performance. We report significant application speed-up, as high as 1095× for the most computation intensive SNN neuron model, against a serial implementation on the Intel Core 2 Quad host. A comprehensive study presented in this paper establishes connections between programming models, architectures and applications.