Power efficiency in high performance computing

For the High Performance Linpack (HPL) benchmark at the coming Exascale and beyond, silent errors like bit flipping in memory are expected to become inevitable. However, since bit flipping errors are difficult to be detected and located, their impact to the numerical correctness of HPL has not been evaluated thoroughly and quantitatively, while the impact at Exascale is especially susceptible. In this paper, an initial quantitative analysis of the impact of bit flipping errors to the numerical correctness of HPL has been investigated. To validate the numerical correctness of computed solution using HPL, there is a residual check after the approximate solution obtained. This paper has shown that in the case of only one bit flipping to any element in the original data matrix, if the flipped position is not the leading position of exponent, the residual check in HPL will almost surely pass at the scale of Exa flops and beyond. Experiments on modified HPL in single precision at small scales have verified the theoretical results in double precision at Exascale. The results obtained in this paper can provide a better understanding to the impact of bit flipping errors to numerical correctness of scientific computing applications.

Typically, a High Performance Computing (HPC) cluster loosely couples multiple Symmetric Multi-Processor (SMP) platforms into a single processing complex. Each SMP uses shared memory for its processors to communicate, whereas communication across SMPs goes through the intra-cluster interconnect. By analyzing the communication pattern of processes, it is possible to arrive at a mapping of processes to processors to ensure optimal communication paths for critical traffic. This critical traffic refers to the communication pattern of the program, which can be characterized by either frequency or size (or both) of the messages. To find an ideal mapping, it is imperative to understand the communication characteristics of the SMP memory system, intra-cluster interconnection, and the Message Passing Interface (MPI) program running on a cluster.Our approach is to study the ideal mapping for two classes of interconnects: 1) standard, high-volume Ethernet interconnects, and 2) proprietary, low-latency high-bandwidth interconnects. In the first installment of our work presented in this paper, we have focused on the ideal mapping for the first class.We configured a 16-node dual-processor cluster, interconnected with Fast Ethernet, Gigabit Ethernet, Giganet, and Myrinet. We used High Performance Linpack (HPL) benchmark to demonstrate that re-mapping of processes to processors (or changing the order of processors used) can affect the overall performance. The mappings are based on the HPL program analysis obtained from running a MPI profiling tool. Our results suggest that the performance of HPL using Fast Ethernet as the interconnect can be improved from 10% to 50% depending on the process mapping and the problem size. Conversely, an ad hoc mapping can adversely affect the cluster performance.

High-performance Linpack (HPL) benchmark is used to evaluate the performance of super computers. It implements blocked, right-looking Gaussian elimination with row partial pivoting. Block cyclic distribution is used in the parallel variant.In this paper, we implement the HPL in X10. X10 is a high level high-performance programming language based on the sequential semantics of Java augmented with new concurrency constructs, namely, places, asyncs, finish, atomic and clock. X10 also provides a rich array language which includes region, distributions and distributed arrays. We use HPL as a case study to evaluate the global view paradigm of X10. A global view program uses globally distributed arrays and global array indices without any process-specific code. We compare the elegance of global view programs over the local view version. We also show that the performance of global view program is no worse than the local view version.

Summary form only given. The effect of Intel/spl reg/ hyper-threading (HT) technology on a system's performance varies according to the characteristics of the application running on the system and the configuration of the system. High performance computing (HPC) clusters introduce additional variables, such as the math libraries used in solving linear algebra equations that can affect the performance of scientific and engineering applications in particular. We study the effect of HT on MPI-based applications by varying the math library. We configure an Intel-based HPC cluster and used the High Performance Linpack (HPL) benchmark to study the performance characteristics without changing hardware or communication parameters, but by linking the application with different math libraries. What we have found is that, even though the application or hardware parameters remain the same, HT may help or hinder the overall performance of the cluster depending on the computational efficiency of the mathematical functions.

Aiming at realizing a high performance computing (HPC) cloud with vector supercomputers, this paper presents the world’s first prototype of wide-area vector meta computing infrastructure named a vector computing cloud by virtualizing remote vector computing resources as an HPC service over the Internet. The prototype system consists of two remote NEC SX-9 nodes connected through a long fat-pipe network (LFN), and each node is located at Tohoku University and Osaka University with a distance of 800 km. The vector computing cloud also provides a single sign on environment and jobs are automatically assigned to appropriate sites. Wide-area co-operation of distributed vector supercomputers is realized by adopting the NAREGI Grid Middleware, and a virtual machine for NEC SX vector supercomputer series, job scheduling algorithms, and an MPI operating environment are newly developed to enhance the job and resource management capabilities of the NAREGI Grid Middleware. In addition, to achieve fairness and efficient job scheduling on the vector computing cloud, this paper presents a history-based job scheduling mechanism for a queue system. Based on the estimation, the job scheduling mechanism automatically allocates the job to an appropriate site, which can execute the job earlier. The operation tests and experiment results indicate that the prototype system realizes single sign on for multiple vector resources and has enough potential for transparently operating jobs between the two SX-9 systems. This paper also evaluates and discusses the performance of the proposed job scheduling mechanism and MPI operation between both SX-9 systems using the High Performance Linpack (HPL) benchmark.

Dense LU factorization has a high ratio of computation to communication and, as evidenced by the High Performance Linpack (HPL) benchmark, this property makes it scale well on most parallel machines. Nevertheless, the standard algorithm for this problem has non-trivial dependence patterns which limit parallelism, and local computations require large matrices in order to achieve good single processor performance. We present an alternative programming model for this type of problem, which combines UPC's global address space with lightweight multithreading. We introduce the concept of memory-constrained lookahead where the amount of concurrency managed by each processor is controlled by the amount of memory available. We implement novel techniques for steering the computation to optimize for high performance and demonstrate the scalability and portability of UPC with Teraflop level performance on some machines, comparing favourably to other state-of-the-art MPI codes.

The probability that a failure will occur before the end of the computation increases as the number of processors used in a high performance computing application increases. For long running applications using a large number of processors, it is essential that fault tolerance be used to prevent a total loss of all finished computations after a failure. While checkpointing has been very useful to tolerate failures for a long time, it often introduces a considerable overhead especially when applications modify a large amount of memory between checkpoints and the number of processors is large. In this paper, we propose an algorithm-based recovery scheme for the High Performance Linpack benchmark (which modifies a large amount of memory in each iteration) to tolerate fail-stop failures without checkpointing. It was proved by Huang and Abraham that a checksum added to a matrix will be maintained after the matrix is factored. We demonstrate that, for the right-looking LU factorization algorithm, the checksum is maintained at each step of the computation. Based on this checksum relationship maintained at each step in the middle of the computation, we demonstrate that fail-stop process failures in High Performance Linpack can be tolerated without checkpointing. Because no periodical checkpoint is necessary during computation and no roll-back is necessary during recovery, the proposed recovery scheme is highly scalable and has a good potential to scale to extreme scale computing and beyond. Experimental results on the supercomputer Jaguar demonstrate that the fault tolerance overhead introduced by the proposed recovery scheme is negligible.

This paper presents a description and the evaluation of the Netuno supercomputer, a high-performance cluster installed at Federal University of Rio de Janeiro in Brazil. The results for the High Performance Linpack (HPL) benchmark and two real applications are reported. Since building a high-performance cluster for running a wide range of applications is a non-trivial task, some lessons learned from assembling and operating this cluster, such as the excelent performance of the OpenMPI library, and the importance of the use an efficient parallel file system over the traditional NFS system, can be useful knowledge to support the design of new systems. Currently, Netuno is being heavily used to run large scale simulations in the areas of ocean modeling, meteorology, engineering, physics, and geophysics.

INTRODUCTION With the breakdown of Dennard’s scaling rule, chip power for a given area size is not constant as transistors shrink from generation to generation [1]. In particular, the power wall problem worsens with the continual improvement in performance of high-performance computers. The power challenge has been identified as the biggest research challenge [2]. Practical exascale systems of the future should meet a power constraint of 20 MW [2]. The Tianhe-2 system, the fastest supercomputer on the TOP500 list of June 2015, consumes 17.8 MW in running the high-performance Linpack (HPL) benchmark, achieving a performance of 33.86 Pflops. Compared with the Tianhe-2, an exascale system must obtain 30 times the performance improvement with 1.12 times the increase in power consumption. Clearly, in creating such exascale systems, satisfying such power requirements using evolutionary approaches is near impossible. Realizing this challenging power target depends on improvements in many technologies, especially, the viable and revolutionary technologies including new devices, innovative architectures and systems, and energy-aware intelligent technologies; see Figure 1.

The rising concern for power consumption of large-scale computer systems puts a research focus on the respective measurement methods. Varying workload patterns and energy efficiency optimizations cause highly dynamic power consumption on today’s compute nodes—a challenge for every measurement infrastructure. We identify five partly contradictory requirements that characterize such infrastructures: temporal granularity, spatial granularity, well-defined accuracy, scalability, and cost. In two projects we push the boundaries for these criteria: a scalable measurement solution for hundreds of nodes at millisecond granularity that is tightly integrated into the HPC system, and a sophisticated single-node instrumentation to measure the power consumption of application events in the microsecond range. Both measurement solutions are calibrated and their accuracy is carefully studied. We discuss scalable processing of the measurements for global monitoring in large-scale systems and use this data for energy efficiency analyses in combination with contextual information such as application performance trace data.

A system architecture with high-density general purpose graphic processing unit (GPGPU) is emerging as a promising solution that can offer high compute performance and performance-per-watt for building cluster supercomputers. The raw compute power of these heterogeneous systems greatly exceeds the current prevailing homogenous systems, motivating their rapid adoption. These heterogeneous systems do however increase the complexity of developing parallel applications and there is a need to investigate the compute performances and associated power consumption of common benchmarks and scientific computing applications. In this paper, we present the performance and power studies through using the Dell C4130 server that integrates up to 4 GPGPU cards and NVIDIA GPGPU K80 is used. The high performance Linpack (HPL) and molecular dynamics (MD) simulators including NAMD, LAMMPS and GROMACS are tested. Through comparing 4-K80 and 2-Xeon E5-2690 v3 systems, we show that: (1) for HPL tests, the 4- GPU server delivers up to 7 TFLOPS that is 9 times faster than the 2-CPU system and its power efficiency is 4 GFLOPS per Watt, (2) for MD tests, NAMD on 4-GPU server achieves 7.8 times speedup and it uses 2.3 times power consumption compared to 2-CPU system, and LAMMPS achieves 16 times speedup and it uses 2.6 times power consumption, and GROMACS achieves 3.3 times speed up and it uses 2.6 times power consumption. These preliminary results demonstrated that the novel high-density multi-GPGPU architecture offers high performances for computing intensive applications and molecular simulators with superior power efficiencies in a space efficient design. In future, such heterogeneous architecture could be a powerful alternative solution for next generation supercomputer systems.

On-site study was held to investigate the influences of load factor on power consumption and energy performance of water-cooled air conditioning (WCAC) system. A comprehensive measurement has been conducted to a WCAC plant of a supermall building located in West Java, Indonesia. Hourly and daily power consumption and performance characteristics of the WCAC system at varied load factors for a period of one year were evaluated. Regression analyses were developed to evaluate the influences of load factor on the WCAC power consumption and performance. The WCAC system was found to operate with varied load factor ranging from 70 percent to 100 percent. Power consumption of the system was strongly influenced by the load factor. Power consumption boosted when the power factor increased. The increase of load factor was also found to significantly improve the cooling capacity but reduce the coefficient of performance (COP). The best potential performance of the investigated WCAC system could be obtained at load factor between 70 percent and 85 percent.

Widely used benchmarks, such as High Performance Linpack (HPL), do not always provide direct insights are notoriously poor indicators of into the actual application performance of systems. When real applications are used, and there have been are criticisms indicating that the performance of simplified benchmarks such as HPL no longer strongly correlate to real application performance. In contrast, performance evaluations based on real or mini applications may give a direct estimation into application performance. The Sustained System Performance (SSP) metric, which is used to evaluate systems based on the performance at scale of various applications, has been successfully adopted to procure systems at the National Energy Research Scientific Computing Center (NERSC), the National Center for Supercomputing Applications (NCSA) and other facilities. However, significant effort is required to tune and optimize several mini applications for each of systems. In this paper, we propose a new performance metric — the Simplified Sustained System Performance (SSSP) metric — based on a suite of simple benchmarks, which enables performance projection that correlates with full applications, but use onto a suite of mini applications. While the SSP metric is calculated over a set of applications, the SSSP metric applies its methodology to a set of benchmarks. Preliminary weighting factors for benchmarks are introduced to approximate the original SSP metric more accurately by the SSSP metric. To define the weighting factors, we perform a simple learning algorithm. Our preliminary experiments show that even though our metric is still easy to measure because it is based on a combination of simple benchmarks, it can provide projections of the performance of applications.

Comparable to time performance models, it is now possible to estimate performance based upon energy consumption for HPC systems. The predictive ability of the analytical modeling is an interesting feature that motivates us to approach this methodology for the case of energy consumption. In this paper, we present an analytical model for predicting the energy consumption for the High Performance Linpack (HPL). The derived model can be used to know in advance the energy consumed by the HPL over a target architecture, and can be integrated into the schedulers of operating systems or queue managers. We established an experimental setup using a standard metered PDU that allowed us to measure the energy consumption for the HPL benchmark on our cluster. With the monitoring system in place, we can obtain the architectural and algorithmic parameters associated for both performance and energy analytical models. Also this has made possible watts and gflops-per-watt prediction when we execute Linpack executions with concrete algorithm parameters in our cluster.

This paper focuses on theoretical analysis, computational test and optimization of High Performance Computing Cluster (HPCC). Also known as Data Analytics Supercomputer (DAS), HPCC is built on Intel Many Integrated Core (MIC) for High Performance Linpack (HPL) test. Initially, a platform is configured by 5 nodes with an Intel® Xeon Phi <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">™</sup> Coprocessor 31S1P per node to analyze power consumption as well as parallel level of Intel MIC accelerator. Compiling with Message Passing Interface (MPI) library and Math Kernel Library (MKL), the "Make" file is modified and debugged by adjusting parameters in hpccinf.txt. According to multiple Infiniband nodes evaluation, the libhpl library of Intel is employed and value of NB is set to be 960 for a single node with one MIC while debugging. Moreover, this paper optimized the algorithm on Double Precision General Matrix Multiplication (DGEMM) test and PTRANS to acquire efficient, precise and truncated working time duration. From HPL and Gridding Program experimental results, it's clear that the theoretical analysis and experiments were performed successfully.