TriC: Distributed-memory Triangle Counting by Exploiting the Graph Structure

The Complex neuronal network models require support from simulation environment for efficient network simulations. To compute the models increasing complexity necessitated the efforts to parallelize the NEURON simulation environment. The computational neuroscientists have extended NEURON by dividing the equations for its subnet among multiple processors for increasing the competence of hardware. For spiking neuronal networks inter-processor spikes exchange consume significant portion of overall simulation time on parallel machines. In NEURON Message Passing Interface (MPI) is used for inter processor spikes exchange, MPI_Allgather collective operation is used for spikes exchange generated after each interval across distributed memory systems. However, as the number of processors become larger and larger MPI_Allgather method become bottleneck and needs efficient exchange method to reduce the spike exchange time. This work has improved MPI_Allgather method to Remote Memory Access (RMA) based on MPI-3.0 for NEURON simulation environment, MPI based on RMA provides significant advantages through increased communication concurrency in consequence enhances efficiency of NEURON and scaling the overall run time for the simulation of large network models.1

As distributed memory systems have become common, the de facto standard for communication is still the Message Passing Interface (MPI). pupyMPI is a pure Python implementation of a broad subset of the MPI 1.3 specifications that allows Python programmers to utilize multiple CPUs with datatypes and memory handled transparently. pupyMPI also implements a few non-standard extensions such as non-blocking collectives and the option of suspending, migrating and resuming the distributed computation of a pupyMPI program. This paper introduces pupyMPI and presents benchmarks against C implementations of MPI, which show acceptable performance.

OpenMP vs. MPI on a shared memory multiprocessor

Triangles are the most basic non-trivial subgraphs. Triangle counting is used in a number of different applications, including social network mining, cyber security, and spam detection. In general, triangle counting algorithms are readily parallelizable, but when implemented in distributed, shared-memory, their performance is poor due to high communication, imbalance of work, and the difficulty of exploiting locality available in shared memory. In this paper, we discuss four different (but related) triangle counting algorithms and how their performance can be improved in distributed, shared-memory by reducing in-node load imbalance, improving cache utilization, minimizing network overhead, and minimizing algorithmic work. We generalize the four different triangle counting algorithms into a common framework and show that for all four algorithms the in-node load imbalance can be minimized while utilizing caches by partitioning work into blocks of vertices, the network overhead can be minimized by aggregation of blocks of work, and algorithm work can be reduced by partitioning vertex neighbors by degree.We experimentally evaluate the weak and the strong scaling performance of the proposed algorithms with two types of synthetic graph inputs and three real-world graph inputs. We also compare the performance of our implementations with the distributed, shared-memory triangle counting algorithms available in PowerGraph-GraphLab and show that our proposed algorithms outperform those algorithms, both in terms of space and time.

A variety of real-world data and networks can be described by a heavy-tailed probability distribution of its values, vertex degrees, or other significant properties, that follows the power law. Such a scale-free data and networks can be found in both natural phenomena such as protein interaction networks and gene regulation networks and man-made structures like the Internet, language, and various social networks. An efficient analysis of large scale data and networks is often impractical and various heuristic and metaheuristc sampling techniques are deployed to select smaller subsets of the data for analysis and visualisation. A key goal of data and network sampling is to select such a subset of the original data that would accurately represent the original data with respect to selected attributes. In this work we propose a novel genetic algorithm for scale-free data and network sampling and evaluate the algorithm in a series of computational experiments.

Message passing interface (MPI) is the de facto standard in writing parallel scientific applications on distributed memory systems. Performance prediction of MPI programs on current or future parallel systems can help to find system bottleneck or optimize programs. To effectively analyze and predict performance of a large and complex MPI program, an efficient and accurate communication model is highly needed. A series of communication models have been proposed, such as the LogP model family, which assume that the sending overhead, message transmission, and receiving overhead of a communication is not overlapped and there is a maximum overlap degree between computation and communication. However, this assumption does not always hold for MPI programs because either sending or receiving overhead introduced by MPI implementations can decrease potential overlap for large messages. In this paper, we present a new communication model, named LogGPO, which captures the potential overlap between computation with communication of MPI programs. We design and implement a trace-driven simulator to verify the LogGPO model by predicting performance of point-to-point communication and two real applications CG and Sweep3D. The average prediction errors of LogGPO model are 2.4% and 2.0% for these two applications respectively, while the average prediction errors of LogGP model are 38.3% and 9.1% respectively.

Current trends in HPC (high performance computing) suggest that clusters will soon consist with hundreds, if not thousands, processors and the size of current scientific problems becomes much larger than before. Many researchers have predicted that the communication among these processors has dominated the execution time of the scientific parallel applications. Users will need well understanding on communication patterns among scientific parallel applications and their similarities so that users benefit not only from cost saving on constructing the running environment for these applications but also from obtaining better performance. In this paper, we address the communication pattern matching, and focus on point-to-point communication, which is primarily utilized (over 90% all MPI (message passing interface) calls) in most MPI codes and has much more impact on the communication performance than collective communication does. In this work, our contribution is that we propose a new approach MPACP (matching of parallel application communication patterns) to automate the analysis of the similarity between two parallel applications and provide a reliable report which will help users or developers understand the similarity among communication patterns of parallel applications. Furthermore, experimental results demonstrate the effective performance of our scheme in terms of the automatic matching of parallel application communication patterns.

Partitioned Global Address Space (PGAS) parallel programming models can provide an efficient mechanism for managing shared data stored across multiple nodes in a distributed memory system. However, these models are traditionally directly addressed and, for applications with loosely-structured or sparse data, determining the location of a given data element within a PGAS can incur significant overheads. Applications incur additional overhead from the network latency of lookups from remote location resolution structures. Further, for large data, caching such structures locally incurs space and coherence overheads that can limit scaling. We observe that the matching structures used by implementations of the Message Passing Interface (MPI) establish a separation between incoming data writes and the location where data will be stored. In this work, we investigate extending such structures to add a layer of indirection between incoming data reads and the location from which data will be read, effectively extending PGAS models with logical addressing.

A new class of normalized explicit approximate inverse matrix techniques, based on normalized approximate factorization procedures, for solving sparse linear systems resulting from the finite difference discretization of partial differential equations in three space variables are introduced. A new parallel normalized explicit preconditioned conjugate gradient square method in conjunction with normalized approximate inverse matrix techniques for solving efficiently sparse linear systems on distributed memory systems, using Message Passing Interface (MPI) communication library, is also presented along with theoretical estimates on speedups and efficiency. The implementation and performance on a distributed memory MIMD machine, using Message Passing Interface (MPI) is also investigated. Applications on characteristic initial/boundary value problems in three dimensions are discussed and numerical results are given.

The 64-bit OS provides ample memory address space that is beneficial for applications using a large amount of data. This paper proposes using a cluster as a memory resource for sequential applications requiring a large amount of memory. This system is an extension of our previously proposed socket-based distributed large memory system (DLM), which offers large virtual memory by using remote memory distributed over nodes in a cluster. The newly designed DLM is based on MPI (message passing interface) to exploit higher portability. MPI-based DLM provides fast and large virtual memory on widely available open clusters managed with an MPI batch queuing system. To access this remote memory, we rely on swap protocols adequate for MPI thread support levels. In experiments, we confirmed that it achieves 493 MB/s and 613 MB/s of remote memory bandwidth with the STREAM benchmark on 2.5 GB/s and 5 GB/s links (Myri-10G x2, x4) and high performance of applications with NPB and Himeno benchmarks. Additionally, this system enables users unfamiliar with parallel programming to use a cluster.

With processor speeds no longer doubling every 18—24 months owing to the exponential increase in power consumption and heat dissipation, modern high-end computing systems tend to rely less on the performance of single processing units and instead rely on achieving high performance by using the parallelism of a massive number of low-frequency/low-power processing cores. Using such low-frequency cores, however, puts a premium on end-host pre- and post-communication processing required within communication stacks, such as the Message Passing Interface (MPI) implementation. Similarly, small amounts of serialization within the communication stack that were acceptable on small/medium systems can be brutal on massively parallel systems. Thus, in this paper, we study the different non-data-communication overheads within the MPI implementation on the IBM Blue Gene/P system. Specifically, we analyze various aspects of MPI, including the MPI stack overhead itself, overhead of allocating and queueing requests, queue searches within the MPI stack, multi-request operations, and various others. Our experiments, that scale up to 131,072 cores of the largest Blue Gene/P system in the world (80% of the total system size), reveal several insights into overheads in the MPI stack, which were not previously considered significant, but can have a substantial impact on such massive systems.

In this paper, we propose a parallel implementation of the branch-and- bound optimization technique on distributed-memory systems using Message Passing Interface (MPI). We employ parallel branch-and-bound to accelerate a real-world example application: optimal selection of production equipment for multi-product batch plants. We describe the master-worker organization of our parallel algorithm: a single master process dispatches a subset of computations to multiple worker processes and gathers computed results from them. For exchanging messages between master and worker we use MPI's point-to-point communication functions. We report experimental results about the speedup and efficiency of our parallel implementation. We observe that the master process may become a bottleneck for the overall application performance if it controls too many workers processes, because of the increasing communication overhead.

Objective: Due to the recent development of multiple parallel programming tools with varying features, it is difficult to choose the best tool according to the needs of the user. Methods: This problem is addressed by making a comparative analysis study of different features like license type, source code availability, targeted platforms and languages supportedby these diverse tools. There are different parallel programming languages that support the present multi-core architecture like Message Passing Interface (MPI) and Open Multi-Processing (OpenMP). These are widely used to provide different performance characteristics of parallelism in different test cases. The new architecture and the complexity strengthens the need to monitor and analyze the performance of the various OpenMP kernels and constructs on multi-core processors. Findings: There are many papers that have been published in the past but non of them focuses on a comparative study among the performance analysis tools (PATs) that we mostly opt for. This paper intends to analyze the parallel computing ability of OpenMP and MPI, besides helping the user to understand which tool suites his task the best. Improvement: This study shows that MPI offers the best performance characteristics in the field of shared memory programming whereas OpenMP is a better choice because of the global style of the resulting program. It also provides a roadmap to select the best tool when designing a parallel programming system.

The demand of processing speed in modern digital signal processing is huge and requires more computational power than a single processor can provide. To address these demands, distributed and parallel processing in embedded systems is becoming a requirement. So it is essential to do research on algorithm and the parallel programming in the parallel embedded system. MPI (message passing interface) is one of the most important parallel programming standards. Porting MPI to the embedded system will enhance application portability and make parallel programming much easier. This paper presents a novel design and implementation of MPI based on link DMA for DSP-based systems. Through computation the value of and PingPong, the results show that the implementation is suitable for parallel programming, especially suitable for large volume of data.

Achieving high-speed network I/O on distributed-memory systems is difficult because their architecture is in general ill-suited for communication processing. Some of the common problems are: inability to do protocol processing, inefficient handling of data distribution, and poor management of the I/O. In this paper we present an I/O architecture that addresses these problems and supports high-speed network I/O on distributed-memory systems. The key to good performance is to partition the work appropriately between the system and the network interface. We perform some communication tasks on the distributed-memory parallel system since it is more powerful, and less likely to become a bottleneck than the network interface. Tasks that do not parallelize well are performed on the network interface and hardware support is provided for the most time-critical operations. We emphasize the use of simple I/O mechanisms that can be used by programming tools that map applications on the distributed-memory system to implement efficient I/O for the class of applications they support.This architecture has been implemented for the iWarp distributed-memory system. We describe this implementation and present performance results.