An Experimental Study of Parallel Biconnected Components Algorithms on Symmetric Multiprocessors (SMPs)

The all-pairs shortest path problem is a classic problem to study characteristics of the given graphs. Though many efficient all-pairs shortest path algorithms have been published, it is still a very expensive computing task, especially with large graph datasets. In this paper, we propose an efficient parallel all-pairs shortest path algorithm based on Peng et al.'s fast sequential algorithm on shared-memory parallel environments to achieve faster and more efficient calculation for large-scale real-world networks. Peng et al.'s algorithm needs to sort vertices with respect to their degrees. However, it turns out the original algorithm uses less efficient sorting method, which is a significant portion of parallel overhead. Therefore, we also propose an efficient parallel method to sort data within a fixed range, in order to minimize the parallel overhead in our parallel algorithm. The optimized efficient sorting method can be used for general sorting purposes. Our experimental analysis shows that our proposed parallel algorithm achieves very high parallel speedup, even hyper-linear speedup, with real-world test datasets on two different shared-memory multi-core systems.

Unlocking massively parallel spectral proper orthogonal decompositions in the PySPOD package

A new method to retrieve aerosol optical thickness from satellite images on a parallel system

At first thought, one may not realize the need for a special issue of Parallel and Distributed Computing Practices focused on Cluster Computing when our community has already invested years of research and spent a wealth of resources on traditional Big Iron supercomputers, for instance, the SGI/Cray Origin and IBM SP. Academia, industry, and national laboratories are still using, and for the foreseeable future, will continue to use, supercomputers to solve both grand-challenge scale and high-throughput applications. Then why the interest and new popularity of research into clusters for high-performance parallel and distributed applications? Clusters incorporate the tremendous successes that are attributable to our parallel and distributed processing community. Over the next decade, clusters will span the entire range of high-performance computing platforms. Instead of being restricted solely to supercomputers for achieving high-performance computing, the spectrum of available platforms will spread out to Beowulf-class clusters, to superclusters, and on to supercomputers. Moving along this continuum represents increasing computational resources, which in turn allows the capability for solving larger classes of applications or instances of problems. Compute-bound applications on workstations often may take advantage of the extra horsepower provided by symmetric multiprocessors. Applications with even larger computational requirements and a high-degree of concurrency may use Beowulf clusters, that is, clusters that are home-built from mass-market, commercial off-the-shelf workstations and networks, for instance, PC's interconnected by Fast Ethernet. A supercluster may contain commodity SMP nodes interconnected with scalable, low-latency, high-bandwidth networks, and is integrated by a system vendor rather than the computational scientist. Applications with greater complexity, for example, grand challenges from the sciences, demand scalable systems with high-performance, interconnects, perhaps coupled with terascale I/O subsystems, and thus, are appropriate for these superclusters. Finally, supercomputers, the largest of the clusters, may provide the ultimate platform for the highest-fidelity models and simulations and data-intensive applications. At each step up this ladder of cluster architectures, the system cost increases approximately by an order of magnitude, but having the high-performance capability to solve the problem of interest may require an entry point at any of these levels, the Beowulf, or supercluster, or even supercomputer platform. Cluster computing with Beowulf- or superclusters, thus, may be one or two orders of magnitude less expensive than a supercomputer, and provide a cost-effective solution and capability that was not available on a workstation. But this cost comparison measures just the basic system price. What about the applications? As recently as a few years ago, the lack of a canonical high-performance architectural paradigm meant that migrating to a new computer system, even from the same vendor, typically required redesigning applications to efficiently use the high-performance system. Thus, the application must be redesigned with parallel algorithms and libraries that are optimized for each high-performance system. This process of porting, redesigning, optimizing, debugging, and analyzing, the application may in fact be prohibitively expensive, and certainly frustrating when the process is completed just in time to greet the next generation system! That was then, this is now. With the successful standardization of system-level interfaces, many of these issues no longer hinder our expedited use of these new, high-performance clusters. The message-passing interface (MPI) is a clear example of a standard that has contributed significantly to the success of cluster computing. Supercomputer vendors working together with the high-performance computing community embarked on a process to standardize the user-level interface and behavior for passing messages between concurrent tasks. Now, using ordinary sequential compilers and vendor-supplied or freely-available implementations of this message-passing interface, applications can be implemented efficiently on one supercomputer and ported with little or no effort to other parallel machines. The message-passing paradigm, thus, provides a reasonably good canonical high-performance model and methodology for implementing high-performance applications. An algorithm that exhibits efficient execution and scaling on one machine then is likely to perform well on another with little or no additional programming effort. Using this cluster computing paradigm, algorithms typically are optimized by minimizing the number of remote memory accesses (reducing the amount of messages passed), and carefully organizing the data placement to increase memory locality. Thus, high-performance algorithms and applications that have been optimized for supercomputers, using the same cluster paradigm, may be appropriate for Beowulf and superclusters. Cluster computing, realizing the importance of a standard high-performance programming environment that applies equally well to both ends of the architectural spectrum, highly leverages these and other successes from parallel and distributed processing research and development. This is in an exciting, new era for high-performance computing where the canonical architectural paradigm stretches from home-built Beowulf clusters, to superclusters, and on to supercomputers. But cluster computing faces some challenges ahead of it. Continued research is required for designing 1) scalable, high-performance algorithms, applications, and methodologies, that can tolerate the communication overheads, and adapt to hybrid, SMP, and heterogeneous, nodes; 2) high-performance computer architectures, scalable interconnection networks, and I/O subsystems; and 3) scalable systems software, compilers, and monitoring tools. Cluster computing has come of age, and it is high time for the parallel and distributed processing community to lay the groundwork for our own legacy of success stories. David A. Bader University of New Mexico

In this article, an optimal parallel algorithm is suggested for fast computation of a crucial yet fundamental numerical problem in science named ‘prefix sum’ on a popular three-dimensional torus network. The proposed parallel algorithm demands total ‘4n + 2’ communication moves for mapping prefix sum problem on odd network sizes of n × n×n three-dimensional torus network, whereas total ‘4n’ communication moves is required for parallel mapping of same on even network sizes. In fact, a unit increase in each dimension from odd to even and even to odd network sizes resulted in the constant increase of 2 and 6 units, respectively, in the required communication moves for mapping prefix sum. The algorithmic performance of the proposed parallel algorithm can be compared to counter-part parallel prefix sum approaches on grid-based networks.

Parallel graph algorithms for hypercube computers

Random sampling is an important tool in the design of parallel algorithms. Using random sampling it is possible to obtain simple parallel algorithms which are efficient in practice. We will focus on the use of random sampling in fundamental problems such as sorting, selection, list ranking and graph connectivity.

Emulations between QSM, BSP and LogP: a framework for general-purpose parallel algorithm design

Betweenness, a widely employed centrality measure in network science, is a decent proxy for investigating network loads and rankings. However, its extremely high computational cost greatly hinders its applicability in large networks. Although several parallel algorithms have been presented to reduce its calculation cost for unweighted networks, a fast solution for weighted networks, which are commonly encountered in many realistic applications, is still lacking. In this study, we develop an efficient parallel GPU-based approach to boost the calculation of the betweenness centrality (BC) for large weighted networks. We parallelize the traditional Dijkstra algorithm by selecting more than one frontier vertex each time and then inspecting the frontier vertices simultaneously. By combining the parallel SSSP algorithm with the parallel BC framework, our GPU-based betweenness algorithm achieves much better performance than its CPU counterparts. Moreover, to further improve performance, we integrate the work-efficient strategy, and to address the load-imbalance problem, we introduce a warp-centric technique, which assigns many threads rather than one to a single frontier vertex. Experiments on both realistic and synthetic networks demonstrate the efficiency of our solution, which achieves 2.9× to 8.44× speedups over the parallel CPU implementation. Our algorithm is open-source and free to the community; it is publicly available throughhttps://dx.doi.org/10.6084/m9.figshare.4542405. Considering the pervasive deployment and declining price of GPUs in personal computers and servers, our solution will offer unprecedented opportunities for exploring betweenness-related problems and will motivate follow-up efforts in network science.

The support of range, segment and rectangle queries are fundamental problems in computational geometry, and have extensive applications in many domains. Despite significant theoretical work on these problems, efficient implementations can be complicated, and most implementations do not have useful theoretical bounds. In this paper, we focus on simple and efficient parallel algorithms and implementations for range, segment and rectangle queries, which have worst-case bounds in theory and good performance in practice, both sequentially and in parallel. We propose to use a framework based on the abstract data type augmented map, to model the problems. Based on the augmented map interface, we develop both multi-level tree structures and sweepline algorithms supporting range, segment and rectangle queries in two dimensions. For the sweepline algorithms, we also propose a parallel paradigm and show corresponding cost bounds. Theoretically, the construction algorithms of all of our data structures are work-efficient and highly parallelized.We have implemented all the data structures described in the paper, ten in all, using a parallel augmented map library. Based on the library, each data structure only requires about 100 lines of C++ code. We test their performance on large data sets (up to 108 elements) and a machine with 72-cores (144 hyperthreads). The parallel construction achieves 32-68x speedup, and the speedup numbers for queries are up to 126-fold. Sequentially, each of our implementations outperforms the CGAL library by at least 2x in both construction and queries. Our sequential implementation has approximately the same construction time as the R-tree in the Boost library, but has significantly better query performance (1.6-1200x). We believe this paper provides the most comprehensive experimental study of data structures on range, segment and rectangle queries, both in parallel and sequential setting.

A comparison of OpenMP and MPI for neural network simulations on a SunFire 6800

Concurrent and parallel algorithms are different. However, in the case of dictionaries, both kinds of algorithms share many common points. We present a unified approach emphasizing these points. It is based on a careful analysis of the sequential algorithm, extracting from it the more basic facts, encapsulated later on as local rules. We apply the method to the insertion algorithms in AVL trees. All the concurrent and parallel insertion algorithms have two main phases: a percolation phase, moving the keys to be inserted down, and a rebalancing phase. Finally, some other algorithms and balanced structures are discussed.

An efficient parallel algorithm is presented for computing selected components of $A^{-1}$ where A is a structured symmetric sparse matrix. Calculations of this type are useful for several applications, including electronic structure analysis of materials in which the diagonal elements of the Green's functions are needed. The algorithm proposed here is a direct method based on a block $LDL^T$ factorization. The selected elements of $A^{-1}$ we compute lie in the nonzero positions of $L+L^T$. We use the elimination tree associated with the block $LDL^T$ factorization to organize the parallel algorithm, and reduce the synchronization overhead by passing the data level by level along this tree using the technique of local buffers and relative indices. We demonstrate the efficiency of our parallel implementation by applying it to a discretized two dimensional Hamiltonian matrix. We analyze the performance of the parallel algorithm by examining its load balance and communication overhead, and show that our parallel implementation exhibits an excellent weak scaling on a large-scale high performance distributed-memory parallel machine.

Schedulers for symmetric multiprocessing (SMP) machines use sophisticated algorithms to schedule processes onto the available processor cores. Hardware-dependent code and the use of locks to protect shared data structures from simultaneous access lead to poor portability, the difficulty to prove correctness, and a myriad of problems associated with locking such as limiting the available parallelism, deadlocks, starvation, interrupt handling, and so on. In this work we explore what can be achieved in terms of portability and simplicity in an SMP scheduler that achieves similar performance to state-of-the-art schedulers. By strictly limiting ourselves to only lock-free data structures in the scheduler, the problems associated with locking vanish altogether. We show that by employing implicit cooperative scheduling, additional guarantees can be made that allow novel and very efficient implementations of memory-efficient unbounded lock-free queues. Cooperative multitasking has the additional benefit that it provides an extensive hardware independence. It even allows the scheduler to be used as a runtime library for applications running on top of standard operating systems. In a comparison against Windows Server and Linux running on up to 64 cores we analyze the performance of the lock-free scheduler and show that it matches or even outperforms the performance of these two state-of-the-art schedulers in a variety of benchmarks.

We consider the problem of computing a weighted edge matching in a large graph using a parallel algorithm. This problem has application in several areas of combinatorial scientific computing. Since an exact algorithm for the weighted matching problem is both fairly expensive to compute and hard to parallelise we instead consider fast approximation algorithms.We analyse a distributed algorithm due to Hoepman [8] and show how this can be turned into a parallel algorithm. Through experiments using both complete as well as sparse graphs we show that our new parallel algorithm scales well using up to 32 processors.KeywordsParallel AlgorithmSparse GraphDominant EdgeFast Approximation AlgorithmDrop MessageThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.