Theoretically-Efficient and Practical Parallel DBSCAN

This paper presents new parallel algorithms for generating Euclidean minimum spanning trees and spatial clustering hierarchies (known as HDBSCAN*). Our approach is based on generating a well-separated pair decomposition followed by using Kruskal's minimum spanning tree algorithm and bichromatic closest pair computations. We introduce a new notion of well-separation to reduce the work and space of our algorithm for HDBSCAN*. We also give a new parallel divide-and-conquer algorithm for computing the dendrogram and reachability plots, which are used in visualizing clusters of different scale that arise for both EMST and HDBSCAN*. We show that our algorithms are theoretically efficient: they have work (number of operations) matching their sequential counterparts, and polylogarithmic depth (parallel time). We implement our algorithms and propose a memory optimization that requires only a subset of well-separated pairs to be computed and materialized, leading to savings in both space (up to 10x) and time (up to 8x). Our experiments on large real-world and synthetic data sets using a 48-core machine show that our fastest algorithms outperform the best serial algorithms for the problems by 11.13--55.89x, and existing parallel algorithms by at least an order of magnitude.

Traditionally, interest in parallel computation centered around the speedup provided by parallel algorithms over their sequential counterparts. In this paper, we ask a different type of question: Can parallel computers, due to their speed, do more than simply speed up the solution to a problem? We show that for real-time optimization problems, a parallel computer can obtain a solution that is better than that obtained by a sequential one. Specifically, a sequential and a parallel algorithm are exhibited for the problem of computing the best-possible approximation to the minimum-weight spanning tree of a connected, undirected and weighted graph whose vertices and edges are not all available at the outset, but instead arrive in real time. While the parallel algorithm succeeds in computing the exact minimum-weight spanning tree, the sequential algorithm can only manage to obtain an approximate solution. In the worst case, the ratio of the weight of the solution obtained sequentially to that of the solution computed in parallel can be arbitrarily large.

Over the years many interesting and efficient parallel algorithms have been developed to solve a wide variety of problems, but not much attention has been paid to studying the inherent parallelism in sequential algorithms---i.e., understanding the depth of their dependence structure, and how shallow dependence structures might beused to develop efficient parallel implementations. In this talk I will describe recent work on analyzing the dependence depth of iterative sequential algorithms---ones that loop over a collection of elements. Many of these algorithms have deep dependence chains in the worst case, but shallow chains (polylog w.h.p.) if the elements are randomly ordered. Examples include many fundamental algorithms: the Knuth shuffle for random permutations, sorting by insertion into a binary search tree, greedy maximal independent set (MIS), greedy maximal matching, greedy graph-coloring, counting cycles in a permutation, incremental k-dimensional linear programming, and incremental 2d Delaunay triangulation. An advantage of the approach is that it can lead to very simple and efficient parallel algorithms. Our MIS algorithm, for example can be coded in a dozen or so lines, and is significantly faster than Luby's algorithm on modern multicore machines. Also the approach encourages snapping the view that sequential and parallel algorithms are distinct, and instead thinking of algorithms, in general, as collections of instructions with dependences among them.

We present a parallel Delaunay refinement algorithm for generating well-shaped meshes in both two and three dimensions. Like its sequential counterparts, the parallel algorithm iteratively improves the quality of a mesh by inserting new points, the Steiner points, into the input domain while maintaining the Delaunay triangulation. The Steiner points are carefully chosen from a set of candidates that includes the circumcenters of poorly-shaped triangular elements. We introduce a notion of independence among possible Steiner points that can be inserted simultaneously during Delaunay refinements and show that such a set of independent points can be constructed efficiently and that the number of parallel iterations is O( log 2Δ), where Δ is the spread of the input — the ratio of the longest to the shortest pairwise distances among input features. In addition, we show that the parallel insertion of these set of points can be realized by sequential Delaunay refinement algorithms such as by Ruppert's algorithm in two dimensions and Shewchuk's algorithm in three dimensions. Therefore, our parallel Delaunay refinement algorithm provides the same shape quality and mesh-size guarantees as these sequential algorithms. For generating quasi-uniform meshes, such as those produced by Chew's algorithms, the number of parallel iterations is in fact O( log Δ). To the best of our knowledge, our algorithm is the first provably polylog(Δ) time parallel Delaunay-refinement algorithm that generates well-shaped meshes of size within a constant factor of the best possible.

Data partitioning algorithms aiming to minimize the execution time and the energy of computations in self-adaptable data-parallel applications on modern extreme-scale multicore platforms must address two critical challenges. First, they must take into account the new complexities inherent in these platforms such as severe resource contention and non-uniform memory access. Second, they must have low practical runtime and memory costs. The sequential data partitioning algorithms addressing the first challenge have a theoretical time complexity of O( $m * m * p * p$ ) where $m$ is the number of points in the discrete speed/energy function and $p$ is the number of available processors. They, however, exhibit high practical runtime cost and excessive memory footprint, therefore, rendering them impracticable for employment in self-adaptable applications executing on extreme-scale multicore platforms. We present, in this paper, the parallel data partitioning algorithms that address both the challenges. They take as input the functional models of performance and energy consumption against problem size and output workload distributions, which are globally optimal solutions. They have a low time complexity of O( $m * m * p$ ) thereby providing a linear speedup of O( $p$ ) and low memory complexity of O( $n$ ) where $n$ is the workload size expressed as a multiple of granularity. They employ dynamic programming approach, which also facilitates the easier integration of performance and energy models of communications. We experimentally study the practical cost of application of our algorithms in two data-parallel applications, matrix multiplication and fast Fourier transform, on a cluster in Grid’5000 platform. We demonstrate that their practical runtime and memory costs are low making them ideal for employment in self-adaptable applications. We also show that the parallel algorithms exhibit tremendous speedups over the sequential algorithms. Finally, using theoretical analysis for a forecast exascale platform, we demonstrate that the parallel algorithms have negligible execution times compared to the matrix multiplication application executing on the platform.

We present new shared-memory parallel algorithms for the bi-core decomposition problem, which discovers dense subgraphs in bipartite graphs and is the bipartite analogue of the classic k-core decomposition problem. We develop a theoretically-efficient parallel bi-core decomposition algorithm that discovers a hierarchy by peeling vertices from the graph in parallel. Our algorithm improves the span (parallel running time) over the state-of-the-art parallel bi-core decomposition algorithm, while matching the state-of-the-art sequential algorithm in work. We additionally prove the bi-core decomposition problem to be P-complete, meaning that a polylogarithmic span solution is unlikely under standard assumptions. We also devise a theoretically-efficient parallel bi-core index structure to allow for fast parallel queries of vertices in given cores. Finally, we propose a novel practical optimization that prunes unnecessary computations, and we provide optimized parallel implementations of our bi-core decomposition algorithms that are scalable and fast. Using 30 cores with two-way hyper-threading, our implementation achieves up to a 4.9x speedup over the state-of-the-art parallel algorithm. Our parallel index structure can be constructed up to 27.7x faster than the state-of-the-art sequential counterpart. Due to the improved storage format of our index structure, our parallel queries are up to 116.3x faster than the state-of-the-art sequential queries.

Three diagnostic rapid test kits (Chembio HIV-1/2 Stat-Pak, Alere DetermineTM HIV-1/2, and Core HIV-1/2) for human immunodeficiency virus types 1 and 2 (HIV-1/2) antibodies detection were evaluated against a gold standard fourth generation ELISA (Dia-Pro). Hundred and seventy two serum samples were collected from Lulubriggs Health Center, University of Port Harcourt, Rivers State. All samples were tested for HIV-1/2 using the three rapid test (parallel algorithm and serial algorithm), and retested using the 4th generation ELISA. The results obtained were compared with those from the ELISA. The DetermineTM rapid strips had sensitivity of 66.7%, specificity of 100%, positive predictive value 100%, negative predictive value 89.4% and total agreement of 91.3%. Stat-pak and Core HIV-1/2 rapid strips both had sensitivity of 71.7%, specificity of 100%, positive predictive value 100%, and negative predictive value 90.7% and total agreement of 92.4%. Determine was mostly used by the participant laboratories 18 of 30 (60%). The performance evaluation of the testing algorithm showed that parallel algorithm is more accurate in comparison to serial algorithm. The result of this study reveals that most rapid test are less sensitive and the accuracy of serial testing algorithm is low, this implies that most acute HIV infection will be missed with rapid assay and false negative results will be reported as actual negative with serial algorithm.

Parallel and sequential transformations on digital images

The Course Timetabling problem is one of the most difficult combinatorial problems that arises with a University. The main objective of this problem is to obtain a timetable that minimises student conflicts between assigned activities. This is a discrete combinatorial problem that can be extremely difficult to solve for a human expert so computational heuristics are usually implemented in order to find good solutions within a reasonable time. With the advent of multi-core and hyper-threading technologies, parallel heuristics can speed up the solution process and with a proper parallel design these heuristics can improve the quality of solutions with the same number of Fitness evaluations than sequential algorithms. This paper explores the implementation of a parallel set of heuristic algorithms based on genetic algorithms, Scatter Search and discrete PSO for CTTP problem. Our experiments used as benchmark set instances from ITC2007 Track 2. Also the algorithms described in this paper make use of a layer of independence called methodology of design in order to be easily adaptable to new instances. Every parallel algorithm is compared against its sequential counterpart through speed-up metrics like Weak speed-up proposed by Alba et al.

High-resolution adaptive-optical systems with thousands to millions of pixels will most likely have to employ serial- or matrix-addressed spatial light modulators (e.g., microelectromechanical-system-on-VLSI spatial light modulators). We compare parallel gradient descent adaptive-optics algorithms with serial gradient descent algorithms running on serially addressed modulators. While serial algorithms have previously been shown to require more iterations than parallel algorithms, we show that, because of the limitations of the databus, each serial iteration of the algorithm on a serial modulator requires significantly less time to complete than a parallel iteration, thereby favoring the serial algorithm when time to convergence is used as the performance metric. Thus, such high-resolution serially addressed devices are generally better matched to the serial-update wavefront correction algorithm owing to the data load penalty imposed by the bandwidth-limited databus of these modulators.

Unlocking massively parallel spectral proper orthogonal decompositions in the PySPOD package

The fast iterative shrinkage-thresholding algorithm (FISTA) is a well-known first order method used to minimize <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{1}$</tex> regularized problems. However, it is also a non-monotone algorithm that can exhibit a sudden and gradual oscillatory behavior during the convergence. One of the parameters that directly affects the convergence of the FISTA method, whose optimal value is typically unknown, is the step-size (SS) that is linked to the Lipschitz constant. Depending on a suitable selection of the SS either manual or automatic, and the SS evolution throughout iterations, e.g. constant, decreasing, or increasing sequence, the practical performance can differ in orders of magnitude with or without stability issues (oscillations or, in the worst case, divergence). In this paper, we propose an algorithm, which has two variants, to address the stability issues in case of ill-chosen parameters for a given SS policy (either manual or adaptive). The proposed method structurally consists of an instability prediction rule based on the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell_{\infty}$</tex> norm of the gradient, and a correction for it, which can interpreted as an under-relaxation technique.

To evaluate serologic testing algorithms for human immunodeficiency virus (HIV) based on a combination of rapid assays among persons with HIV-1 (non-B subtypes) infection, HIV-2 infection, and HIV-1-HIV-2 dual infections in Abidjan, Ivory Coast, a total of 1,216 sera with known HIV serologic status were used to evaluate the sensitivity and specificity of four rapid assays: Determine HIV-1/2, Capillus HIV-1/HIV-2, HIV-SPOT, and Genie II HIV-1/HIV-2. Two serum panels obtained from patients recently infected with HIV-1 subtypes B and non-B were also included. Based on sensitivity and specificity, three of the four rapid assays were evaluated prospectively in parallel (serum samples tested by two simultaneous rapid assays) and serial (serum samples tested by two consecutive rapid assays) testing algorithms. All assays were 100% sensitive, and specificities ranged from 99.4 to 100%. In the prospective evaluation, both the parallel and serial algorithms were 100% sensitive and specific. Our results suggest that rapid assays have high sensitivity and specificity and, when used in parallel or serial testing algorithms, yield results similar to those of enzyme-linked immunosorbent assay-based testing strategies. HIV serodiagnosis based on rapid assays may be a valuable alternative in implementing HIV prevention and surveillance programs in areas where sophisticated laboratories are difficult to establish.

ABSTRACTWith the increasing sizes of digital elevation models (DEMs), there is a growing need to design parallel schemes for existing sequential algorithms that identify and fill depressions in raster DEMs. The Priority-Flood algorithm is the fastest sequential algorithm in the literature for depression identification and filling of raster DEMs, but it has had no parallel implementation since it was proposed approximately a decade ago. A parallel Priority-Flood algorithm based on the fastest sequential variant is proposed in this study. The algorithm partitions a DEM into stripes, processes each stripe using the sequential variant in many rounds, and progressively identifies more slope cells that are misidentified as depression cells in previous rounds. Both Open Multi-Processing (OpenMP)- and Message Passing Interface (MPI)-based implementations are presented. The speed-up ratios of the OpenMP-based implementation over the sequential algorithm are greater than four for all tested DEMs with eight computing threads. The mean speed-up ratio of our MPI-based implementation is greater than eight over TauDEM, which is a widely used MPI-based library for hydrologic information extraction. The speed-up ratios of our MPI-based implementation generally become larger with more computing nodes. This study shows that the Priority-Flood algorithm can be implemented in parallel, which makes it an ideal algorithm for depression identification and filling on both single computers and computer clusters.

With the increase of devices equipped with location sensors, mining spatio-temporal data for interesting behavioral patterns has gained attention in recent years. One of such well-known patterns is the convoy pattern which can be used, e.g., to find groups of people moving together in public transport or to prevent traffic jams. A convoy consists of at least m objects moving together for at least k consecutive time instants where m and k are user-defined parameters. Convoy mining is an expensive task and existing sequential algorithms do not scale to real-life dataset sizes. Existing sequential as well as parallel algorithms require a complex set of data-dependent parameters which are hard to set and tune. Therefore, in this paper, we propose a new fast exact sequential convoy pattern mining algorithm "k/2-hop" that is free of data-dependent parameters. The proposed algorithm processes the data corresponding to a few specific key timestamps at each step and quickly prunes objects with no possibility of forming a convoy. Thus, only a very small portion of the complete dataset is considered for mining convoys. Our experimental results show that k/2-hop outperforms existing sequential as well as parallel convoy pattern mining algorithms by orders of magnitude, and scales to larger datasets which existing algorithms fail on.