A Parallel Implementation of the Push-Relabel Algorithm for the Maximum Flow Problem

This paper proposes a method to maximize the rate of reliable communication for polar codes operating on channels with memory. The channel is learned implicitly from data by optimizing a neural polar decoder (NPD). This approach enables simultaneous optimization of the code rate over the input distribution and the design of a practical coding scheme within the framework of polar codes. The proposed approach applies to scenarios where the channel model is unknown and treated as a black-box that produces output samples from input samples. We use NPDs to estimate the mutual information (MI) between the channel inputs and outputs, and optimize a parametric model of the input distribution. The methodology involves a two-phase process: a training phase and an inference phase. In the training phase, two steps are repeated iteratively. The first step optimizes the NPD to estimate the MI of the channel inputs and outputs. The second step improves the input distribution parameters by maximizing the MI estimate obtained with the NPD. In the inference phase, the optimized model is used to construct polar codes. This approach uses the Honda-Yamamoto (HY) scheme, which implements polar codes with optimized input distributions, together with list decoding. Experimental results on memoryless and finite state channels (FSCs) demonstrate the effectiveness of this approach, particularly in cases where the channel’s capacity-achieving input distribution is non-uniform. For these cases, significant improvements in MI and bit error rates (BERs) are shown over those achieved by uniform and independent and identically distributed (i.i.d.) input distributions, validating our method for block lengths up to 1024. This data-driven approach can be utilized in real-world communication systems, bridging theoretical capacity estimation and practical coding performance.

The support of range, segment and rectangle queries are fundamental problems in computational geometry, and have extensive applications in many domains. Despite significant theoretical work on these problems, efficient implementations can be complicated, and most implementations do not have useful theoretical bounds. In this paper, we focus on simple and efficient parallel algorithms and implementations for range, segment and rectangle queries, which have worst-case bounds in theory and good performance in practice, both sequentially and in parallel. We propose to use a framework based on the abstract data type augmented map, to model the problems. Based on the augmented map interface, we develop both multi-level tree structures and sweepline algorithms supporting range, segment and rectangle queries in two dimensions. For the sweepline algorithms, we also propose a parallel paradigm and show corresponding cost bounds. Theoretically, the construction algorithms of all of our data structures are work-efficient and highly parallelized.We have implemented all the data structures described in the paper, ten in all, using a parallel augmented map library. Based on the library, each data structure only requires about 100 lines of C++ code. We test their performance on large data sets (up to 108 elements) and a machine with 72-cores (144 hyperthreads). The parallel construction achieves 32-68x speedup, and the speedup numbers for queries are up to 126-fold. Sequentially, each of our implementations outperforms the CGAL library by at least 2x in both construction and queries. Our sequential implementation has approximately the same construction time as the R-tree in the Boost library, but has significantly better query performance (1.6-1200x). We believe this paper provides the most comprehensive experimental study of data structures on range, segment and rectangle queries, both in parallel and sequential setting.

A new maximal flow algorithm for solving optimization problems with linguistic capacities and flows

AES, Advanced Encryption Standard, can be considered the most widely used modern symmetric key encryption standard. To encrypt/decrypt a file using the AES algorithm, the file must undergo a set of complex computational steps. Therefore a software implementation of AES algorithm would be slow and consume large amount of time to complete. The immense increase of both stored and transferred data in the recent years had made this problem even more daunting when the need to encrypt/decrypt such data arises. As a solution to this problem, in this paper, we present an extensive study of enhancing the throughput of AES encryption algorithm by utilizing the state of the art multicore architectures. We take a sequential program that implements the AES algorithm and convert the same to run on multicore architectures with minimum effort. We implement two different parallel programmes, one with the fork system call in Linux and the other with the pthreads, the POSIX standard for threads. Later, we ran both the versions of the parallel programs on different multicore architectures and compared and analysed the throughputs between the implementations and among different architectures. The pthreads implementation outperformed in all the experiments we conducted and the best throughput obtained is around 7Gbps on a 32-core processor (the largest number of cores we had) with the pthreads implementation.

Discovery of parallelization opportunities in sequential programs can greatly reduce the time and effort required to parallelize any application. Identification and analysis of code that contains little to no internal parallelism can also help expose potential parallelism. This paper provides a technique to identify a block of code called Computational Unit (CU) that performs a unit of work in a program. A CU can assist in discovering the potential parallelism in a sequential program by acting as a basic building block for tasks. CUs are used along with dynamic analysis information to identify the tasks that contain tightly coupled code within them. This process in turn reveals the tasks that are weakly dependent or independent. The independent tasks can be run in parallel and the dependent tasks can be analyzed to check if the dependences can be resolved. To evaluate our technique, different benchmark applications are parallelized using our identified tasks and the speedups are reported. In addition, existing parallel implementations of the applications are compared with the identified tasks for the respective applications.

Discovering which code sections in a sequential program can be made to run in parallel is the first step in parallelizing it, and programmers routinely struggle in this step. Most of the current parallelism discovery techniques focus on specific language constructs while trying to identify such code sections. In contrast, we propose to concentrate on the computations performed by a program. In our approach, a program is treated as a collection of computations communicating with one another using a number of variables. Each computation is represented as a Computational Unit (CU). A CU contains the inputs and outputs of a computation, and the three phases of a computation: read, compute, and write. Based on the notion of CU, We present a unified framework to identify both loop and task parallelism in sequential programs.

We present the spectral element method to simulate elastic-wave propagation in realistic geological structures involving complieated free-surface topography and material interfaces for two- and three-dimensional geometries. The spectral element method introduced here is a high-order variational method for the spatial approximation of elastic-wave equations. The mass matrix is diagonal by construction in this method, which drastically reduces the computational cost and allows an efficient parallel implementation. Absorbing boundary conditions are introduced in variational form to simulate unbounded physical domains. The time discretization is based on an energy-momentum conserving scheme that can be put into a classical explicit-implicit predictor/multi-corrector format. Long-term energy conservation and stability properties are illustrated as well as the efficiency of the absorbing conditions. The associated Courant condition behaves as ΔtC < O (nel−1/ndN−2), with nel the number of elements, nd the spatial dimension, and N the polynomial order. In practice, a spatial sampling of approximately 5 points per wavelength is found to be very accurate when working with a polynomial degree of N = 8. The accuracy of the method is shown by comparing the spectral element solution to analytical solutions of the classical two-dimensional (2D) problems of Lamb and Garvin. The flexibility of the method is then illustrated by studying more realistic 2D models involving realistic geometries and complex free-boundary conditions. Very accurate modeling of Rayleigh-wave propagation, surface diffraction, and Rayleigh-to-body-wave mode conversion associated with the free-surface curvature are obtained at low computational cost. The method is shown to provide an efficient tool to study the diffraction of elastic waves by three-dimensional (3D) surface topographies and the associated local effects on strong ground motion. Complex amplification patterns, both in space and time, are shown to occur even for a gentle hill topography. Extension to a heterogeneous hill structure is considered. The efficient implementation on parallel distributed memory architectures will allow to perform real-time visualization and interactive physical investigations of 3D amplification phenomena for seismic risk assessment.

The calculation of multipoint likelihoods of pedigree data is crucial for extracting the full available information needed for both parametric and nonparametric linkage analysis. Recent mathematical advances in both the Elston-Stewart and Lander-Green algorithms for computing exact multipoint likelihoods of pedigree data have enabled researchers to analyze data sets containing more markers and more individuals both faster and more efficiently. This paper presents novel algorithms that further extend the computational boundary of the Elston-Stewart algorithm. They have been implemented into the software package VITESSE v. 2 and are shown to be several orders of magnitude faster than the original implementation of the Elston-Stewart algorithm in VITESSE v. 1 on a variety of real pedigree data. VITESSE v. 2 was faster by a factor ranging from 168 to over 1,700 on these data sets, thus making a qualitative difference in the analysis. The main algorithm is based on the faster computation of the conditional probability of a component nuclear family within the pedigree by summing over the joint genotypes of the children instead of the parents as done in the VITESSE v. 1. This change in summation allows the parent-child transmission part of the calculation to be not only computed for each parent separately, but also for each locus separately by using inheritance vectors as is done in the Lander-Green algorithm. Computing both of these separately can lead to substantial computational savings. The use of inheritance vectors in the nuclear family calculation represents a partial synthesis of the techniques of the Lander-Green algorithm into the Elston-Stewart algorithm. In addition, the technique of local set recoding is introduced to further reduce the complexity of the nuclear family computation. These new algorithms, however, are not universally faster on all types of pedigree data compared to the method implemented in VITESSE v. 1 of summing over the parents. Therefore, a hybrid algorithm is introduced which combines the strength of both summation methods by using a numerical heuristic to decide which of the two to use for a given nuclear family within the pedigree and is shown to be faster than either method on its own. Finally, this paper discusses various complexity issues regarding both the Elston-Stewart and Lander-Green algorithms and possible future directions of further synthesis.

Efficient parallel algorithms and software for compressed octrees with applications to hierarchical methods

An efficient hardware implementation of a training algorithm is presented, combining supervised and unsupervised techniques. In VLSI circuits, effects including random offsets and mismatch, system distortion, frequency response, and temperature deviations perturb the system outputs. Analysis of the generalized Hebbian algorithm shows that these small deviations result in small perturbations of the output statistics and the weight matrix. Also, the conjugate-gradient optimization algorithm is formulated in continuous-time. Both the generalized Hebbian algorithm and the optimization system can be efficiently implemented on a mesh of synapses. The low distortion and high bandwidth (>100 MHz) for the matrix operations and a high-performance analog memory indicate high performance for the generalized Hebbian algorithm. From the mathematical theory, the error of the output statistics is less than 1%. >

Scaling properties of a parallel implementation of the multicanonical algorithm

A process is proposed for refining specifications of abstract data types into efficient sequential implementations. The process needs little manual intervention. It is split into three stages, not all of which need always be carried out. The three stages entail interpreting equalities as behavioural equivalences, converting functions into procedures, and replacing axioms by programs. The stages can be performed as automatic transformations which are certain to produce results that meet the specifications, provided that simple conditions hold. These conditions describe the adequacy of the specifications, the freedom from interference between the procedures, and the mode of construction of the procedures. Sufficient versions of these conditions can be checked automatically. Varying the conditions could produce implementations for different classes of specification. Though the transformations could be automated, the intermediate results, in styles of specification which cover both functions procedures, have interest in their own right and may be particularly appropriate to object-oriented design.

In autonomous navigation technologies, the Multi-State Constraint Kalman Filter (MSCKF) is one of the most accurate and robust tightly-coupled fusion frameworks for Visual-Inertial Navigation (VIN). However, the adoption of the MSCKF VIN system in real-time embedded applications depends heavily on an efficient implementation of its tangled pipeline. This work initially proposes a novel parallel multi-thread implementation of the MSCKF VIN pipeline on an embedded CPU-enabled hardware that has speeded up the per-epoch processing time of the pipeline by 41% compared to the conventional sequential implementation. The heart of the MSCKF pipeline’s visual backend is an inertially-aided 3D localization of visual feature points that are reduced to a set of nonlinear optimization problems which were conventionally solved in a serial fashion using the single-objective Gauss-Newton optimization algorithm. This work leveraged the parallel architecture of an embedded GPU and further proposes an efficient parallel implementation of a multi-objective Gauss-Newton algorithm. Integration of the proposed GPU-accelerated feature localization technique in the MSCKF parallel pipeline has resulted in 33% faster per-epoch processing time and consequently, the satisfaction of strict real-time constraints. The proposed parallel MSCKF VIN pipelines have been developed using C++ and CUDA on the NVIDIA Jetson TX2 embedded board. Experimental evaluations on a real visual-inertial odometry dataset have been provided to validate the efficacy and real-time performance enhancement of the proposed parallel implementation.

There are many proposed vision based methods to perform obstacle detection and avoidance for autonomous or semi-autonomous vehicles. A system capable of supporting autonomous helicopter navigation will need to extract obstacle information from imagery at rates varying from ten images per second to thirty or more images per second depending on the vehicle speed. This paper describes an efficient and flexible parallel implementation of a multisensor feature-based range-estimation algorithm, targeted for automated helicopter flight. The algorithm can track hundreds of features in multiple image sensors using an extended Kalman filter to estimate the feature's location in a master sensor coordinate frame. The feature-tracking algorithm has reached relative maturity in the laboratory and is now being ported to several real-time architectures to support autonomous helicopter navigation research, The focus of this paper is not the core theory of the vision algorithm itself, but those aspects of it that affect the method of parallelization. The performance of the parallel algorithm is analyzed, with respect to three load balancing schemes, on both a distributed-memory and shared-memory parallel computer. >

The real-time processing is getting more and more important in many image processing applications. Image segmentation is one of the most fundamental tasks image analysis. As a consequence, many different approaches for image segmentation have been proposed. The watershed transform is a well-known image segmentation tool. The watershed transform is a very data intensive task. To achieve acceleration and obtain real-time processing of watershed algorithms, parallel architectures and programming models for multicore computing have been developed. This paper focuses on the survey of the approaches for parallel implementation of sequential watershed algorithms on multicore general purpose CPUs: homogeneous multicore processor with shared memory. To achieve an efficient parallel implementation, it’s necessary to explore different strategies (parallelization/distribution/distributed scheduling) combined with different acceleration and optimization techniques to enhance parallelism. In this paper, we give a comparison of various parallelization of sequential watershed algorithms on shared memory multicore architecture. We analyze the performance measurements of each parallel implementation and the impact of the different sources of overhead on the performance of the parallel implementations. In this comparison study, we also discuss the advantages and disadvantages of the parallel programming models. Thus, we compare the OpenMP (an application programming interface for multi-Processing) with Ptheads (POSIX Threads) to illustrate the impact of each parallel programming model on the performance of the parallel implementations.