Controlled Asynchronous GVT

Recent studies investigated the performance of Parallel Discrete Event Simulation (PDES) on Intel Xeon Phi many-core processors, but generally reported underwhelming performance results, especially at high scales when all cores and thread contexts are fully loaded. While the lack of scalability in an earlier study on a Knights Corner (KC) processor is an artifact of physical limitations of the KC system, performance challenges on a Knights Landing (KNL) system partially stem from a slower global virtual time (GVT) computation algorithm used in that study. In this paper, we re-examine PDES performance on KNL under more efficient GVT algorithms to alleviate the GVT bottleneck. Specifically, we compare a synchronous GVT algorithm based on barrier synchronization, and two asynchronous GVT implementations: a modified Mattern's algorithm for shared memory systems and a recently-proposed wait-free algorithm. Using the ROSS simulator, we demonstrate that minimizing the GVT bottleneck results in significant improvement in scalability, allowing the simulation to scale with performance all the way to 250 threads (per chip). Interestingly, we observe that while for the balanced models the wait-free algorithm is a clear winner, barrier-based GVT provides significantly better results for imbalanced models executed at high scale. We also perform detailed simulation profiling to understand the underlying reasons for these performance trends.

Global virtual time (GVT) is used in the Time Warp synchronization mechanism to perform irrevocable operations such as I/O and to reclaim storage. Most existing algorithms for computing GVT assume a message-passing programming model. Here, GVT computation is examined in the context of a shared-memory model. We observe that computation of GVT is much simpler in shared-memory multiprocessors because these machines normally guarantee that no two processors will observe a set of memory operations as occurring in different orders. Exploiting this fact, we propose an efficient, asynchronous, shared-memory GVT algorithm and prove its correctness. This algorithm does not require message acknowledgments, special GVT messages, or FIFO delivery of messages, and requires only a minimal number of shared variables and data structures. The algorithm only requires one round of interprocessor communication to compute GVT, in contrast to many message-based algorithms that require two. An efficient implementatin is described that eliminates the need for a processor to explicitly compute a local minimum for time warp systems using a lowest-timestamp-first scheduling policy in each processor. In addition, we propose a new mechanism called on-the-fly fossil collection that enables efficient storage reclamation for simulations containing large numbers, e.g., hundreds of thousand or even millions of simulator objects. On-the-fly fossil collection can be used in time warp systems executing on either shared-memory or message-based machines. Performance measurements of the GVT algorithm and the on-the-fly fossil collection mechanism on a Kendall Square Research KSR-2 machine demonstrate that these techniques enable frequent GVT and fossil collections, e.g., every millisecond, without incurring a significant performance penalty

Global Virtual Time (GVT) is the fundamental synchronization concept in optimistic simulations. It is defined as the earliest time tag within the set of unprocessed pending events in distributed simulation. A number of techniques for determining GVT have been proposed in recent years, each having their own intrinsic properties. However, most of these techniques either focus on specific types of simulation problems or assume specific hardware support. This paper specifically addresses the GVT problem in the context of the following area A new GVT algorithm, called SPEEDES GVT, has been developed in the Synchronous Parallel Environment for Emulation and Discrete-Event Simulation (SPEEDES) framework. The new algorithm runs periodically but does not disrupt event processing. It provides flow control by processing events risk-free while flushing out messages during the GVT computation. SPEEDES GVT is built from a set of global reduction operations that are easily implementable on any hardware system.

Fine-Grained Parallel and Distributed Spatial Stochastic Simulation of Biological Reactions

The increasing diffusion of shared-memory multi-core machines has given rise to a change in the design of Parallel Discrete Event Simulation (PDES) platforms. In particular, the possibility to share large amounts of memory by many worker threads has lead to a boost in the adoption of non-blocking coordination algorithms, which have been proven to offer higher scalability when compared to their blocking counterparts based on critical sections. In this article we present an innovative non-blocking algorithm for computing Global Virtual Time (GVT)---namely, the current commit horizon---in multi-thread PDES engines to be run on top of multi-core machines. Beyond being non-blocking, our proposal has the advantage of providing a logarithmic (rather than linear) number of per-thread memory operations---read/write operations of values involved in the reduction for computing the GVT value---vs the amount of threads participating in the GVT computation. This allows for keeping low the actual CPU time that is required for determining the new GVT value. We compare our algorithm with a literature solution, still based on the non-blocking approach, but entailing a linear number of memory operations, quantifying the advantages from our proposal especially for very large numbers of threads participating in the GVT computation.

Global virtual time (GVT) is used in parallel discrete event simulations to reclaim memory, commit output, detect termination, and handle errors. Mattern 's [I] has proposed G VT approximation with distributed termination detection algorithm. This algorithm works fine and gives optimal performance in terms of accurate GVT computation at the expense of slower execution rate. This slower execution rate results a high GVT latency. Due to the high GVT latency, the processors involve in communication remain idle during that period of time. As a result, the overall throughput of a discrete event parallel simulation system degrades significantly. Thus, the high G VT latency prevents the widespread use of this algorithm in discrete event parallel simulation system. However, if we could improve the latency of GVT computation, most of the discrete event parallel simulation system would likely take advantage of this technique in terms of accurate G VT computation. In this paper, we examine the potential use of tress and butterflies barriers with the Mattern's GVT structure using a ring. Simulation results demonstrate that the use of tree barriers with the Mattern's GVT structure can significantly improve the latency time and thus increase the overall throughput of the parallel simulation system. The performance measure adopted in this paper is the achievable latency for a fixed number of processors and the number of message transmission during the G VT computation.

Parallel discrete-event simulation (PDES) is an important tool in the codesign of extreme-scale systems because PDES provides a cost-effective way to evaluate designs of high-performance computing systems. Optimistic synchronization algorithms for PDES, such as Time Warp, allow events to be processed without global synchronization among the processing elements. A rollback mechanism is provided when events are processed out of timestamp order. Although optimistic synchronization protocols enable the scalability of large-scale PDES, the performance of the simulations must be tuned to reduce the number of rollbacks and provide an improved simulation runtime. To enable efficient large-scale optimistic simulations, one has to gain insight into the factors that affect the rollback behavior and simulation performance. We developed a tool for ROSS model developers that gives them detailed metrics on the performance of their large-scale optimistic simulations at varying levels of simulation granularity. Model developers can use this information for parameter tuning of optimistic simulations in order to achieve better runtime and fewer rollbacks. In this work, we instrument the ROSS optimistic PDES framework to gather detailed statistics about the simulation engine. We have also developed an interactive visualization interface that uses the data collected by the ROSS instrumentation to understand the underlying behavior of the simulation engine. The interface connects real time to virtual time in the simulation and provides the ability to view simulation data at different granularities. We demonstrate the usefulness of our framework by performing a visual analysis of the dragonfly network topology model provided by the CODES simulation framework built on top of ROSS. The instrumentation needs to minimize overhead in order to accurately collect data about the simulation performance. To ensure that the instrumentation does not introduce unnecessary overhead, we perform a scaling study that compares instrumented ROSS simulations with their noninstrumented counterparts in order to determine the amount of perturbation when running at different simulation scales.

Parallel discrete event simulation (PDES) is an important application in use in many DoD projects; for example, PDES is used in large-scale war-gaming, and in complex system design, analysis and verification. Improving PDES performance and capacity allows faster simulation times and more extensive analysis of more detailed models. These benefits are not application-specific: they should reflect to any application that uses the improved simulation kernel. In this work, we overview our efforts for optimizing PDES in a heterogeneous high performance computing (HHPC) environment. We profile the SPEEDES simulator and identify several opportunities. We report on our experiences on two fronts: (1) optimizing the communication subsystem - a critical system for PDES since it is a fine-grained application and (2) exploring the use of augmented FPGA boards to accelerate simulation. While such approaches have been attempted for sequential and data path intensive applications, we believe that their use in clustered environments is novel. Both efforts are works in progress; we present our designs and some preliminary analysis results. For example, removing the centralized communication server from event message exchange path with a number of other small improvements to the simulation cycle, improved performance by an average of 20% performance improvement for one of our large benchmarks.

Nowadays simulation modeling is applied for solving a wide range of problems. There are simulations which require significant performance and time resources. To decrease overall simulation time a model can be converted to a distributed system and executed on a computer network. The goal of this project is to create a library enabling clear and rapid development parallel discrete event models in AnyLogic. The library is aimed for professionals in computer simulation and helps to reduce code amount. The project includes a research on different synchronization algorithms. In this paper we present techniques which can be used in creating distributed models. We present comparison of a single threaded model with a distributed model implementing optimistic algorithm. The comparison shows a significant improvement in wallclock time achieved by separating the model into independent submodels with minimal communications.

Fast barrier synchronization in wormhole k-ary n-cube networks with multidestination worms

As network simulations scale to hundreds of thousands of nodes, parallel discrete event simulation (PDES) has become indispensable for sustaining performance—yet its efficacy hinges on frequent time synchronization steps. Existing synchronization algorithms suffer from the inter-machine communication overhead in distributed environments, eroding the benefits of parallelism. We observe that time synchronization in PDES often involves frequent allreduce operations, and that in-network computing has the potential to substantially accelerate such collectives. In this work, we present SINA (Synchronization using In-Network Allreduce)—the first integration of in-network computing into time synchronization to address the demands of large-scale simulations. We implemented SINA in a testbed with Mellanox SHArP-enabled switches and ConnectX-5 network cards, offloading allreduce to the network hardware while preserving software‑level correctness. Our evaluation shows that SINA achieves up to 88.6% acceleration compared to state‑of‑the‑art methods and achieves up to 67.5% optimization in topologies of tens of thousands of nodes, demonstrating its suitability for high‑performance, large‑scale parallel simulations.

Barrier synchronization is a useful parallel programming construct for ensuring that all processors are at a particular location in the code before any processor is allowed to continue. Barrier synchronization is integral to programming models such as the Bulk Synchronous Parallel model. Specialized hardware is often used to improve the performance of a barrier synchronization operation. With continued improvement in processor performance, more efficient synchronization mechanisms are required to counter the rising relative cost of synchronization operations. A high-speed, distributed barrier synchronization mechanism has been developed for broadcast-based optical interconnection networks. This mechanism avoids multiple conversions between optical and electrical signals by having each processor locally decide whether the barrier in which it is participating has been satisfied. It also allows arbitrary sized partitions to be built dynamically during the execution of a program. Simulations of the current hardware design estimate that the barrier synchronization requires less than 300 ns for a 128-processor system.

Existing network architectures fall short when handling networking trends, e.g., mobility, server virtualization, and cloud computing, as well as market requirements with rapid changes. Software-defined networking (SDN) is designed to transform network architectures by decoupling the control plane from the data plane. Intelligence is shifted to the logically centralized controller with direct programmability, and the underlying infrastructures are abstracted from applications. The wide adoption of SDN in network industries has motivated development of large-scale, high-fidelity testbeds for evaluation of systems that incorporate SDN. We leverage our prior work on a hybrid network testbed with a parallel network simulator and a virtual-machine-based emulation system. In this paper, we extend the testbed to support OpenFlow-based SDN simulation and emulation; show how to exploit typical SDN controller behavior to deal with potential performance issues caused by the centralized controller in parallel discrete-event simulation; and investigate methods for improving the model scalability, including an asynchronous synchronization algorithm for passive controllers and a two-level architecture for active controllers. The techniques not only improve the simulation performance, but also are valuable for designing scalable SDN controllers.

This paper deals with the numerical solution of financial applications, more specifically the computation of American and European options derivatives modeled by boundary value problems. In such applications we have to solve large-scale algebraic linear systems. We concentrate on synchronous and asynchronous parallel iterative algorithms carried out on Grid’5000, by using an experimental peer-to-peer platform. The properties of the operators arising in the discretized problem ensure the convergence of the parallel iterative synchronous and asynchronous algorithms. Different detection convergence algorithms are tested for asynchronous iterations. Computational experiments performed on distributed architectures are presented and analyzed.

This article describes various algorithms on heterogeneous architecture applied to parallel polynomial root extracting. The use of a set of clusters in order to compute great parallel applications is an ongoing challenge. Asynchronous algorithms are well suited for this type of architecture, because no synchronous barriers are needed. The scalability of such architecture is a hard task to achieve. In this paper, we compare local calculus and calculus on the Internet in order to focus on the efficiency of such methods.