Synchronization methods in parallel and distributed discrete-event simulation

Fine-Grained Parallel and Distributed Spatial Stochastic Simulation of Biological Reactions

Parallel Discrete Event simulation (PDEs) is an important technology for studying large-scale complex systems[1]. And there are some problems in the present PDEs modeling, such as poor intuitiveness, low efficiency, difficulty in expansion and reuse, insufficient support of modeling environment for model checking, difficulty in model checking and so on. For these problems, a parallel discrete event simulation modeling method, named Object Class Interaction Graph(OCIG), based on basic event graph and object-oriented thinking is proposed. It has the natural and simple advantages of event graph modeling for discrete event simulation, and it can well meet the requirements of scalable, easy assembly, and layered parallelism for parallel discrete event simulation applications. The article also uses the set theory to formally define it, laying the foundation for the subsequent formal checking of the model [2].

Parallel discrete-event simulation (PDES) is an important tool in the codesign of extreme-scale systems because PDES provides a cost-effective way to evaluate designs of high-performance computing systems. Optimistic synchronization algorithms for PDES, such as Time Warp, allow events to be processed without global synchronization among the processing elements. A rollback mechanism is provided when events are processed out of timestamp order. Although optimistic synchronization protocols enable the scalability of large-scale PDES, the performance of the simulations must be tuned to reduce the number of rollbacks and provide an improved simulation runtime. To enable efficient large-scale optimistic simulations, one has to gain insight into the factors that affect the rollback behavior and simulation performance. We developed a tool for ROSS model developers that gives them detailed metrics on the performance of their large-scale optimistic simulations at varying levels of simulation granularity. Model developers can use this information for parameter tuning of optimistic simulations in order to achieve better runtime and fewer rollbacks. In this work, we instrument the ROSS optimistic PDES framework to gather detailed statistics about the simulation engine. We have also developed an interactive visualization interface that uses the data collected by the ROSS instrumentation to understand the underlying behavior of the simulation engine. The interface connects real time to virtual time in the simulation and provides the ability to view simulation data at different granularities. We demonstrate the usefulness of our framework by performing a visual analysis of the dragonfly network topology model provided by the CODES simulation framework built on top of ROSS. The instrumentation needs to minimize overhead in order to accurately collect data about the simulation performance. To ensure that the instrumentation does not introduce unnecessary overhead, we perform a scaling study that compares instrumented ROSS simulations with their noninstrumented counterparts in order to determine the amount of perturbation when running at different simulation scales.

Discrete Event Simulation (DES) is a widely used technique in which the state of the simulator is updated by events happening at discrete points in time (hence the name). DES is used to model and analyze many kinds of systems, including computer architectures, communication networks, street traffic, and others. Parallel and Distributed Simulation (PADS) aims at improving the efficiency of DES by partitioning the simulation model across multiple processing elements, in order to enabling larger and/or more detailed studies to be carried out. The interest on PADS is increasing since the widespread availability of multicore processors and affordable high performance computing clusters. However, designing parallel simulation models requires considerable expertise, the result being that PADS techniques are not as widespread as they could be. In this paper we describe ErlangTW, a parallel simulation middleware based on the Time Warp synchronization protocol. ErlangTW is entirely written in Erlang, a concurrent, functional programming language specifically targeted at building distributed systems. We argue that writing parallel simulation models in Erlang is considerably easier than using conventional programming languages. Moreover, ErlangTW allows simulation models to be executed either on single-core, multicore and distributed computing architectures. We describe the design and prototype implementation of ErlangTW, and report some preliminary performance results on multicore and distributed architectures using the well known PHOLD benchmark.

The large size and complexity of the modern digital hardware impose great challenges to design and validation. Hardware Description Languages (HDLs) and System-Level Description Languages (SLDLs) rely on sequential discrete event semantics. Parallel Discrete Event Simulation (PDES) has recently gained extensive attention for parallelizing these languages due to the ever-increasing complexity of embedded and cyber physical systems. However, PDES application has not yet reached acceptable maturity and pervasiveness for accelerating computer architecture problems. This is due to inherent complexity of hardware components that require using different advanced PDES techniques. In this paper, we look at the main problem from a radically different angle. First, we suppose there is only a single universal discrete event model of computation for simulation purely defined by distributed optimistic PDES, i.e., logical-process-based event scheduling worldview. Second, we construct a new parallel system-level simulation language called OSML for ESL. Third, we propose a unified Cloud-based CAD tool called Troodon to automatically parallelize existing hardware languages atop OSML. To the best of our knowledge, OSML is the first work on optimistic synchronization applied to hardware-specific SLDLs and hardware models at different levels of abstraction in ESL, which contain complex data structures by proposing a hybrid checkpointing scheme.

Parallel discrete event simulation (PDES), sometimes called distributed simulation, refers to the execution of a single discrete event simulation program on a parallel computer. PDES has attracted ...

In contrast to traditional Parallel Discrete Event Simulation (PDES) systems using dedicated machines, an alternative master/worker approach for high throughput execution on shared computing platforms is proposed. Master/worker systems offer potential advantages over traditional PDES systems such as better support for heterogeneous machines, client-driven load balancing, fault tolerance, and execution on non-dedicated machines. We examine feasibility of executing PDES codes on master/worker systems. We characterise application properties that are important for efficient execution and define suitable performance metrics. The Aurora Parallel and Distributed Simulation System utilising the master/worker approach is described along with performance measurements in executing conservative PDES codes.

One of the major overheads that prohibits the wide spread deployment of parallel discrete event simulation (PDES) is the need to synchronize the distributed processes in the simulation. Considerable investigations have been conducted to analyze and optimize the two widely used synchronization strategies, namely the conservative and the optimistic simulation paradigms. However, little attention has been focussed on the definition and strictness of causality. Does causality need to be preserved in all types of simulations? Previously, we had suggested an answer to this question. We had argued that significant performance gains can be achieved by reconsidering this definition to decide if the parallel simulation really needs to subscribe to the preservation of causality. In this paper, we investigate this issue even more closely. An in depth analysis using several example simulation models is presented in this paper. In addition, a comparative analysis between unsynchronized and Time Warp simulation is presented.

Parallel discrete event simulation (PDES) is an important application in use in many DoD projects; for example, PDES is used in large-scale war-gaming, and in complex system design, analysis and verification. Improving PDES performance and capacity allows faster simulation times and more extensive analysis of more detailed models. These benefits are not application-specific: they should reflect to any application that uses the improved simulation kernel. In this work, we overview our efforts for optimizing PDES in a heterogeneous high performance computing (HHPC) environment. We profile the SPEEDES simulator and identify several opportunities. We report on our experiences on two fronts: (1) optimizing the communication subsystem - a critical system for PDES since it is a fine-grained application and (2) exploring the use of augmented FPGA boards to accelerate simulation. While such approaches have been attempted for sequential and data path intensive applications, we believe that their use in clustered environments is novel. Both efforts are works in progress; we present our designs and some preliminary analysis results. For example, removing the centralized communication server from event message exchange path with a number of other small improvements to the simulation cycle, improved performance by an average of 20% performance improvement for one of our large benchmarks.

Parallel discrete event simulation (PDES) decreases a simulation's runtime by splitting the simulation's work between multiple processors. Many users avoid PDES because it is difficult to specify a large and complicated model using existing PDES tools. In this paper we describe how the PARASOL PDES system uses migrating user level threads to support the process interaction world view. The process interaction world view is popular in sequential simulation languages and is a major departure from the logical process view supported by most PDES systems.

As network simulations scale to hundreds of thousands of nodes, parallel discrete event simulation (PDES) has become indispensable for sustaining performance—yet its efficacy hinges on frequent time synchronization steps. Existing synchronization algorithms suffer from the inter-machine communication overhead in distributed environments, eroding the benefits of parallelism. We observe that time synchronization in PDES often involves frequent allreduce operations, and that in-network computing has the potential to substantially accelerate such collectives. In this work, we present SINA (Synchronization using In-Network Allreduce)—the first integration of in-network computing into time synchronization to address the demands of large-scale simulations. We implemented SINA in a testbed with Mellanox SHArP-enabled switches and ConnectX-5 network cards, offloading allreduce to the network hardware while preserving software‑level correctness. Our evaluation shows that SINA achieves up to 88.6% acceleration compared to state‑of‑the‑art methods and achieves up to 67.5% optimization in topologies of tens of thousands of nodes, demonstrating its suitability for high‑performance, large‑scale parallel simulations.

Virtual machine (VM) technologies, especially those offered via Cloud platforms, present new dimensions with respect to performance and cost in executing parallel discrete event simulation (PDES) applications. Due to the introduction of overall cost as a metric, the choice of the highest-end computing configuration is no longer the most economical one. Moreover, runtime dynamics unique to VM platforms introduce new performance characteristics, and the variety of possible VM configurations give rise to a range of choices for hosting a PDES run. Here, an empirical study of these issues is undertaken to guide an understanding of the dynamics, trends and trade-offs in executing PDES on VM/Cloud platforms. Performance results and cost measures are obtained from actual execution of a range of scenarios in two PDES benchmark applications on the Amazon Cloud offerings and on a high-end VM host machine. The data reveals interesting insights into the new VM-PDES dynamics that come into play and also leads to counter-intuitive guidelines with respect to choosing the best and second-best configurations when overall cost of execution is considered. In particular, it is found that choosing the highest-end VM configuration guarantees neither the best runtime nor the least cost. Interestingly, choosing a (suitably scaled) low-end VM configuration provides the least overall cost without adversely affecting the total runtime.

The master/worker (MW) paradigm can be used to implement parallel discrete event simulations (PDES) on metacomputing systems. MW PDES applications incur overheads not found in conventional PDES executions executing on tightly coupled machines. We introduce four techniques for reducing these overheads on public resource and desktop grid infrastructures Work unit caching, pipelined state updates, expedited message delivery, and adaptive work unit scheduling mechanisms are described that provide significant reduction in overall overhead when used in tandem. We present performance results showing that an optimized MW PDES system can exhibit performance comparable to a traditional PDES system for a queueing network and a particle physics simulation.

Null message algorithm (NMA) is one of the efficient conservative time management algorithms that use null messages to provide synchronization between the logical processes (LPs) in a parallel discrete event simulation (PDES) system. However, the performance of a PDES system could be severely degraded if a large number of null messages need to be generated by LPs to avoid deadlock. In this paper, we present a mathematical model based on the quantitative criteria specified in (Rizvi et al., 2006) to optimize the performance of NMA by reducing the null message traffic. Moreover, the proposed mathematical model can be used to approximate the optimal values of some critical parameters such as frequency of transmission, Lookahead (L) values, and the variance of null message elimination. In addition, the performance analysis of the proposed mathematical model incorporates both uniform and non-uniform distribution of L values across multiple output lines of an LP. Our simulation and numerical analysis suggest that an optimal NMA offers better scalability in PDES system if it is used with the proper selection of critical parameters.

The performance and scalability of Parallel Discrete Event Simulation (PDES) is often limited by communication latencies and overheads. The emergence of multi-core processors and their expected evolution into many-cores offers the promise of low latency communication and tight memory integration between cores; these properties should significantly improve the performance of PDES in such environments. However, on clusters of multi-cores (CMs), the latency and processing overheads incurred when communicating between different machines (nodes) far outweigh those between cores on the same chip, especially when commodity networking fabrics and communication software are used. It is unclear if there is any benefit to the low latency among cores on the same node given that communication links across nodes are significantly worse. In this study, we examine the performance of a multi-threaded implementation of PDES on CMs. We demonstrate that the inter-node communication costs impose a substantial bottleneck on PDES and demonstrate that without optimizations addressing these long latencies, multi-threaded PDES does not significantly outperform the multiprocess version despite direct communication through shared memory on the individual nodes. We then propose three optimizations: message consolidation and routing, infrequent polling and latency-sensitive model partitioning. We show that with these optimizations in place, threaded implementation of PDES significantly outperforms process-based implementation even on CMs.