Global Virtual Time and distributed synchronization

Global virtual time (GVT) is used in the Time Warp synchronization mechanism to perform irrevocable operations such as I/O and to reclaim storage. Most existing algorithms for computing GVT assume a message-passing programming model. Here, GVT computation is examined in the context of a shared-memory model. We observe that computation of GVT is much simpler in shared-memory multiprocessors because these machines normally guarantee that no two processors will observe a set of memory operations as occurring in different orders. Exploiting this fact, we propose an efficient, asynchronous, shared-memory GVT algorithm and prove its correctness. This algorithm does not require message acknowledgments, special GVT messages, or FIFO delivery of messages, and requires only a minimal number of shared variables and data structures. The algorithm only requires one round of interprocessor communication to compute GVT, in contrast to many message-based algorithms that require two. An efficient implementatin is described that eliminates the need for a processor to explicitly compute a local minimum for time warp systems using a lowest-timestamp-first scheduling policy in each processor. In addition, we propose a new mechanism called on-the-fly fossil collection that enables efficient storage reclamation for simulations containing large numbers, e.g., hundreds of thousand or even millions of simulator objects. On-the-fly fossil collection can be used in time warp systems executing on either shared-memory or message-based machines. Performance measurements of the GVT algorithm and the on-the-fly fossil collection mechanism on a Kendall Square Research KSR-2 machine demonstrate that these techniques enable frequent GVT and fossil collections, e.g., every millisecond, without incurring a significant performance penalty

In this paper, we investigate the performance of Parallel Discrete Event Simulation (PDES) on a cluster of many-core Intel KNL processors. Specifically, we analyze the impact of different Global Virtual Time (GVT) algorithms in this environment and contribute three significant results. First, we show that it is essential to isolate the thread performing MPI communications from the task of processing simulation events, otherwise the simulation is significantly imbalanced and performs poorly. This applies to both synchronous and asynchronous GVT algorithms. Second, we demonstrate that synchronous GVT algorithm based on barrier synchronization is a better choice for communication-dominated models, while asynchronous GVT based on Mattern's algorithm performs better for computation-dominated scenarios. Third, we propose Controlled Asynchronous GVT (CA-GVT) algorithm that selectively adds synchronization to Mattern-style GVT based on simulation conditions. We demonstrate that CA-GVT outperforms both barrier and Mattern's GVT and achieves about 8% performance improvement on mixed computation-communication models. This is a reasonable improvement for a simple modification to a GVT algorithm.

Global virtual time (GVT) is used in parallel discrete event simulations to reclaim memory, commit output, detect termination, and handle errors. Mattern 's [I] has proposed G VT approximation with distributed termination detection algorithm. This algorithm works fine and gives optimal performance in terms of accurate GVT computation at the expense of slower execution rate. This slower execution rate results a high GVT latency. Due to the high GVT latency, the processors involve in communication remain idle during that period of time. As a result, the overall throughput of a discrete event parallel simulation system degrades significantly. Thus, the high G VT latency prevents the widespread use of this algorithm in discrete event parallel simulation system. However, if we could improve the latency of GVT computation, most of the discrete event parallel simulation system would likely take advantage of this technique in terms of accurate G VT computation. In this paper, we examine the potential use of tress and butterflies barriers with the Mattern's GVT structure using a ring. Simulation results demonstrate that the use of tree barriers with the Mattern's GVT structure can significantly improve the latency time and thus increase the overall throughput of the parallel simulation system. The performance measure adopted in this paper is the achievable latency for a fixed number of processors and the number of message transmission during the G VT computation.

Recent studies investigated the performance of Parallel Discrete Event Simulation (PDES) on Intel Xeon Phi many-core processors, but generally reported underwhelming performance results, especially at high scales when all cores and thread contexts are fully loaded. While the lack of scalability in an earlier study on a Knights Corner (KC) processor is an artifact of physical limitations of the KC system, performance challenges on a Knights Landing (KNL) system partially stem from a slower global virtual time (GVT) computation algorithm used in that study. In this paper, we re-examine PDES performance on KNL under more efficient GVT algorithms to alleviate the GVT bottleneck. Specifically, we compare a synchronous GVT algorithm based on barrier synchronization, and two asynchronous GVT implementations: a modified Mattern's algorithm for shared memory systems and a recently-proposed wait-free algorithm. Using the ROSS simulator, we demonstrate that minimizing the GVT bottleneck results in significant improvement in scalability, allowing the simulation to scale with performance all the way to 250 threads (per chip). Interestingly, we observe that while for the balanced models the wait-free algorithm is a clear winner, barrier-based GVT provides significantly better results for imbalanced models executed at high scale. We also perform detailed simulation profiling to understand the underlying reasons for these performance trends.

The increasing diffusion of shared-memory multi-core machines has given rise to a change in the design of Parallel Discrete Event Simulation (PDES) platforms. In particular, the possibility to share large amounts of memory by many worker threads has lead to a boost in the adoption of non-blocking coordination algorithms, which have been proven to offer higher scalability when compared to their blocking counterparts based on critical sections. In this article we present an innovative non-blocking algorithm for computing Global Virtual Time (GVT)---namely, the current commit horizon---in multi-thread PDES engines to be run on top of multi-core machines. Beyond being non-blocking, our proposal has the advantage of providing a logarithmic (rather than linear) number of per-thread memory operations---read/write operations of values involved in the reduction for computing the GVT value---vs the amount of threads participating in the GVT computation. This allows for keeping low the actual CPU time that is required for determining the new GVT value. We compare our algorithm with a literature solution, still based on the non-blocking approach, but entailing a linear number of memory operations, quantifying the advantages from our proposal especially for very large numbers of threads participating in the GVT computation.

This paper introduces a unifying technique that is able to combine the advantages of synchronous, conservative and optimistic simulations. The unifying framework is based on optimistic simulation. The optimized optimistic simulation exploits synchronous parallelism by modifying the definition of GVT (Global Virtual Time). Conservative parallelism is exploited by embedding lookahead computation into the optimistic simulation. In the unified framework, a logical process may execute in synchronous mode, conservative mode and optimistic mode. Synchronous mode and conservative mode executions have smaller overhead than optimistic mode execution. The Benchmark results on logic simulation show that the unified technique performs almost as good as the best case of synchronous simulation and performs better than both synchronous and optimistic simulation when the number of processors is large.

The Air Force is developing a Distributed Information Enterprise Modeling and Simulation (DIEMS) framework under sponsorship of the High Performance Computer Modernization Office Common High Performance Computing Software Support Initiative (HPCMO/CHSSI). The DIEMS framework provides a design analysis environment for deployable distributed information management systems. DIEMS establishes the necessary analysis capability allowing developers to identify and mitigate programmatic risk early within the development cycle to allow successful deployment of the associated systems. The enterprise-modeling framework builds upon the Synchronous Parallel Environment for Emulation and Discrete-Event Simulation (SPEEDES) foundation. This simulation framework will utilize 'Challenge Problem' class resources to address more than five million information objects and hundreds of thousands of clients comprising the future information based force structure. The simulation framework will be capable of assessing deployment aspects such as security, quality of service, and fault tolerance. SPEEDES provides an ideal foundation to support simulation of distributed information systems on a multiprocessor platform. SPEEDES allows the simulation builder to perform optimistic parallel processing on high performance computers, networks of workstations, or combinations of networked computers and HPC platforms.

Fine-Grained Parallel and Distributed Spatial Stochastic Simulation of Biological Reactions

Conceptual Modeling (CM) is a fundamental step in a simulation project. Nevertheless, it is only recently that structured approaches towards the definition and formulation of conceptual models have gained importance in the Discrete Event Simulation (DES) community. As a consequence, frameworks and guidelines for applying CM to DES have emerged and discussion of CM for DES is increasing. However, both the organization of model-components and the identification of behavior and system control from standard CM approaches have shortcomings that limit CM’s applicability to DES. Therefore, we discuss the different aspects of previous CM frameworks and identify their limitations. Further, we present the Hierarchical Control Conceptual Modeling framework that pays more attention to the identification of a models’ system behavior, control policies and dispatching routines and their structured representation within a conceptual model. The framework guides the user step-by-step through the modeling process and is illustrated by a worked example.

The computation of Global Virtual Time is of fundamental importance in Time Warp based Parallel Discrete Event Simulation Systems. Shared memory multiprocessor architectures can support interprocess communication with much smaller overheads than distributed memory systems. This paper presents a new, completely asynchronous, Gvt algorithm which provides very fast and accurate Gvt estimation with significantly lower overhead than previous approaches. The algorithm presented is able to support more efficient memory management, termination, and other global control mechanisms. The Gvt algorithm described enables any Time Warp entity to compute Gvt at any time without slowing down other entities, in particular, those executing on the critical path. Experimental results are presented for a shared memory Time Warp system that employs a two tiered distributed memory management scheme. The proof of the correctness and the accuracy of the algorithm are also presented. Finally, some suggestions on possible further optimization of the implementation are given.

A general framework for dependability modelling coupling discrete-event and time-driven simulation

The Block Oriented Network Simulator (BONeS) DESIGNER provides a graphical environment for modeling and discrete-event simulation of computer and communication networks, distributed computer systems, and computer architectures. DESIGNER uses a hierarchical data flow block diagram modeling paradigm which provides for definition of data structures that flow between blocks. This modeling paradigm provides sufficient flexibility to model and simulate a broad range discrete-event systems, including computer network and distributed processing systems. >

: C++ library classes for Force Structure Simulation (FSS) based on the parallel simulation framework called SPEEDES (Synchronous Parallel Environment for Emulation and Discrete-Event Simulation) was developed. These classes derive from and/or utilize the SPEEDES Application Programming Interface (API), and the SPEEDES simulation methodology for execution on high-performance multiprocessor systems. These FSS classes were designed to be the foundation of a R&D simulation framework for researching predictive course of action analysis. The software will prove an easily extensional means to accommodate an increasingly detailed, powerful and elaborate assessment tool.

Object-Oriented Discrete Event Simulation Modeling Environment for Aerospace Vehicle Maintenance and Logistics Process

This chapter discusses efficient detection of global predicates in a distributed program. Previous work in detection of global predicates was restricted to predicates that could be specified as a boolean formula of local predicates. Many properties in distributed systems, however, use the state of channels. In this chapter, we introduce the concept of a channel predicate and provide an efficient algorithm to detect any boolean formula of local and channel predicates. We define a property called monotonicity for channel predicates. Monotonicity is crucial for efficient detection of global predicates. We show that many problems studied earlier, such as detection of termination and computation of global virtual time are special cases of the problem considered in this chapter. The message complexity of our algorithm is bounded by the number of messages used by the program. The main application of our results are in debugging and testing of distributed programs. Our algorithms have been incorporated in a distributed programming environment which runs on a network of IBM RS/6000 Workstations under AIX with the PVM environment.