Statistical Model Checking Using Perfect Simulation

Transaction‐level modeling with SystemC has been very successful in describing the behavior of embedded systems by providing high‐level executable models, in which many of them have inherent probabilistic behaviors, eg, random data and unreliable components. It is thus crucial to have both quantitative and qualitative analysis of the probabilities of system properties.Such analysis can be conducted by constructing a formal model of the system under verification and using Probabilistic Model Checking. However, this method is infeasible for large systems, due to the state space explosion. In this article, we demonstrate the successful use of statistical model checking to conduct such analysis directly from large SystemC models and allow designers to express a wide range of useful properties.The first contribution of this work is a framework to verify properties expressed in Bounded Linear Temporal Logic for SystemC models with both timed and probabilistic characteristics.Second, the framework allows users to expose a rich set of user code primitives as atomic propositions in Bounded Linear Temporal Logic. Moreover, users can define their own fine‐grained time resolution rather than the boundary of clock cycles in the SystemC simulation.The third contribution is an implementation of a statistical model checker. It contains an automatic monitor generation for producing execution traces of the model‐under‐verification, the mechanism for automatically instrumenting the model‐under‐verification, and the interaction with statistical model checking algorithms.

Neural networks (NN) are taking over ever more decisions thus far taken by humans, even though verifiable system-level guarantees are far out of reach. Neither is the verification technology available, nor is it even understood what a formal, meaningful, extensible, and scalable testbed might look like for such a technology. The present paper is an attempt to improve on both the above aspects. We present a family of formal models that contain basic features of automated decision-making contexts and which can be extended with further orthogonal features, ultimately encompassing the scope of autonomous driving. Due to the possibility to model random noise in the decision actuation, each model instance induces a Markov decision process (MDP) as verification object. The NN in this context has the duty to actuate (near-optimal) decisions. From the verification perspective, the externally learnt NN serves as a determinizer of the MDP, the result being a Markov chain which as such is amenable to statistical model checking. The combination of an MDP and an NN encoding the action policy is central to what we call “deep statistical model checking” (DSMC). While being a straightforward extension of statistical model checking, it enables to gain deep insight into questions like “how high is the NN-induced safety risk?”, “how good is the NN compared to the optimal policy?” (obtained by model checking the MDP), or “does further training improve the NN?”. We report on an implementation of DSMC inside the ModestToolset in combination with externally learnt NNs, demonstrating the potential of DSMC on various instances of the model family, and illustrating its scalability as a function of instance size as well as other factors like the degree of NN training.

Fault tree (FT) is a standardized notation for representing relationships between a system’s reliability and the faults and/or the events associated with it. However, the existing FT fault models are only capable of portraying permanent events in the system. This is a major hindrance since these models fail to reflect accurately the other classes of faults, such as soft-faults, which are often temporary events that usually disappear after the source of the interference is no longer present. This paper proposes a new fault tree modeling paradigm, to capture the impact of temporal events in systems, called temporal dynamic fault trees (TDFTs) . TDFTs are utilized to model the characteristics and dependencies between different temporal events, soft-faults, and permanent faults. These features are integrated into the proposed probabilistic models of the temporal gates, which are modeled as priced-timed automata . This paper also proposes a new FT analysis methodology, based on statistical model checking , designed to circumvent the state-explosion problem that is inherent to other model-checking approaches. The proposed analysis is able to evaluate the impact of temporal faults in systems, as well as to estimate the reliability and availability of the system over extended periods of time. The experiments reported in this paper demonstrate the versatility and scalability of the proposed approach. For instance, the results display the impact that temporal events may have in a digital system. Our observations indicate that while regular soft-fault analyses tend to underestimate metrics such as system reliability, TDFT analysis shows remarkable consistency with radiation testing, with differences of under 2%, in the conducted analysis.

A parameter generation method based on model checking is proposed to tackle the parameter selection of hierarchical scheduling systems in Integrated Modular Avionics (IMA) by combining the classical symbolic model checking and the Statistical Model Checking (SMC). It builds a generic timed automata network to describe the temporal behavior of hierarchical systems. A distributed genetic algorithm is adopted to search the optimum partition parameters with respect to processor utilization while guaranteeing the schedulability of the system, which is formulated as safety properties of symbolic model checking and hypothesis testing of SMC. Comparing with the widely-used response time analysis, the formal model of this method is more expressive to cover complex features. The application of SMC alleviates the "state space explosion" of classical model checking. Finally, the parameter generation experiments show that the present method is able to find the global optimum solutions in the parameter space.

Rare event simulation for highly dependable systems with fast repairs

The paper presents the existing verification methods for control algorithms in power electronics systems, including the application of model checking techniques. In the industry, the most frequently used verification methods are simulations and experiments; however, they have to be performed manually and do not give a 100% confidence that the system will operate correctly in all situations. Here we show the recent advancements in verification and performance assessment of power electronics systems with the usage of formal methods. Symbolic model checking can be used to achieve a guarantee that the system satisfies user-defined requirements, while statistical model checking combines simulation and statistical methods to gain statistically valid results that predict the behavior with high confidence. Both methods can be applied automatically before physical realization of the power electronics systems, so that any errors, incorrect assumptions or unforeseen situations are detected as early as possible. An additional functionality of verification with the use of formal methods is to check the converter operation in terms of reliability in various system operating conditions. It is possible to verify the distribution and uniformity of occurrence in time of the number of transistor switching, transistor conduction times for various current levels, etc. The information obtained in this way can be used to optimize control algorithms in terms of reliability in power electronics. The article provides an overview of various verification methods with an emphasis on statistical model checking. The basic functionalities of the methods, their construction, and their properties are indicated.

Statistical model checking (SMC) for the analysis of multi-agent systems has been studied in the recent past. A feature peculiar to multi-agent systems in the context of statistical model checking is that of aggregate queries–temporal logic formula that involves a large number of agents. To answer such queries through Monte Carlo sampling, the statistical approach to model checking simulates the entire agent population and evaluates the query. This makes the simulation overhead significantly higher than the query evaluation overhead. This problem becomes particularly challenging when the model checking queries involve multiple attributes of the agents. To alleviate this problem, we propose a population sampling algorithm that simulates only a subset of all the agents and scales to multiple attributes, thus making the solution generic. The population sampling approach results in increased efficiency (a gain in running time of 50%–100%) for a marginal loss in accuracy (between 1% and 5%) when compared with the exhaustive approach (which simulates the entire agent population to evaluate the query), especially for queries that involve limited time horizons. Finally, we report parallel versions of the above algorithms. We explore different strategies of core allocation, both for exhaustive simulations of all agents and the sampling approach. This yields further gains in running time, as expected. The parallel approach, when combined with the sampling idea, results in improving the efficiency (a gain in running time of 100%–150%) with a minor loss when compared with the exhaustive approach in accuracy (between 1% and 5%).

In the railway domain, an interlocking is the system ensuring safe train traffic inside a station by controlling its active elements such as the signals or points. Modern interlockings are configured using particular data, called application data, reflecting the track layout and defining the actions that the interlocking can take. The safety of the train traffic relies thereby on application data correctness, errors inside them can cause safety issues such as derailments or collisions. Given the high level of safety required by such a system, its verification is a critical concern. In addition to the safety, an interlocking must also ensure that availability properties, stating that no train would be stopped forever in a station, are satisfied. Most of the research dealing with this verification relies on model checking. However, due to the state space explosion problem, this approach does not scale for large stations. More recently, a discrete event simulation approach limiting the verification to a set of likely scenarios, was proposed. The simulation enables the verification of larger stations, but with no proof that all the interesting scenarios are covered by the simulation. In this paper, we apply an intermediate statistical model checking approach, offering both the advantages of model checking and simulation. Even if exhaustiveness is not obtained, statistical model checking evaluates with a parametrizable confidence the reliability and the availability of the entire system.

Statistical Model Checking (SMC) is a powerful and widely used approach that consists in estimating the probability for a system to satisfy a temporal property. This is done by monitoring a finite number of executions of the system, and then extrapolating the result by using statistics. The answer is correct up to some confidence that can be parameterized by the user. It is known that SMC mitigates the state-space explosion problem and allows to approximate undecidable queries. The approach has been implemented in several toolsets such as Plasma Lab, and successfully applied in a wide range of diverse areas such as systems biology, robotic, or automotive. In this paper, we add two new modest contributions to the cathedral of results on SMC. The first contribution is an algorithm that can be used to monitor changes in the probability distribution to satisfy a bounded-time property at runtime. Concretely, the algorithm constantly monitors the execution of the deployed system, and raises a flag when it observes that the probability has changed significantly. This is done by extending the applicability of the CUSUM algorithm used in signal processing into the formal validation setting. Our second contribution is to show how the programming interface of Plasma Lab can be exploited in order to make SMC technology directly available in toolsets used by designers. This integration is done by exploiting simulation facilities of design tools. Our approach thus differs from the one adopted by other SMC/formal verification toolsets which assume the existence of formal semantics for the design language, as well as a compiling chain to the rather academic one used by validation tool. The concept results in the integration of Plasma Lab as a library of the Simulink toolset. The contributions are illustrated by using Plasma Lab to verify a Simulink case study modelling a pig shed temperature controller.

Cyber-Physical Systems (CPSs) as advanced embedded systems integrating computation with physical process are increasingly penetrating into our life. Modeling and analysis for such systems closely involved with us are actively researched. A current challenging problem is how to take advantages of existing technologies like SysML/MARTE, Modelica and Statistical Model Checking (SMC) through effective integration. Moreover, the lack of efficient methodologies or tools for modeling and analysis of CPSs makes the gap between design and analysis models hard to bridge. To solve these problems, we present a framework named Modana to achieve an integrated process from modeling with SysML/MARTE to analysis with SMC for CPSs in terms of Non Functional Properties (NFP) such as time, energy, etc. Functional Mock-up Interface (FMI), as a connecting link between modeling and analysis, plays a major role in coordinating various tools for co-simulation to generate traces as the input of statistical model checker. To demonstrate the capability of Modana framework, we model energy-aware buildings as a case study, and discuss the analysis on energy consumption in different scenarios.

Article Free Access Share on Exact sampling and approximate counting techniques Author: Mark Huber School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NY School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NYView Profile Authors Info & Claims STOC '98: Proceedings of the thirtieth annual ACM symposium on Theory of computingMay 1998 Pages 31–40https://doi.org/10.1145/276698.276709Published:23 May 1998Publication History 25citation666DownloadsMetricsTotal Citations25Total Downloads666Last 12 Months66Last 6 weeks15 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Creol is an executable, formally defined modeling language with advanced object-oriented features, tailored for modeling software systems consisting of physically distributed components, each running on its own processor and communicating by means of asynchronous method calls. Creol is oriented towards qualitative analysis, abstracting away from underlying software and hardware system properties. In this paper we propose a probabilistic extension of Creol, oriented towards quantitative analysis, called PCreol, and give its operational semantics by means of probabilistic rewrite theories. This extension is motivated by the need to model a) communication over networks of different quality, b) software components running randomized algorithms, c) independent processor speeds, d) process scheduling, and e) an open environment exhibiting probabilistic behavior. The syntax of PCreol therefore includes means for modeling random expressions, a probabilistic choice operator, random assignment, call statements with priorities, and object creation statements that set an object's computation speed, as well as its quality of communication with other objects. We give details on the implementation of a prototype PCreol interpreter--on top of the existing one for Creol--which is executable in Maude. Furthermore, we integrate PCreol with the VeStA tool, to support probabilistic reasoning of PCreol models by statistical model checking and quantitative analysis. By probabilistic modeling of independent processor speeds, representative runs of a PCreol model are obtained using discrete-event simulation, and the model checking problem of large models becomes feasible. We provide concrete examples of PCreol models and show how VeStA can be used to reason about them.

This paper presents efficient model checking of distributed software. Key to the achieved efficiency is a novel stateful model checking strategy that is based on the decomposition of states into a relevant and an auxiliary part. We formally show this strategy to be sound, complete, and terminating for general finite-state systems. As a case study, we implement the proposed strategy within Basset/MP-Basset, a model checker for message-passing Java programs. Our evaluation with actual deployed fault-tolerant message-passing protocols shows that the proposed stateful optimization is able to reduce model checking time and memory by up to 69% compared to the naive stateful search, and 39% compared to partial-order reduction.

Schistosomiasis is a parasitic disease that is currently endemic in more than 70 countries with the bulk of infections concentrated in Africa. The interest in schistosomiasis has recently grown due to the commitment of substantial amounts of funding to the control of the so-called neglected tropical diseases. Targeting of interventions and allocation of financial resources should be driven by evidenced-based information on the spatial distribution and schistosomiasis burden estimates in order to increase cost-effectiveness and to meet local needs. Currently, decisions are mainly made based on crude schistosomiasis risk estimates which are largely obsolete due to ongoing control efforts, ecological transformations, demographic changes and improved hygiene, among other reasons. Schistosomiasis transmission depends on the distribution of intermediate host snail species. This distribution is determined by climatic and other environmental conditions, such as temperature, precipitation or water flow velocity. Statistical models can be used to establish the relation between the aforementioned factors and schistosomiasis risk and to predict the risk at unobserved locations. Empirical risk mapping requires observed prevalence data distributed within the area of interest, however contemporary large-scale surveys are not available. To address this issue, the European Union (EU)-funded CONTRAST project initiated the development of the Global Neglected Tropical Disease (GNTD) database. To-date, the GNTD database is the most comprehensive schistosomiasis database in Africa. Schistosomiasis prevalence data are spatially correlated, because locations in close proximity share common spatial exposures, which similarly influence transmission. Standard statistical models are not appropriate because they assume independence between locations, leading to imprecise parameter estimates and risk predictions. Geostatistical models assume take into account potential spatial correlation by introducing location-specific random effects. Bayesian model formulations, implemented via Markov chain Monte Carlo (MCMC) simulations methods, enable model fit overcoming the computational problems of likelihood-based methods. Geostatistical model fit requires the repeated inversion of the correlation matrix of the spatial process. The size of this matrix increases with the number of locations and, for very large number of locations, matrix inversion is infeasible. An important aspect of geostatistical model fit is the choice of predictors driving schistosomiasis transmission. There are a number of environmental factors which are correlated, complicating model fit. Rigorous geostatistical variable selection has not yet been applied in spatial schistosomiasis epidemiology. Data compilations contain heterogeneous surveys across locations in terms of age groups involved and diagnostic methods used. The lack of prevalence data reported in standard age groups complicates the estimation of age-adjusted schistosomiasis risk. A common assumption of geostatistical models is that of isotropy implying that spatial correlation is a function of distance between locations irrespective of direction or location. However in schistosomiasis risk mapping, spatial correlation is likely to be related to the direction of river flow due to water-dependent intermediate host snail species. This might introduce directional dependency (anisotropy). Schistosomiasis tends to be present in areas with other neglected diseases. Cost-effective interventions call for an integrated disease control, which requires estimating of the geographical distribution of high co-endemicity. The co-endemic diseases might be correlated, however surveys screening for multiple diseases are not available over large geographical areas. There rather exist data from independent surveys screening for single diseases on different sets of individuals. The main contributions of this thesis were (i) the development of Bayesian isotropic and anisotropic geostatistical models for high spatial resolution schistosomiasis risk mapping and prediction based on age-heterogeneous historical survey data collected over very large number of locations; (ii) the development of statistical methodology for assessing the geographical distribution of co-infection risk from independent single-disease surveys; and (iii) the estimation of location-specific schistosomiasis risk and number of infected people in 29 countries across West and eastern Africa. Hence, for first time, empirical model-based evidence of schistosomiasis risk and burden in those regions is provided. These estimates are of considerable importance for schistosomiasis control programs, as they indicate high-risk areas requiring interventions, allow calculations of the number of drugs required based on WHO guidelines, and provide baseline maps to assess effectiveness of interventions on the roadmap towards schistosomiasis elimination.

Markov automata (MA) constitute an expressive continuous-time compositional modelling formalism. They appear as semantic backbones for engineering frameworks including dynamic fault trees, Generalised Stochastic Petri Nets, and AADL. Their expressive power has thus far precluded them from effective analysis by probabilistic (and statistical) model checkers, stochastic game solvers, or analysis tools for Petri net-like formalisms. This paper presents the foundations and underlying algorithms for efficient MA modelling, reduction using static analysis, and most importantly, quantitative analysis. We also discuss implementation pragmatics of supporting tools and present several case studies demonstrating feasibility and usability of MA in practice.