Model checking stochastic hybrid systems

Hybrid systems consist of both continuous state evolutions and discrete state (or mode) transitions. In many hybrid systems, mode transitions are governed by guard conditions that are dependent on the continuous state. These kind of mode transitions can be described by continuous-state-dependent mode transition probabilities. However, most multiple model Kalman filter based algorithms, such as the well-known interacting multiple model (IMM) algorithm, assume constant mode transition probabilities. We propose a hybrid estimation algorithm based on the IMM approach for the stochastic hybrid systems with continuous-state-dependent transitions. We first develop two models to describe the guard conditions for various stochastic hybrid systems. From the models of the guard conditions, we derive the corresponding continuous-state-dependent mode transition probabilities, which are expressed in terms of Gaussian probability density functions (pdfs) or Gaussian cumulative density functions (cdfs). We then propose a hybrid estimation algorithm for the stochastic hybrid systems based on the IMM algorithm. Like the IMM algorithm, our algorithm utilizes Gaussian distributions to represent the continuous state pdfs which are updated analytically at each time step. We also use analytical integration methods to reduce the computational cost of updating the mode probabilities. As a result, the proposed algorithm is computationally efficient and is applicable to high-dimensional problems. The performance of the proposed algorithm is illustrated with two examples in air traffic control tracking applications.

Explicit and tractable characterizations of the dynamical behavior of virus particles are pivotal for a thorough understanding of the infection mechanisms of viruses. This thesis deals with the problem of extracting symbolic representations of the dynamical behavior of fluorescent particles from fluorescence microscopy image sequences. The focus is on the behavior of virus particles such as fusion with the cell membrane. A numerical representation is obtained by tracking the particles in the image sequences. We have investigated probabilistic tracking approaches, including approaches based on the Kalman filter as well as based on particle filters. For reasons of efficiency and robustness, we developed a tracking approach based on the probabilistic data association (PDA) algorithm in combination with an ellipsoidal sampling scheme that exploits effectively the image data via parametric appearance models. To track objects in close proximity, we compute the support that each image position provides to each tracked object relative to the support provided to the object's neighbors. After tracking, the problem of mapping the trajectory information computed by the tracking approaches to symbolic representations of the behavior arises. To compute symbolic representations of behaviors related to the fusion of single virus particles with the cell membrane based on their intensity over time, we developed a layered probabilistic approach based on stochastic hybrid systems as well as hidden Markov models (HMMs). We use a maxbelief strategy to efficiently combine both representations. The layered approach describes the intensity, intensity models, and behaviors of single virus particles. We introduce models for the evolution of the intensity and the behavior. To compute estimates for the intensity, intensity models, and behaviors we use a hybrid particle filter and the Viterbi algorithm. The developed approaches have been applied to synthetic images as well as to real microscopy image sequences displaying human immunodeficiency virus (HIV-1) particles. We have performed an extensive quantitative evaluation of the performance and a comparison with several existing approaches. It turned out that our approaches outperform previous ones, thus yielding more accurate and more reliable information about the behavior of virus particles. Moreover, we have successfully applied our tracking approaches to 3D image sequences displaying herpes simplex virus (HSV) replication compartments. We also applied the tracking approaches to image data displaying microtubule tips and analyzed their motion. In addition, our tracking approaches were successfully applied to the 2D and 3D image data of a Particle Tracking Challenge.

Approximate Model Checking of Stochastic Hybrid Systems

This paper provides a compositional approach for approximations of the interconnection of a class of stochastic hybrid systems including both jump linear stochastic systems and linear stochastic hybrid automata. The approximation is based on the recently developed notions of stochastic (bi)simulation functions using which one can quantify the error between original stochastic hybrid systems and their abstractions. Given stochastic (bi)simulation functions between stochastic hybrid subsystems and their corresponding approximations, we provide a stochastic (bi)simulation function between an interconnection of those stochastic hybrid subsystems and that of their corresponding approximations. Consequently, one can leverage the proposed results in this paper to perform the analysis and synthesis over the interconnection of abstract subsystems and then carry the results over the interconnection of concrete ones. We illustrate the effectiveness of the proposed results here by computing a stochastic (bi)simulation function between an interconnection of two identical jump linear stochastic subsystems and that of their corresponding approximations by just using the stochastic (bi)simulation function between one of the jump linear stochastic subsystems and its corresponding approximate abstraction.

A stochastic hybrid system contains a collection of interacting discrete and continuous components, subject to random behaviour. The formal verification of a stochastic hybrid system often comprises a method for the generation of a finite-state probabilistic system which either represents exactly the behaviour of the stochastic hybrid system, or which approximates conservatively its behaviour. We extend such abstraction-based formal verification of stochastic hybrid systems in two ways. Firstly, we generalise previous results by showing how bisimulation-based abstractions of non-probabilistic hybrid automata can be lifted to the setting of probabilistic hybrid automata, a subclass of stochastic hybrid systems in which probabilistic choices can be made with respect to finite, discrete alternatives only. Secondly, we consider the problem of obtaining approximate abstractions for discrete-time stochastic systems in which there are continuous probabilistic choices with regard to the slopes of certain system variables. We restrict our attention to the subclass of such systems in which the approximate abstraction of such a system, obtained using the previously developed techniques of Fraenzle et al., results in a probabilistic rectangular hybrid automaton, from which in turn a finite-state probabilistic system can be obtained. We illustrate this technique with an example, using the probabilistic model checking tool PRISM.

This work is concerned with the safety controller synthesis of stochastic hybrid systems, in which continuous evolutions are described by stochastic differential equations with both Brownian motions and Poisson processes, and instantaneous jumps are governed by stochastic difference equations with additive noises. Our proposed framework leverages the notion of control barrier certificates (CBC), as a discretization-free approach, to synthesize safety controllers for stochastic hybrid systems while providing safety guarantees in finite time horizons. In our proposed scheme, we first provide an augmented framework to characterize each stochastic hybrid system containing continuous evolutions and instantaneous jumps with a unified system covering both scenarios. We then introduce an augmented control barrier certificate (ACBC) for augmented systems and propose sufficient conditions to construct an ACBC based on CBC of original hybrid systems. By utilizing the constructed ACBC, we quantify upper bounds on the probability that the stochastic hybrid system reaches certain unsafe regions in a finite time horizon. The proposed approach is verified over a nonlinear case study.

Static Output-Feedback Ή<inf>∞</inf>Control of a Class of Stochastic Hybrid Systems with Wiener Process

This chapter constitutes a smooth introduction in the field of hybrid systems, discussing issues like modelling classes, analysis and verification, how randomness appears and which stochastic models can be used to capture it. The chapter starts with the presentation of models for deterministic hybrid systems considering two main approaches: the automata perspective and the dynamical system perspective. Then existing algorithmic (model checking) and deductive (theorem proving) techniques for hybrid systems are briefly discussed. The main body of this chapter is dedicated to the description of stochastic models that result by randomising different elements in the structure of classical hybrid systems. Classes of stochastic hybrid systems (like probabilistic hybrid automata, stochastic hybrid automata, switching diffusion processes, general stochastic hybrid automata) are briefly exposed and comparison criteria are underlined.

Automated verification and synthesis of stochastic hybrid systems: A survey

This paper is focused on adapting symmetry reduction, a technique that is highly successful in traditional model checking, to stochastic hybrid systems. We first show that performability analysis of stochastic hybrid systems can be reduced to a stochastic reachability analysis (SRA). Then, we generalize the notion of symmetry reduction as recently proposed for probabilistic model checking, to continuous probabilistic systems. We provide a rigorous mathematical foundation for the reduction technique in the continuous case and also investigate its observability perspective. For stochastic hybrid systems, characterizations for this reduction technique are provided, in terms of their infinitesimal generator.

This dissertation examines an M[x]/G/1 queueing system with a randomized vacation policy and at most J vacations. Whenever the system is empty, the server immediately takes a vacation. If there is at least one customer found waiting in the queue upon returning from a vacation, the server will be immediately activated for service. Otherwise, if no customers are waiting for service at the end of a vacation, the server either remains idle with probability p or leaves for another vacation with probability 1-p. This pattern continues until the number of vacations taken reaches J. If the system is empty by the end of the Jth vacation, the server becomes idle in the system until at least one customer waiting in the queue. In this dissertation, we investigate the following three queueing systems: Reliable server queueing system, un-reliable server queueing system and un-reliable server with a delayed repair queueing system. For the three systems considered in our dissertation, using the supplementary technique, we develop the system size distribution as well as other important system characteristics, such as the system size distribution at busy period initiation epoch, the queue size distribution at a departure epoch, and the distributions of busy period and idle period, etc. Further, for the un-reliable server we also develop main reliability indices of the presented model. Using the renewal reward theorem, a cost model is constructed to determine the optimal randomized vacation policy. Based on the cost model, a heuristic approach is provided to search the joint optimum values of p and J. Some numerical results are presented for illustrative purpose. Our study presents an extension of the existing vacation queueing model and the analysis of the proposed model will provide a useful performance evaluation tool for more general situations arising in real word.

The hybrid estimation problem involves computation of both the continuous state and the discrete state estimates of a hybrid system from the measurements. In our earlier work, we have developed an algorithm, called the State-Dependent-Transition Hybrid Estimation (STDHE) algorithm which treats the discrete-state transitions to be dependent on the continuous state and governed by guard conditions in the linear form. This paper presents a hybrid estimation algorithm for the hybrid system whose guard conditions are in the quadratic form. The algorithm finds its applications in many areas such as Air Traffic Control (ATC), unmanned aerial vehicles coordination and control, and robot control. Simulation results show that our algorithm gives high estimation accuracy similar to that given by the Monte Carlo integration-based estimation algorithm while it is much more computationally efficient than the sample-based algorithms.

A stochastic hybrid system of a form arising in various applications and system formulations (e.g. power, manufacturing, fault-tolerant control) is considered. The most common approach in control design for hybrid systems with a random jump parameter is to use optimal control methods. We study risk-sensitive control for hybrid systems with a Markovian jump parameter. In particular, the objective of the control is to minimize an infinite-horizon risk-sensitive cost functional. It is shown that the relationship between H/sup /spl infin// control, linear differential games and risk-sensitive control does not hold. The main reason for this is that the risk-sensitivity cost functional measures the risk sensitivity of the system to transitions caused by the random jump parameter as well as the noise input. The risk sensitivity of the cost functional to transitions induced by the jump parameter may be of a great value in the design of systems where it is desired to make the system as insensitive to the effects of the jump parameter as possible.

The present thesis studies some important random walk-based algorithms, which are randomized rumor spreading and balanced allocation protocols on networks. In the rst part of the thesis, we study the Push and the Push-Pull protocols introduced by [DGG+87], which are basic randomized protocols for information dissemination on networks. In Chapter 2, we propose a new model where the number of calls of each node in every round is chosen independently according to a probability distribution R with bounded mean determined at the beginning of the process. In addition to the model being a natural extension of the standard protocols, it also serves as a more realistic model for rumor spreading in a network whose entities are not completely uniform and may have di erent levels of power. We provide both lower and upper bounds on the rumor spreading time depending on statistical properties of R such as the mean or the variance. While it is well-known that the standard protocols need Θ(log n) rounds to spread a rumor on a complete network with n nodes, we show that, if R follows a power law distribution with exponent β ∈ (2, 3), then the Push-Pull protocol spreads a rumor in Θ(log log n) rounds. Moreover, when β = 3, we show a runtime of Θ ( logn log logn ) . In Chapter 3, we analyze the behavior of the standard Push-Pull protocol on a class of random graphs, called random k-trees for every integer k > 2, that are suitable to model poorly connected, small-world and scale free networks. Here, we show that the Push-Pull protocol propagates a rumor from a randomly chosen informed node to almost all nodes of a random k-tree with n nodes in O((log n)1+ck) rounds with high probability, where 0 3 can be any integer. Let l be a given positive integer. In each round t, 1 6 t 6 n, ball t picks a node of G uniformly at random and performs a nonbacktracking random walk of length l from the chosen node. Then it deterministically selects a subset of the visited nodes as the potential choices and allocates itself on one of the choices with minimum load (ties are broken uniformly at random). Provided G has a su ciently large girth, we establish an upper bound for the maximum number of balls at any bin after allocating n balls by the algorithm. We also show that the upper bound is tight up to a O(log log n) factor. In particular, we show that if we set l = b(log n) 1+ 2 c, for any constant ∈ (0, 1], and G has girth at least ω(l), then the maximum load is bounded by O(1/ ) with high probability.

We consider the problem of probabilistic safety verification for stochastic hybrid systems. In particular, we propose a method that combines two existing approaches, namely, analytical techniques and randomized algorithms. Analytical techniques, such as using stochastic approximate bisimulation, are able to handle non-deterministic initial states. However, their practical applicability is limited to relatively simple stochastic dynamics. On the other hand, randomized algorithms are able to handle more complex dynamics. However, it typically requires running a large number of simulations, and cannot be used for non-deterministic initial states. Our combined approach basically uses an analytical technique when the stochastic dynamics is simple, and switches to a randomized algorithm when the dynamics is nonlinear. The main idea is that by using the analytical technique, we can bound the gaps between the probability density functions corresponding to the family of non-deterministic initial states. This, in turn, enables randomized algorithms that provide upperand lower-bounds on the safety and unsafety probabilities. We illustrate our approach with an example from air traffic management.