A Recursive Sparse Learning Method: Application to Jump Markov Linear Systems

Jump Markov linear systems are linear systems whose parameters evolve with time according to a finite-state Markov chain. Given a set of observations, our aim is to estimate the states of the finite-state Markov chain and the continuous (in space) states of the linear system. The computational cost in computing conditional mean or maximum a posteriori (MAP) state estimates of the Markov chain or the state of the jump Markov linear system grows exponentially in the number of observations. We present three globally convergent algorithms based on stochastic sampling methods for state estimation of jump Markov linear systems. The cost per iteration is linear in the data length. The first proposed algorithm is a data augmentation (DA) scheme that yields conditional mean state estimates. The second proposed scheme is a stochastic annealing (SA) version of DA that computes the joint MAP sequence estimate of the finite and continuous states. Finally, a Metropolis-Hastings DA scheme based on SA is designed to yield the MAP estimate of the finite-state Markov chain. Convergence results of the three above-mentioned stochastic algorithms are obtained. Computer simulations are carried out to evaluate the performances of the proposed algorithms. The problem of estimating a sparse signal developing from a neutron sensor based on a set of noisy data from a neutron sensor and the problem of narrow-band interference suppression in spread spectrum code-division multiple-access (CDMA) systems are considered.

Real-world control applications often involve complex dynamics subject to abrupt changes or variations. Markov jump linear systems (MJS) provide a rich framework for modeling such dynamics. Despite an extensive history, theoretical understanding of parameter sensitivities of MJS control is somewhat lacking. Motivated by this, we investigate robustness aspects of certainty equivalent model-based optimal control for MJS with a quadratic cost function. Given the uncertainty in the system matrices and in the Markov transition matrix is bounded by ϵ and η respectively, robustness results are established for (i) the solution to coupled Riccati equations and (ii) the optimal cost, by providing explicit perturbation bounds that decay as $\mathcal{O}\left( {\varepsilon + \eta } \right)$ and $\mathcal{O}\left( {{{\left( {\varepsilon + \eta } \right)}^2}} \right)$ respectively.

Robust state estimation for jump Markov linear systems with missing measurements

This paper addresses the feedback stabilization problem for Markov jump linear systems with quantized control input. A simple but powerful input controller for eliminating the input quantization in Markov jump linear systems is used. The proposed mode-dependent input controller comprises two parts: the linear part and the nonlinear part, where the linear part determines the fundamental characteristics of the systems, and the nonlinear part eliminates the effect of the input quantization. The stability is analyzed in detail for both the discrete-time and continuous-time Markov jump linear systems with input quantization. The sufficient conditions and the specific mode-dependent input controller to guarantee the stability of the systems are obtained. Numerical examples are given to illustrate the effectiveness of the results.

This paper is concerned with the distributed model predictive control (MPC) problem for a class of discrete-time Markovian jump linear systems (MJLSs) subject to actuator saturation and polytopic uncertainty in system matrices. The global system is decomposed into several subsystems which coordinate with each other. A set of distributed controllers is designed by solving a min-max optimization problem in terms of the solutions of linear matrix inequalities (LMIs). An iterative algorithm is developed to achieve the online computation. Finally, a simulation example is employed to show the effectiveness of the proposed algorithm.

In this chapter, the problem of \(H_{\infty }\) filtering for a class of discrete-time Markov jump linear systems (MJLSs) with partially unknown TPs is investigated. A mode-dependent full-order filter is constructed and the bounded real lemma (BRL) for the resulting filtering error system is derived via linear matrix inequality (LMI) formulation. Then, an improved version of the BRL is further given by introducing additional slack matrix variables to eliminate the cross coupling between system matrices and Lyapunov matrices among different operation modes. Finally, the existence criterion of the desired filter is obtained such that the corresponding filtering error system is stochastically stable with a guaranteed \(H_{\infty }\) performance index.

Deals with Lyapunov equations for continuous-time Markov jump linear systems (MJLS). We focus on the case in which the Markov chain has a countably infinite state space. It is shown that the infinite MJLS is stochastically stabilizable (SS) if and only if the associated countably infinite coupled Lyapunov equations have a unique norm bounded strictly positive solution. This result does not hold for mean square stabilizability (MSS), since SS and MSS are no longer equivalent in our set up. To some extent, tools from operator theory in Banach space and, in particular, from semigroup theory are the technical underpinning of the paper.

High-order moment stabilization for Markov jump systems with attenuation rate

This work considers fault-tolerant control (FTC) for discrete-time Markov Jump Linear System (MJLS) subject to constraints on the state an control variables. The objective is to design a control law which depends on the jump variable, minimizing an average quadratic function. Improving over previous MJLS literature, input and state constraints are enforced. The initial condition of the system and the transition probability of the Markov chain are available to the controller at each instant of time. Concepts arising from the receding horizon framework and invariant set theory are incorporated in a constrained fault-tolerant predictive control approach.

In this paper a mode-dependent repetitive controller design for discrete-time Markov Jump Linear Systems (MJLS) is proposed for tracking a periodic reference signal. By employing ideas from the lifting technique, the discrete-time MJLS and the repetitive controller structure are transformed into a stochastic two-dimensional model that differentiates the control and learning actions involved in the repetitive controller structure. For this transformed model, sufficient conditions are derived for the closed-loop control system to be mean square asymptotically stable by utilizing tools from stochastic systems theory and Lyapunov techniques. Results from numerical simulations on a stochastic switching tunnel diode circuit are presented to corroborate the proposed method.

This paper investigates a new optimal control strategy for discrete-time Markovian Jump Linear Systems (MJLSs) with controllable Markov chain and Gaussian white noise. Meanwhile, this MJLS is established under a scalar condition, i.e., state variable, input variable and output variable is scalar. For this system, the optimal control strategy is a combination of output-feedback controller to govern system state and decision which means the artificial action to govern MTPM. Motivated by this, a new joint cost function is put forward to evaluate system performance which is a combination of traditional JLQG cost and additional decision cost. Differing from traditional cost function, this joint cost function means a trade-off between control cost and decision cost and can be further minimized by optimal control strategy. To minimize this joint cost function, the designing of the optimal control strategy is deduced to the seeking of the optimal decision, and the optimal decision can be obtained by an iterative algorithm. Numerical examples illustrate the validity of the proposed optimal control strategy.

This paper studies control problems subject to signal-to-noise ratio (SNR) constraints for two-channel MIMO architectures. We first establish a second order moments equivalence between LTI feedback systems subject to SNR constraints and a class of Markov jump linear systems (MJLSs). This equivalence allows one to recast the original SNR constrained problem into an optimal mode-independent controller design problem for MJLSs. Exploiting this fact, and provided some linear matrix inequalities are feasible, we compute an upper bound on the best achievable performance subject to SNR constraints in two-channel architectures, when static state feedback controllers are employed. A numerical example is included to illustrate the results.

The linear quadratic tracking control problem is studied for a class of discrete-time uncertain Markov jump linear systems with time-varying conditional distributions. The controller is designed under the assumption that it has no access to the true states of the Markov chain, but rather it depends on the Markov chain state estimates. To deal with uncertainty, the transition probabilities of Markov state estimates between the different operating modes of the system are considered to belong in an ambiguity set of some nominal transition probabilities. The estimation problem is solved via the one-step forward Viterbi algorithm, while the stochastic control problem is solved via minimax optimization theory. An optimal control policy with some desired robustness properties is designed, and a maximizing time-varying transition probability distribution is obtained. A numerical example is given to illustrate the applicability and effectiveness of the proposed approach.

A Markov model for short term wind speed prediction by integrating the wind acceleration information

Identification of a Wiener System via Semidefinite Programming