The Linear State Space Model

We investigate the model for an industrial isothermal HDPE slurry reactor. The model, consisting of several nonlinear equations, can be linearized to give sets of linear time invariant state space model. The effectiveness of the linearized model is verified by the numerical simulations. A simple model predictive control scheme is constructed based on the linear state space model. The value of the melt index is obtained from the values of the manipulated and controlled variables generated from the control scheme. The control performance can be evaluated from the comparison between the computed melt index values and measured melt index values. The control scheme shows good tracking performance in the numerical simulations. We believe that the model developed in the present study can be effectively used to predict process variables as well as to control the melt index.

In this paper, we define and characterize cointegrated solutions of continuous-time linear state-space models driven by Lévy processes. Cointegrated solutions of such models are shown to be representable as a sum of a Lévy process and a stationary solution of a linear state-space model, analogous to the Granger representation for cointegrated VAR models. Moreover, we prove that the class of cointegrated multivariate Lévy-driven autoregressive moving-average (MCARMA) processes, the continuous-time analogues of the classical vector ARMA processes, is equivalent to the class of cointegrated solutions of continuous-time linear state-space models. Necessary conditions for MCARMA processes to be cointegrated are given as well extending the results of Comte [Discrete and continuous time cointegration, J. Econometrics 88 (1999), pp. 207–226] for MCAR processes. The conditions depend only on the autoregressive polynomial if we have a minimal model. Finally, we investigate cointegrated continuous-time linear state-space models observed on a discrete time-grid and calculate their linear innovations. Based on the representation of the linear innovations, we derive an error correction form. The error correction form uses an infinite linear filter in contrast to the finite linear filter for VAR models. Some sufficient identifiable criteria are also given.

This paper addresses the specification and estimation of the Seemingly Unrelated Regressions (SUR) model when the error terms exhibit first-order autocorrelation (AR(1)). The standard SUR framework, while effectively handling contemporaneous correlation across equations, assumes serially uncorrelated disturbances. This assumption is often violated in economic and financial time-series data, leading to inefficient parameter estimates if not properly addressed. We develop and implement a feasible generalized least squares (FGLS) estimation procedure tailored for the SUR-AR(1) model. The methodology involves a multi-step process: initial estimation of the individual equation autoregressive parameters (ρi​), transformation of the data to correct for serial correlation, and subsequent application of the SUR technique to the transformed model to account for contemporaneous correlation. Through Monte Carlo simulations, we demonstrate that this estimator is consistent and significantly more efficient than both equation-by-equation Ordinary Least Squares (OLS) and the traditional SUR model under conditions of first-order serial correlation. The practical application of the proposed model is illustrated with an empirical example, confirming its superiority in producing reliable and efficient estimates in real-world scenarios. Keywords: Seemingly Unrelated Regressions (SUR), First-Order Autocorrelation, AR(1) Errors, Feasible Generalized Least Squares (FGLS), Serial Correlation, Contemporaneous Correlation, Monte Carlo Simulation.

Cointegration Between Trends and Their Estimators in State Space Models and CVAR Models

Linear parametric state-space models are a ubiquitous tool for analyzing neural time series data, providing a way to characterize the underlying brain dynamics with much greater statistical efficiency than non-parametric data analysis approaches. However, neural time series data are frequently time-varying, exhibiting rapid changes in dynamics, with transient activity that is often the key feature of interest in the data. Stationary methods can be adapted to time-varying scenarios by employing fixed-duration windows under an assumption of quasi-stationarity. But time-varying dynamics can be explicitly modeled by switching state-space models, i.e., by using a pool of state-space models with different dynamics selected by a probabilistic switching process. Unfortunately, exact solutions for state inference and parameter learning with switching state-space models are intractable. Here we revisit a switching state-space model inference approach first proposed by Ghahramani and Hinton. We provide explicit derivations for solving the inference problem iteratively after applying a variational approximation on the joint posterior of the hidden states and the switching process. We introduce a novel initialization procedure using an efficient leave-one-out strategy to compare among candidate models, which significantly improves performance compared to the existing method that relies on deterministic annealing. We then utilize this state inference solution within a generalized expectation-maximization algorithm to estimate model parameters of the switching process and the linear state-space models with dynamics potentially shared among candidate models. We perform extensive simulations under different settings to benchmark performance against existing switching inference methods and further validate the robustness of our switching inference solution outside the generative switching model class. Finally, we demonstrate the utility of our method for sleep spindle detection in real recordings, showing how switching state-space models can be used to detect and extract transient spindles from human sleep electroencephalograms in an unsupervised manner.

PurposeIn order to enable cost-utility analysis of shoulder pain conditions and treatments, this study aimed to develop and evaluate mapping algorithms to estimate the EQ-5D health index from the Oxford Shoulder Score (OSS) when health outcomes are only assessed with the OSS.Methods5437 paired OSS and EQ-5D questionnaire responses from four national multicentre randomised controlled trials investigating different shoulder pathologies and treatments were split into training and testing samples. Separate EQ-5D-3L and EQ-5D-5L analyses were undertaken. Transfer to utility (TTU) regression (univariate linear, polynomial, spline, multivariable linear, two-part logistic-linear, tobit and adjusted limited dependent variable mixture models) and response mapping (ordered logistic regression and seemingly unrelated regression (SUR)) models were developed on the training sample. These were internally validated, and their performance evaluated on the testing sample. Model performance was evaluated over 100-fold repeated training–testing sample splits.ResultsFor the EQ-5D-3L analysis, the multivariable linear and splines models had the lowest mean square error (MSE) of 0.0415. The SUR model had the lowest mean absolute error (MAE) of 0.136. Model performance was greatest in the mid-range and best health states, and lowest in poor health states.For the EQ-5D-5L analyses, the multivariable linear and splines models had the lowest MSE (0.0241–0.0278) while the SUR models had the lowest MAE (0.105–0.113).ConclusionThe developed models now allow accurate estimation of the EQ-5D health index when only the OSS responses are available as a measure of patient-reported health outcome.

The omission of important variables is often encountered in econometrics, which often results in the misspecification of the model. This study considered a misspecified joint seemingly unrelated regression (SUR) model within the Bayesian context. The explanatory variables were generated from a uniform distribution for sample sizes 5, 10, 30, 50, 100 and 1000. The coefficients of were generated from the multivariate normal distribution. Different levels of correlation (0.2, 0.4, 0.5, 0.7 and 0.9) among the independent variables were considered. The choice of hyperprior distribution was explored. Due to the incorrectly defined model, the intended attribute of unbiasedness in the estimated coefficients, which depends on the fitted model's correspondence with the actual underlying data-generating process, was not attained. Irregular patterns were seen in the posterior means and standard deviations except for a few large sample sizes. Keywords: Misspecification, Multicollinearity, Posterior mean, Posterior standard deviation, Seemingly unrelated regression.

This paper investigates change point detection in state space models, in which the pre-change distribution f <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">θ0</sup> is given, while the post-distribution f <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">θ</sup> after change is unknown. The problem is to raise an alarm as soon as possible after the distribution changes from f <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">θ0</sup> to f <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">θ</sup> , under a restriction on the false alarms. We investigate theoretical properties of a weighted Shiryayev-Roberts-Pollak (SRP) change point detection rule in state space models. By making use of a Markov chain representation for the likelihood function, exponential embedding of the induced Markovian transition operator, nonlinear Markov renewal theory, and sequential hypothesis testing theory for Markov random walks, we show that the weighted SRP procedure is second-order asymptotically optimal. To this end, we derive an asymptotic approximation for the expected stopping time of such a stopping scheme when the change time w = 1. To illustrate our method we apply the results to two types of state space models: general state Markov chains and linear state space models. Simulation study is given to illustrate the method.

Based on the problem how to model Seemingly Unrelated linear models with different batches of samples, the paper raises the method of getting rid of the stale and taking in the fresh. The method of taking in the fresh is that first, with a group of samples gotten to establish the linear model and receive least squares estimation; Second , the second batch of samples of data is received, re-use the first batch of samples operations, the outcomes of the two batches of samples are integrated with the least-squares estimation of linear models, and thus obtain the recursive algorithm of least squares estimation; The method of getting rid of the stale, contrary to the method of bringing into the fresh, excludes the old sample information and thus obtains a recursive algorithm of least-squares estimation; the method of getting rid of the stale and taking in the fresh is the recursive algorithm integrated with the above two kinds of recursive algorithms. The paper proposes the recursive algorithm of generalized least squares estimator of Seemingly Unrelated linear regression models under the given conditions, and showed, under the Mean Dispersion Error(MDE) criterion, that increasing the sample size will increase the estimation accuracy .

A method for obtaining linear state space models of the drawing process is developed. Traditionally, computational fluid dynamics methods have been used to model the drawing process. Although these models have the potential to provide very accurate details of the flow field, they incorporate thousands of dynamic states which make them unsuitable for both real-time simulations and control design. Experimental data have also been used to construct frequency response models which are suitable for control design. However, they heavily rely on the particular operating conditions for which they were obtained. Furthermore, they are constructed using large lumping techniques based on the information at the boundary and cannot predict large perturbations in the flow field. The objective of this paper is to bridge the gap between the system theoretic modeling techniques of the control engineer and the more physically motivated modeling methods of computational fluid dynamics. The method presented here consists of using the basic conservation laws (mass, momentum, and energy) to describe the mean flow of glass along the axial direction. Then, a linear state space model is obtained by spatially discretizing and linearizing the nonlinear partial differential equations. The resulting state space model incorporates all the relevant inputs and outputs of the system in a multiple-input/multiple-output framework. Furthermore, it lends itself to the application of modern control design techniques. The method is simple to implement since all that is needed is the steady state operating points, which can be computed from computational fluid dynamics simulations.

Application of a Linear Gaussian State Space Model and the Kalman Filter in Estimating Expectations of the Natural Logarithmic Monthly Nav Returns of the Open – End Funds of Major Investment Banks.

Deep learning has been used to train neural networks to estimate the Remaining Useful Life (RUL) of a machine given sensor signals from that machine. This has resulted in some accurate state-of-the-art RUL estimation models. Here we present a novel way to construct a generative model to estimate the RUL and quantify its uncertainty. It takes the form of a Linear Gaussian State Space Model (LGSSM) and is trained using the Kalman Filter; hence, uncertainty is quantified using this LGSSM instead of by other popular methods like Monte Carlo Dropout or Deep Ensembles. This means we can train by directly using the marginal log-likelihood loss and don’t require multiple samples to represent the uncertainty. However, this method is limited to using Gaussian distributions to quantify uncertainty. We avoid needing to use non-linear variants of the filter by processing the sensors using a neural network to represent noncausal sensor sequences as a “control variable” in the LGSSM. This noncausal representation is shown to be important for achieving state-of-the-art performance. The model is tested on a turbofan engine and dust filter dataset. The code can be found on GitHub. 1 1 https://github.com/StarMarco/Kalman_DVAE • Deep Neural Networks encode time window of sensor signals. • These are used as control/forcing variables of a Linear Gaussian State Space Model (LGSSM). • The “observables" of the LGSSM are the Remaining Useful Life (RUL) values of the machine. • The Kalman filtering algorithm is used to estimate the marginal log-likelihood of these RUL estimates. • After training the LGSSM can be evaluated over time to estimate the RUL trajectories.

A method is presented for classification of trend curves based on the linear state space model. In this approach information about the smoothness of the trend curves is incorporated into the classification model by a nonstationary stochastic trend model and can thereby be used to obtain a better classification. In the case of small data sets the performance of the classification is significantly improved in comparison with the usual cluster analysis. Maximum likelihood estimation can be used to calculate the parameters of this model and to determine the classification. The classification algorithm is described in detail and the results are compared to those of the usual cluster analysis by simulation studies and by an application to tree ring data.

The closed-loop brain stimulation system can adjust stimulation parameters based on neural activity feedback, so as to provide precise and personalized treatment for neurological and neuropsychiatric diseases. The design of a model-based closed-loop system requires the choice of a suitable system identification framework to learn a dynamic mapping from input and output data to quantify the effect of input stimulation on output neural activity, and design the controller with the identification model. We first designed a control-theory system identification framework to build a dynamic input-output (IO) model of neural activity suitable for closed-loop control design. To achieve effective model-based control, we use a linear state space model (LSSM), which uses low-dimensional hidden neural states to characterize the effect of inputs on neural activity. Secondly, in order to train the model parameters, we choose a pulse sequence with independent modulation of parameters by pseudo-random sequence (PRS), and proved that this is the best choice for collecting informational IO data sets in system identification. The input waveform can be used as a deep brain stimulation (DBS) for treating neuropsychiatric diseases while meeting clinical safety requirements. Thirdly, we take the beta frequency band of the local field potential (LFP) as a biomarker of Parkinson's disease (PD) and system feedback signal to identify the system and design a closed-loop controller. The simulation results show that the closed-loop controller designed based on the LSSM model identified by the PRS modulation waveform can achieve the ideal control effect, and the performance of the model is similar to that of a real physiological IO model describing neural activity. The new PRS modulation waveform and system identification framework can help develop future model-based closed-loop stimulation systems, which is of great significance for the treatment of neuropsychiatric diseases.

Predicting the distribution of households and employment: a seemingly unrelated regression model with two spatial processes