Data-based Identifiability and Observability Assessment for Nonlinear Control Systems using the Profile Likelihood Method

When non-linear models are fitted to experimental data, parameter estimates can be poorly constrained albeit being identifiable in principle. This means that along certain paths in parameter space, the log-likelihood does not exceed a given statistical threshold but remains bounded. This situation, denoted as practical non-identifiability, can be detected by Monte Carlo sampling or by systematic scanning using the profile likelihood method. In contrast, any method based on a Taylor expansion of the log-likelihood around the optimum, e.g., parameter uncertainty estimation by the Fisher Information Matrix, reveals no information about the boundedness at all. In this work, we present a geometric approach, approximating the original log-likelihood by geodesic coordinates of the model manifold. The Christoffel Symbols in the geodesic equation are fixed to those obtained from second order model sensitivities at the optimum. Based on three exemplary non-linear models we show that the information about the log-likelihood bounds and flat parameter directions can already be contained in this local information. Whereas the unbounded case represented by the Fisher Information Matrix is embedded in the geometric framework as vanishing Christoffel Symbols, non-vanishing constant Christoffel Symbols prove to define prototype non-linear models featuring boundedness and flat parameter directions of the log-likelihood. Finally, we investigate if those models could allow to approximate and replace computationally expensive objective functions originating from non-linear models by a surrogate objective function in parameter estimation problems.

A self-organizing filter and smoother for the general nonlinear non-Gaussian state-space model is proposed. An expanded state-space model is defined by augmenting the state vector with the unknown parameters of the original state-space model. The state of the augmented state-space model, and hence the state and the parameters of the original state-space model, are estimated simultaneously by either a non-Gaussian filter/smoother or a Monte Carlo filter/smoother. In contrast to maximum likelihood estimation of model parameters in ordinary state-space modeling, for which the recursive filter computation has to be done many times, model parameter estimation in the proposed self-organizing filter/smoother is achieved with only two passes of the recursive filter and smoother operations. Examples such as automatic tuning of dispersion and the shape parameters, adaptation to changes of the amplitude of a signal in seismic data, state estimation for a nonlinear state space model with unknown parameters, and seasonal adjustment with a nonlinear model with changing variance parameters are shown to exemplify the usefulness of the proposed method.

Pharmacokinetic-Pharmacodynamic Modeling and Simulation

GAIN SCHEDULING CONTROL FOR COMBINED FOUR WHEEL STEERING AND INDIVIDUAL WHEEL BRAKING

<abstract><p>This paper is concerned with proposing a generalized Liu-type estimator (GLTE) to address the multicollinearity problem of explanatory variable of the linear part in the logistic partially linear regression model. Using the profile likelihood method, we propose the GLTE as a general class of Liu-type estimator, which includes the profile likelihood estimator, the ridge estimator, the Liu estimator and the Liu-type estimator as special cases. The conditional superiority of the proposed GLTE over the other estimators is derived under the asymptotic mean square error matrix (MSEM) criterion. Moreover, the optimal choices of biasing parameters and function of biasing parameter are given. Numerical simulations demonstrate that the proposed GLTE performs better than the existing estimators. An application on a set of real data arising from the study of Indian Liver Patient is shown for illustrating our theoretical results.</p></abstract>

Nonlinear mixed-effects state space models with applications to HIV dynamics

Modelling of selective catalytic reduction systems using discrete-time Linear Parameter Varying models

The paper deals with nonlinear modeling and identification of an electrohydraulic control system for improving its tracking performance. We build the nonlinear state space model for analyzing the highly nonlinear system and then develop a Hammerstein-Wiener (H-W) model which consists of a static input nonlinear block with two-segment polynomial nonlinearities, a linear time-invariant dynamic block, and a static output nonlinear block with single polynomial nonlinearity to describe it. We simplify the H-W model into a linear-in-parameters structure by using the key term separation principle and then use a modified recursive least square method with iterative estimation of internal variables to identify all the unknown parameters simultaneously. It is found that the proposed H-W model approximates the actual system better than the independent Hammerstein, Wiener, and ARX models. The prediction error of the H-W model is about 13%, 54%, and 58% less than the Hammerstein, Wiener, and ARX models, respectively.

The problem of sequential detection of spontaneous changes in equations for unobserved state variables and observations of linear and nonlinear multivariate state-space models is considered. We consider the case of new additive terms and the case of changing coefficients of these equations. The proposed nonparametric method uses the idea of the “moving window” of observations. In Theorem 4.1 the exponential rate of convergence to zero for type 1 error probability (a wrong decision about a change) is proved. In Theorem 4.2 we consider type 2 error probability and the normalized (by the volume N of the moving window) delay time in change-point detection. Asymptotic optimality of the proposed method is proved in Theorem 5.1. Experimental study includes Monte Carlo examples of sequentially detected changes in coefficients of linear and nonlinear state-space models.

This article describes how diagnostic procedures were derived for symmetrical nonlinear regression models, continuing the work carried out by Cysneiros and Vanegas (2008) and Vanegas and Cysneiros (2010), who showed that the parameters estimates in nonlinear models are more robust with heavy-tailed than with normal errors. In this article, we focus on assessing if the robustness of this kind of models is also observed in the inference process (i.e., partial F-test). Symmetrical nonlinear regression models includes all symmetric continuous distributions for errors covering both light- and heavy-tailed distributions such as Student-t, logistic-I and -II, power exponential, generalized Student-t, generalized logistic, and contaminated normal. Firstly, a statistical test is shown to evaluating the assumption that the error terms all have equal variance. The results of simulation studies which describe the behavior of the test for heteroscedasticity proposed in the presence of outliers are then given. To assess the robustness of inference process, we present the results of a simulation study which described the behavior of partial F-test in the presence of outliers. Also, some diagnostic procedures are derived to identify influential observations on the partial F-test. As ilustration, a dataset described in Venables and Ripley (2002), is also analyzed.

Process models are increasingly used to support upstream process development in the biopharmaceutical industry for process optimization, scale-up and to reduce experimental effort. Parametric unstructured models based on biological mechanisms are highly promising, since they do not require large amounts of data. The critical part in the application is the certainty of the parameter estimates, since uncertainty of the parameter estimates propagates to model predictions and can increase the risk associated with those predictions. Currently Fisher-Information-Matrix based approximations or Monte-Carlo approaches are used to estimate parameter confidence intervals and regularization approaches to decrease parameter uncertainty. Here we apply profile likelihood to determine parameter identifiability of a recent upstream process model. We have investigated the effect of data amount on identifiability and found out that addition of data reduces non-identifiability. The likelihood profiles of nonidentifiable parameters were then used to uncover structural model changes. These changes effectively alleviate the remaining non-identifiabilities except for a single parameter out of 21 total parameters. We present the first application of profile likelihood to a complete upstream process model. Profile likelihood is a highly suitable method to determine parameter confidence intervals in upstream process models and provides reliable estimates even with nonlinear models and limited data.

Extended Least Squares (ELS) for Pharmacokinetic Models

Sequential Monte Carlo (SMC) methods (also known as particle filter) provide a way to solve the state estimation problem in nonlinear non-Gaussian state space models (SSM) through numerical approximation. Particle smoothing is one retrospective state estimation method based on particle filtering. In this paper, we propose a new particle smoother. The basic idea is easy and leads to a forward-backward procedure, where the Metropolis-Hastings algorithm is used to resample the filtering particles. The goodness of the new scheme is assessed using a nonlinear SSM. It is concluded that this new particle smoother is suitable for state estimation in complicated dynamical systems.

We present a detailed methodological study of the application of the modified profile likelihood method for the calibration of nonlinear financial models characterized by a large number of parameters. We apply the general approach to the Log-Periodic Power Law Singularity (LPPLS) model of financial bubbles. This model is particularly relevant because one of its parameters, the critical time signalling the burst of the bubble, is arguably the target of choice for dynamical risk management. However, previous calibrations of the LPPLS model have shown that the estimation of is in general quite unstable. Here, we provide a rigorous likelihood inference approach to determine , which takes into account the impact of the other nonlinear (so-called ‘nuisance’) parameters for the correct adjustment of the uncertainty on . This provides a rigorous interval estimation for the critical time, rather than the point estimation in previous approaches. As a bonus, the interval estimates can also be obtained for the nuisance parameters (, damping), which can be used to improve filtering of the calibration results. We show that the use of the modified profile likelihood method dramatically reduces the number of local extrema by constructing much simpler smoother log-likelihood landscapes. The remaining distinct solutions can be interpreted as genuine scenarios that unfold as the time of the analysis flows, which can be compared directly via their likelihood ratio. Finally, we develop a multi-scale profile likelihood analysis to visualize the structure of the financial data at different scales (typically from 100 to 750 days). We test the methodology successfully on synthetic price time series and on three well-known historical financial bubbles.

We propose the single-index hazards model for censored survival data. As an extension of the Cox model and many transformation models, this model allows nonparametric modeling of covariate effects in a parsimonious way via a single index. In addition, the relative importance of covariates can be assessed via this model. We consider two commonly used profile likelihood methods for parameter estimation: the local profile likelihood method and the stratified profile likelihood method. It is shown that both methods may give consistent estimators under certain restrictive conditions, but in general they can yield biased estimation. Simulation studies are also conducted to demonstrate these bias phenomena. The existence and nature of the failures of these two commonly used approaches is somewhat surprising.