Maximum Likelihood Estimation

Adaptive signal processing algorithms are often used in order to track an unknown time-varying parameter vector. This work develops an upper bound on the mean of the norm-squared error between the unknown parameter vector being tracked and the value obtained by the algorithm. The results require very mild covariance decay rate conditions on the training data and a bounded algorithm. The upper bound illustrates the relationship between the algorithm step size and the maximum rate of variation in the parameter vector being tracked.

The iterative method, which can be applied to estimation of the unknown vector of discrete parameters in combination with filtering of the unknown vector of continuous signal parameters, is proposed. It is demonstrated that this method makes it possible to construct highly efficient algorithms for joint demodulation and filtering of communications channel parameters in code-division multiple-access and orthogonal frequency-division multiplexing systems.

The paper considers the problem of estimating the parameters in a continuous time regression model with a non-Gaussian noise of pulse type. The vector of unknown parameters is assumed to belong to a compact set. The noise is specified by the Ornstein–Uhlenbeck process driven by the mixture of a Brownian motion and a compound Poisson process. Improved estimates for the unknown regression parameters, based on a special modification of the James–Stein procedure with smaller quadratic risk than the usual least squares estimates, are proposed. The developed estimation scheme is applied for the improved parameter estimation in the discrete time regression with the autoregressive noise depending on unknown nuisance parameters.

A robust output feedback control which globally stabilizes a class of SISO minimum-phase nonlinear systems with known and constant relative degree containing a vector of unknown parameters is developed. The constant parameter vector is not restricted to enter linearly in the state equations but it is assumed to belong to a known compact set and an imprecise knowledge of the nonlinearities (e.g., lookup tables) is allowed. The class of nonlinear systems is determined by geometric conditions; an additional assumption, which generalizes the knowledge of the sign of high-frequency gain for linear systems, is also required. The nonlinearities are restricted to depend, in suitable coordinates, on the output only: no growth conditions, such as sector or Lipschitz, are required. The robust output feedback control stabilizes the system for every value of the parameter vector in a known compact set. The order of the compensator is equal to the relative degree minus one. >

: The general multiplicative model represents the hazard function as the product of an 'underlying hazard rate, gamma (t), of unspecified form and a certain function of known form, g(z; beta), where z is a vector of concomitant variables, and Beta is a vector of unknown parameters. Assuming that gamma (t) can be approximated by a constant between any two consecutive failures, the general forms of likelihood function are derived. The likelihood utilizes the available information on the time of exposure to risk of each individual (until failure or withdrawal). Special cases, when the z's do not depend on t are discussed in some detail. Multiple failures are handled in a simple manner - no ordering of failures is required. Estimation of empirical survival function when there are no covariates is discussed. An example using heart transplant data, is given (for illustrative purpose only).

This paper presents a solution to a robust optimal regulation problem for a nonlinear polynomial system affected by parametric and matched uncertainties, which is based only on partial state information. The parameters describing the dynamics of the nonlinear polynomial plant depend on a vector of unknown parameters, which belongs to a finite parametric set, and the application of a certain control input is associated with the worst or least favorable value of the unknown parameter. A high-order sliding mode state reconstructor is designed for the nonlinear plant in such a way that the previously designed control can be applied for a system with incomplete information. Additionally, the matched uncertainty is also compensated by means of the same output-based regulator. The obtained algorithm is applied to control a mechanical pendulum of the third order, successfully verifying the effectiveness of the developed approach.

In this paper, we present a solution to the problem of the quadratic Mini-Max regulator for polynomial uncertain systems. The main characteristic of this type of problems is that the parameters describing the dynamic of the nonlinear plant depend on a vector of unknown parameters, which belongs to a finite parametric set, and the solution is given in terms of the worst case scenario. That is to say, the result of the application of a certain control input (in terms of the cost function value) is associated with the worst or least favorable value of the unknown parameter. Based on the general necessary conditions for mini-max optimality, a closed form for the control is provided, which makes use of a p-linear form tensor representation of the polynomial system. In its turn, this allows one to present the solution in a way similar to the so called Riccati technique. The final control is shown to be a convex combination (with some weights) of the optimal controls for each fixed parameter of the polynomial system. A numerical example for the robust regulation of the well-known Duffing equation which represents a typical (and challenging to control) nonlinear polynomial system is presented, to show the effectiveness of our approach.

On best unbiased prediction and its relationships to unbiased estimation

SELF-TUNING OUTPUT FEEDBACK STABILIZATION OF A CLASS OF NONLINEAR SYSTEMS

We consider a generalized partially linear model E(Y|X, T) = G{X T β + m(T)}, where G is a known function, β is an unknown parameter vector, and m is an unknown function. We introduce a test statistic that allows one to decide between a parametric and a semiparametric model: (a) m is linear (i.e., m(t) = t T γ for a parameter vector γ), and (b) m is a smooth (nonlinear) function. Under linearity (a), we show that the test statistic is asymptotically normal. Moreover, we prove that the bootstrap works asymptotically. Simulations suggest that (in small samples) the bootstrap outperforms the calculation of critical values from the normal approximation. The practical performance of the test is demonstrated in applications to data on East–West German migration and credit scoring.

Stability analysis of high-order Hopfield type neural networks with uncertainty

We consider estimation in a semiparametric generalized linear model for clustered data using estimating equations. Our results apply to the case where the number of observations per cluster is finite, whereas the number of clusters is large. The mean of the outcome variable μ is of the form g(μ) = XTβ + θ(T), where g(·) is a link function, X and T are covariates, β is an unknown parameter vector, and θ(t) is an unknown smooth function. Kernel estimating equations proposed previously in the literature are used to estimate the infinite-dimensional nonparametric function θ(t), and a profile-based estimating equation is used to estimate the finite-dimensional parameter vector β. We show that for clustered data, this conventional profile-kernel method often fails to yield a √n-consistent estimator of β along with appropriate inference unless working independence is assumed or θ(t) is artificially undersmoothed, in which case asymptotic inference is possible. To gain insight into these results, we derive the semiparametric efficient score of β, which is found to have a complicated form, and show that, unlike for independent data, the profile-kernel method does not yield a score function asymptotically equivalent to the semiparametric efficient score of β, even when the true correlation is assumed and θ(t) is undersmoothed. We illustrate the methods with an application to infectious disease data and evaluate their finite-sample performance through a simulation study.

Adaptive signal processing algorithms derived from LS (least squares) cost functions are known to converge extremely fast and have excellent capabilities to "track" an unknown parameter vector. This paper treats analytically and experimentally the steady-state operation of RLS (recursive least squares) adaptive filters with exponential windows for stationary and nonstationary inputs. A new formula for the "estimation-noise" has been derived involving second- and fourth-order statistics of the filter input as well as the exponential windowing factor and filter length. Furthermore, it is shown that the adaptation process associated with "lag effects" depends solely on the exponential weighting parameter λ. In addition, the calculation of the excess mean square error due to the lag for an assumed Markov channel provides the necessary information about tradeoffs between speed of adaptation and steady-state error. It is also the basis for comparison to the simple LMS algorithm, in a simple case of channel identification, it is shown that the LMS and RLS adaptive filters have the same tracking behavior. Finally, in the last part, we present new RLS restart procedures applied to transversal structures for mitigating the disastrous results of the third source of noise, namely, finite precision arithmetic.

This paper considers the estimation problem for linear continuous-time systems with the interrupted observation mechanism which is characterized in terms of the jump Markov process taking on the values of 0 or 1. The minimum variance estimator algorithm is derived. The approach adopted here is that we express the jump process in terms of the initial value and the jump times instead of its instantaneous values and that in order to apply Lainiotis' formula we regard the initial value and jump times as the unknown system parameter vector. The parameter vector is, however, infinite dimensional so that Lainiotis' formula cannot be applied directly. This is because the formula is applicable only to linear systems with a finite dimensional parameter vector. Therefore, the minimum variance estimator algorithm is derived by using some limiting procedure. The resultant optimal algorithm is infinite dimensional, so that feasible approximate algorithms should be developed for practical implementation. A simple illustrating example is also worked out.

The present chapter deals with the study of an important but least known case in the theory of parametric estimation of controlled MS, the case when the vector of unknown system parameters, or that of unknown perturbations affecting the control object, is an unknown (uncontrollable, nonmeasurable) vector function of time. Certain models of solving the problems of adaptive stabilization and optimization under parametric drift conditions, based on utilizing the formalism of Lyapunov functions, were given in earlier researches under a variety of restrictions, for example on the information about the drift model and the rate of change of the parameters. The dominant difficulties that occur in these problems of forming convergent estimation algorithms, concern the proof of the fact that the Lyapunov function monotonically decreases on the trajectories of the controlled process.