Robust estimation via generalized quasi-gradients

We provide a general framework for designing Generative Adversarial Networks (GANs) to solve high-dimensional robust statistics problems, which aim at estimating unknown parameter of the true distribution given adversarially corrupted samples. Prior work [1], [2] focus on the problem of robust mean and covariance estimation when the true distribution lies in the family of Gaussian distributions or elliptical distributions, and analyze depth or scoring rule based GAN losses for the problem. Our work extend these to robust mean estimation, second-moment estimation, and robust linear regression when the true distribution only has bounded Orlicz norms, which includes the broad family of sub-Gaussian, sub-exponential and bounded moment distributions. We also provide a different set of sufficient conditions for the GAN loss to work: we only require its induced distance function to be a cumulative density function of some light-tailed distribution, which is easily satisfied by neural networks with sigmoid activation. In terms of techniques, our proposed GAN losses can be viewed as a smoothed and generalized Kolmogorov-Smirnov distance, which overcomes the computational intractability of the original Kolmogorov-Smirnov distance used in the weaken the distance approach in [3].

Robust covariance estimates based on resampling

The authors focus on the robust 3D-3D single and multiple pose estimation problems. The robust 3D-3D single pose estimation problem was formulated by R. M. Haralick et al. (1989) as a general regression in terms of a contaminated Gaussian error noise model. The robust 3D-3D multiple pose estimation problem appears much more difficult. This problem is formulated as a series of general regressions involving successively decreasing data size, with each regression related to one particular pose of interest. Since the first few regressions may carry a highly contaminated Gaussian error noise model, a very robust estimator called the MF-estimator, given by X. Zhuang et al. (1992), is used to solve each regression for each desired pose estimation. Computer experiments with real imagery and simulated data were conducted and results are promising. >

This article considers simultaneous estimation of means from several strata, where each stratum contains a finite number of elements. For example, quite often estimates of annual incomes, unemployment rates, and so forth must be made simultaneously for many areas. Our results find application in small area estimation where very few samples are available from an individual area, and an estimate of a certain area mean or simultaneous estimates of several area means can be improved by incorporating information from similar neighboring areas. Ghosh and Meeden (1986) considered empirical Bayes estimation of the finite population mean assuming a normal superpopulation model. Their estimators generalize automatically when one estimates a vector of stratum means. This article relaxes the normality assumption and assumes instead that the posterior expectation of any stratum mean is a linear function of the sample observations. This is the so-called “posterior linearity” property as described in Ericson (1...

In this paper, we consider a robust state estimation problem for uncertain discrete-time, homogeneous, first-order, finite-state finite-alphabet hidden Markov models (HMMs). A class of time-varying uncertain HMMs is considered in which the uncertainty is sequentially described by a regular conditional relative entropy constraint on perturbed regular conditional probability measures given the observation sequence. For this class of uncertain HMMs, the robust state estimation problem is formulated as a constrained optimization problem. Using a Lagrange multiplier technique and a duality relationship for regular conditional relative entropy, the above problem is converted into an unconstrained optimization problem and a problem related to partial information risk-sensitive filtering. Furthermore, a measure transformation technique and an information state method are employed to solve this equivalent problem related to risk-sensitive filtering.

In this paper, we consider the robust covariance estimation problem in the non-Gaussian set-up. In particular, Tyler's M-estimator is adopted for samples drawn from a heavy-tailed elliptical distribution. For some applications, the covariance matrix naturally possesses certain structure. Therefore, incorporating the prior structure information in the estimation procedure is beneficial to improving estimation accuracy. The problem is formulated as a constrained minimization of the Tyler's cost function, where the structure is characterized by the constraint set. A numerical algorithm based on majorization-minimization is derived for general structures that can be characterized as a convex set, where a sequence of convex programming is solved. For the set of matrices that can be decomposed as the sum of rank one positive semidefinite matrices, which has a wide range of applications, the algorithm is modified with much lower complexity. Simulation results demonstrate that the proposed structure-constrained Tyler's estimator achieves smaller estimation error than the unconstrained case.

EWMA control charts based on robust estimators: A powerful tool for monitoring a process with a non-normal distribution

The problem of fitting a straight line to a finite collection of points in the plane is an important problem in statistical estimation. Recently there has been a great deal of interest is robust estimators, because of their lack of sensitivity to outlying data points. The basic measure of the robustness of an estimator is its breakdown point, that is, the fraction (up to 50%) of outlying data points that can corrupt the estimator. One problem with robust estimators is that achieving high breakdown points (near 50%) has proved to be computationally demanding. In this paper we present the best known theoretical algorithm and a practical subquadratic algorithm for computing a 50% breakdown point line estimator, the Siegel or repeated median line estimator. We first present an O(n\log n) randomized expected-time algorithm, where n is the number of given points. This algorithm relies, however, on sophisticated data structures. We also present a very simple O(n log 2 n) randomized algorithm for this problem, which uses no complex data structures. We provide empirical evidence that, for many realistic input distributions, the running time of this second algorithm is actually O(n log n) expected time.

SummaryFor networked mixed uncertain time‐varying systems with uncertain noise variances, random one‐step measurement delay, state‐dependent and noise‐dependent multiplicative noises, and linearly dependent additive white noises, the robust local, centralized, and distributed fusion estimation problems are addressed. Three new approaches are presented, which include a new augmented state approach with fictitious white noises, an extended Lyapunov equation approach with two Lyapunov equations, and a universal integrated covariance intersection (ICI) fusion approach of integrating the minimax robust local Kalman estimators and their conservative cross‐covariances. They constitute a new important methodology of solving robust fusion estimation problems. Applying them, the local, centralized, and distributed ICI fusion time‐varying and steady‐state robust Kalman estimators (predictor, filter, and smoother) are presented in the sense that for all admissible uncertainties, their actual estimation error variances are guaranteed to have the corresponding minimal upper bounds. Their robustness, convergence, and accuracy relations are proved. Specially, the proposed ICI fusers improve the robust accuracies of the original covariance intersection fusers, and overcome their drawbacks, such that the local estimators and their conservative variances are assumed to be known, and the conservative cross‐variances are ignored. A simulation example with application to a vehicle suspension system shows the effectiveness of the proposed approaches and results.

This article is concerned with the robust distributed state estimation problem for a class of uncertain sensor networks with multiple stochastic communication delays. A sequence of mutually independent random variables obeying the Bernoulli distribution is introduced to account for the randomly occurred communication delays. Both the discrete-time target plant and the sensor model are subject to parameter uncertainties as well as stochastic disturbance. The parameter uncertainties are norm-bounded and enter both the system and the measurement matrices. The external stochastic disturbance is given in the form of a scalar Wiener process. Through available output measurements from not only each individual sensor but also its neighbouring sensors, we aim to design distributed state estimators in order to approximate the state of the target plant. By using the Kronecker product, stochastic analysis is carried out to derive a sufficient criterion ensuring the estimation error systems to be convergent in the mean square sense for all randomly occurred delays, admissible stochastic disturbance and parameter uncertainties. Then, an explicit expression of the individual estimator is given in terms of the solution to a convex optimisation problem that can be easily solved by using the semi-definite programme method. A numerical example is given at the end of this article to demonstrate the usefulness of the developed theoretical results.

Robust state estimation for jump Markov linear systems with missing measurements

Optimal robust Bayes estimation

Statistical Process Control (SPC) chart is a graphical tool that helps to identify a possible process shift. Though the SPC chart is, in theory, a monitoring tool that reveals only the result of a statistical hypothesis testing, in practice the chart's signaling is often used for making process adjustment. In this paper we develop a robust estimator of the process mean for processes monitored by SPC charts. This mean estimate can serve as an important reference for investigating assignable causes and taking appropriate corrective actions. When the control chart is implemented as a control device, the mean estimate can also provide a more accurate assessment of the amount of adjustment to be made to the process. We also demonstrate in this paper that when the process shift is relatively small, the proposed mean estimate outperforms the mean estimate usually used for the CUSUM scheme.

In this paper, the problem of robust estimation and validation of location-scale families is revisited. The proposed methods exploit the joint asymptotic normality of sample quantiles (of i.i.d. random variables) to construct the ordinary and generalized least squares estimators of location and scale parameters. These quantile least squares (QLS) estimators are easy to compute because they have explicit expressions, their robustness is achieved by excluding extreme quantiles from the least-squares estimation, and efficiency is boosted by using as many non-extreme quantiles as practically relevant. The influence functions of the QLS estimators are specified and plotted for several location-scale families. They closely resemble the shapes of some well-known influence functions yet those shapes emerge automatically (i.e., do not need to be specified). The joint asymptotic normality of the proposed estimators is established and their finite-sample properties are explored using simulations. Also, computational costs of these estimators, as well as those of MLE, are evaluated for sample sizes n=106,107,108,109. For model validation, two goodness-of-fit tests are constructed and their performance is studied using simulations and real data. In particular, for the daily stock returns of Google over the years 2020-2023, both tests strongly support the logistic distribution assumption and reject other bell-shaped competitors.

Linear discriminant analysis, such as Fisher's criterion, is a statistical learning tool traditionally devoted to separating a training dataset into two or even several classes by the way of linear decision boundaries. In this paper, we show that this tool can formalize the robust linear regression problem as a robust estimator will do. More precisely, we develop a one-class Fischer's criterion in which the maximization provides both the regression parameters and the separation of the data in two classes: typical data and atypical data or outliers. This new criterion is built on the statistical properties of the subspace decomposition of the hat matrix. From this angle, we improve the discriminative properties of the hat matrix which is traditionally used as outlier diagnostic measure in linear regression. Naturally, we call this new approach discriminative hat matrix. The proposed algorithm is fully nonsupervised and needs only the initialization of one parameter. Synthetic and real datasets are used to study the performance both in terms of regression and classification of the proposed approach. We also illustrate its potential application to image recognition and fundamental matrix estimation in computer vision.