Extended Stochastic Gradient Identification Method for Hammerstein Model Based on Approximate Least Absolute Deviation

Least absolute deviation (LAD) is a well-known criterion to fit statistical models, but little is known about LAD estimation in structural equation modeling (SEM). To address this gap, the authors use the LAD criterion in SEM by minimizing the sum of the absolute deviations between the observed and the model-implied covariance matrices. Using Monte Carlo simulations, the authors compare the performance of this LAD estimator along several dimensions (bias, efficiency, convergence, frequencies of improper solutions, and absolute percentage deviation) to the full information maximum likelihood (ML) and unweighted least squares (ULS) estimators in structural equation modeling. The results for LAD are mixed: There are special conditions under which the LAD estimator outperforms ML and ULS, but the simulation evidence does not support a general claim that LAD is superior to ML and ULS in small samples.

In general linear modeling, an alternative to the method of least squares (LS) is the least absolute deviations (LAD) procedure. Although LS is more widely used, the LAD approach yields better estimates in the presence of outliers. In this paper, we examine the performance of LAD estimators for the parameters of the first‐order autoregressive model in the presence of outliers. A simulation study compared these estimates with those given by LS. The general conclusion is that LAD does not deal successfully with additive outliers. A simple procedure is proposed which allows exception reporting when outliers occur.

Observations at the first opposition are used to determine the orbits of 16 near-Earth asteroids with two or more observed oppositions. The orbits are improved by the differential method. This paper considers two modifications of the improvement technique, which are compared to the classical method based on the principle of the least square method (LSM). The first modification uses the principle of least absolute deviations (LAD). In the second modification, the differences O - C (between the observed and calculated positions) are transformed to fit into a new coordinate system whereby the axes go parallel and perpendicular to the asteroid's apparent path (the modified differential method (MDM)). The orbits determined from one opposition by the classical LSM, LAD, and MDM are compared to a more accurate orbit calculated by the LSM from all the available oppositions. The calculations show that in 13 cases out of 16, the asteroid orbits calculated by LAD are more accurate than those calculated by the classical LSM. The improvement by the modified differential method, which includes the O - C transformation, does not produce any perceptible increase in accuracy when compared to the orbits calculated by the classical method.

Compact representation of signals and images is a key for many applications. Compactness is often achieved through linear transforms with good energy concentration property. We present an adaptive wavelet filter bank with fixed number of vanishing moments, plus additional local adaptation. Proposed adaptation method is conducted at each sample according to the least absolute deviation (LAD) criterion. Fixed vanishing moments provide for polynomial annihilation. Adaptation is aimed to achieve maximum sparseness for a wider class of signals, such as sine waves. LAD criterion results in more accurate adaptation on sudden changes of signal statistics. In this paper, an efficient LAD realization is proposed, in spite of nonexistence of the closed form solution. Combining least squares and LAD criterion, we have achieved unbiased adaptation, robust to noise. Due to its simplicity and acceptable computational speed, the proposed scheme is a good candidate for the real-world applications. In this paper, advantages of the proposed scheme are shown in signal denoising and reconstruction.

In this letter, a novel factor graph approach to target localization in distributed MIMO radars is proposed. To achieve robust localization in the presence of outliers, target localization can be formulated as a least absolute deviation (LAD) problem, which, however, is difficult to solve. We then reformulate the LAD problem as a reweighted least square (LS) one, which is converted to a product of some functions, enabling the use of factor graph techniques. Based on a factor graph representation, a highly efficient message passing algorithm is developed, where the target location is estimated in an iterative way. Comparisons with state-of-the-art methods show that the proposed method is superior in terms of computational complexity, robustness and accuracy.

Common econometric estimators such as least squares, least absolute deviations (LAD), instrumental variables, maximum likelihood, and semiparametric estimators are non-robust against data contamination. Despite the known superiority of high-breakdown point (HBP) estimators in such situations, the HBP estimators have rarely been used in economics. This article presents some applications of an HBP estimator called the S-estimator (Rousseeuw and Yohai, Robust and Nonlinear Time Series Analysis (Eds) W.H. Franke and R.D. Martin, Springer-Verlag, NY, pp. 256–72, 1984) to the estimation of a linear regression model and compares the results with those obtained by ordinary least squares (OLS) and LAD methods. It is found that significance of variables as well as signs of coefficient estimates can be quite different under HBP estimation than under OLS and LAD estimation.

This chapter focuses on an alternative robust estimation/testing procedure for possibly infinite variance time series models. In the context of inference for heavy-tailed observation, least absolute deviations (LAD) estimators are known to be less sensitive to outliers than the classical least squares regression. This section generalizes the LAD regression-based inference procedure to the self-weighted version, which is a concept originally introduced by Ling (2005) for AR processes. Using the self-weighting method, we extend the generalized empirical likelihood (GEL) method to possibly infinite variance process, and construct feasible and robust estimation/testing procedures. The former half of this chapter provides a brief introduction to the LAD regression method for possibly infinite variance ARMA models, and construct the self-weighted GEL statistic following Akashi (2017). The desirable asymptotic properties of the proposed statistics will be elucidated. The latter half of this chapter illustrates an important application of the self-weighted GEL method to the change point problem of time series models.

We consider an L 1 analogue of the least squares estimator for the parameters of stationary, finite-order autoregressions. This estimator, the least absolute deviation (LAD), is shown to be strongly consistent via a result that may have independent interest. The striking feature is that the conditions are so mild as to include processes with infinite variance, notably the stationary, finite autoregressions driven by stable increments in Lα, α > 1. Finally, sampling properties of LAD are compared to those of least squares. Together with a known convergence rate result for least squares, the Monte Carlo study provides evidence for a conjecture on the convergence rate of LAD.

Many IN mitigation techniques have been proposed to mitigate impulsive noise (IN) over powerlines, the most common of which is the blanking technique. The conventional way to implement this technique however requires prior knowledge about the IN characteristics to identify the optimal blanking threshold (OBT). When such knowledge cannot be obtained the performance deteriorates rapidly. To alleviate this, a look-up table (LUT) based algorithm with uniform quantization is deployed to utilize estimates of the peak to average power ratio at the receiver to determine the OBT. In this paper, we investigate the impact of quantization bits on the system performance as well as the performance loss due to the impact of IN on the side information. Two aspects of the achievable performance are considered namely, output signal-to-noise ratio (SNR) and symbol error rate under various IN scenarios. The results reveal that a 5 bit LUT is sufficient to achieve a gain of up to 3dB SNR improvement relative to the conventional blanking method. Furthermore, it will be shown that the loss due to the practical impact of IN on the side information is insignificant.

Say that a regression method is "unstable" at a data set if a small change in the data can cause a relatively large change in the fitted plane. A well-known example of this is the instability of least squares regression (LS) near (multi)collinear data sets. It is known that least absolute deviation (LAD) and least median of squares (LMS) linear regression can exhibit instability at data sets that are far from collinear. Clear-cut instability occurs at a "singularity"--a data set, arbitrarily small changes to which can substantially change the fit. For example, the collinear data sets are the singularities of LS. One way to measure the extent of instability of a regression method is to measure the size of its "singular set" (set of singularities). The dimension of the singular set is a tractable measure of its size that can be estimated without distributional assumptions or asymptotics. By applying a general theorem on the dimension of singular sets, we find that the singular sets of LAD and LMS are at least as large as that of LS and often much larger. Thus, prima facie, LAD and LMS are frequently unstable. This casts doubt on the trustworthiness of LAD and LMS as exploratory regression tools.

Asymptotics for argmin processes: Convexity arguments

Underdetermined inverse sparse signal reconstruction problems in the presence of non-Gaussian noise interference are often encountered in high-mobility wireless communications and signal processing. These problems can be solved by finding the minimizer of a suitable objective function which consists of a data-fitting term and a regularization term with different mixed-norms. Based on the Gaussian-noise assumption, two mixed norms (i.e. l 2 /l 1 and l ∞ /l 1 ) were confirmed as effective as well as stable algorithms for reconstructing sparse signals. However, the two algorithms are unable to reconstruct signal stable under non-Gaussian noise environments. In this paper, we propose a stable least absolute deviation (LAD) algorithm (i.e., l 1 /l 1 ) for achieving two aspects: exploiting signal sparse structure information as well as mitigating the non-Gaussian noise interference. First of all, regularization parameter of the proposed algorithm is selected via Monte Carlo simulations. Then, experimental results in different non-Gaussian environments are used to demonstrate the effectiveness of the proposed algorithm.

This paper discussed robust estimation for point estimation of the shape and scale parameters for generalized exponential (GE) distribution using a complete dataset in the presence of various percentages of outliers. In the case of outliers, it is known that classical methods such as maximum likelihood estimation (MLE), least square (LS) and maximum product spacing (MPS) in case of outliers cannot reach the best estimator. To confirm this fact, these classical methods were applied to the data of this study and compared with non-classical estimation methods. The non-classical (Robust) methods such as least absolute deviations (LAD), and M-estimation (using M. Huber (MH) weight and M. Bisquare (MB) weight) had been introduced to obtain the best estimation method for the parameters of the GE distribution. The comparison was done numerically by using the Monte Carlo simulation study. The two real datasets application confirmed that the M-estimation method is very much suitable for estimating the GE parameters. We concluded that the M-estimation method using Huber object function is a suitable estimation method in estimating the parameters of the GE distribution for a complete dataset in the presence of various percentages of outliers.

Image restoration problems are often solved by finding the minimizer of a suitable objective function. Usually this function consists of a data-fitting term and a regularization term. For the least squares solution, both the data-fitting and the regularization terms are in the ℓ2 norm. In this paper, we consider the least absolute deviation (LAD) solution and the least mixed norm (LMN) solution. For the LAD solution, both the data-fitting and the regularization terms are in the ℓ1 norm. For the LMN solution, the regularization term is in the ℓ1 norm but the data-fitting term is in the ℓ2 norm. The LAD and the LMN solutions are formulated as the solutions of a linear and a quadratic programming problems respectively, and solved by interior point methods. At each iteration of the interior point method, a structured linear system must be solved. The preconditioned conjugate gradient method with factorized sparse inverse preconditioners is employed to such structured inner systems. Experimental results are used to demonstrate the effectiveness of our approach. We also show the quality of the restored images using the minimization of ℓ1 norm/mixed ℓ1 and ℓ2 norms is better than that using ℓ2 norm approach.

Local Walsh-average regression