Least absolute deviation estimates in autoregression with infinite variance

We consider an L 1 analogue of the least squares estimator for the parameters of stationary, finite-order autoregressions. This estimator, the least absolute deviation (LAD), is shown to be strongly consistent via a result that may have independent interest. The striking feature is that the conditions are so mild as to include processes with infinite variance, notably the stationary, finite autoregressions driven by stable increments in Lα, α > 1. Finally, sampling properties of LAD are compared to those of least squares. Together with a known convergence rate result for least squares, the Monte Carlo study provides evidence for a conjecture on the convergence rate of LAD.

This chapter focuses on an alternative robust estimation/testing procedure for possibly infinite variance time series models. In the context of inference for heavy-tailed observation, least absolute deviations (LAD) estimators are known to be less sensitive to outliers than the classical least squares regression. This section generalizes the LAD regression-based inference procedure to the self-weighted version, which is a concept originally introduced by Ling (2005) for AR processes. Using the self-weighting method, we extend the generalized empirical likelihood (GEL) method to possibly infinite variance process, and construct feasible and robust estimation/testing procedures. The former half of this chapter provides a brief introduction to the LAD regression method for possibly infinite variance ARMA models, and construct the self-weighted GEL statistic following Akashi (2017). The desirable asymptotic properties of the proposed statistics will be elucidated. The latter half of this chapter illustrates an important application of the self-weighted GEL method to the change point problem of time series models.

Least absolute deviation (LAD) is a well-known criterion to fit statistical models, but little is known about LAD estimation in structural equation modeling (SEM). To address this gap, the authors use the LAD criterion in SEM by minimizing the sum of the absolute deviations between the observed and the model-implied covariance matrices. Using Monte Carlo simulations, the authors compare the performance of this LAD estimator along several dimensions (bias, efficiency, convergence, frequencies of improper solutions, and absolute percentage deviation) to the full information maximum likelihood (ML) and unweighted least squares (ULS) estimators in structural equation modeling. The results for LAD are mixed: There are special conditions under which the LAD estimator outperforms ML and ULS, but the simulation evidence does not support a general claim that LAD is superior to ML and ULS in small samples.

In order to identify the parameters of nonlinear Hammerstein model which are contaminated by colored noise and peak noise, the least absolute deviation (LAD) is selected as the objective function to solve the problem of large residual square when the identification data is disturbed by the impulse noise which obeys symmetrical alpha stable (SαS) distribution. However, LAD cannot meet the need of differentiability required by most algorithms. To improve robustness and to solve the nondifferentiable problem, an approximate least absolute deviation (ALAD) objective function is established by introducing a deterministic function to replace absolute value under certain situations. The proposed method is derived from ALAD criterion and extended stochastic gradient method. Due to the differentiability of the objective function, we can get a recursive identification algorithm which is simple and easy to calculate compared with LAD. The convergence of the proposed identification method is also proved by Lyapunov stability theory, and the simulation experiments show that the proposed method has higher accuracy and stronger robustness than the least square (LS) method in the identification of Hammerstein model with colored noise and impulse noise. The impact of impulse noise can be restrained effectively.

Compact representation of signals and images is a key for many applications. Compactness is often achieved through linear transforms with good energy concentration property. We present an adaptive wavelet filter bank with fixed number of vanishing moments, plus additional local adaptation. Proposed adaptation method is conducted at each sample according to the least absolute deviation (LAD) criterion. Fixed vanishing moments provide for polynomial annihilation. Adaptation is aimed to achieve maximum sparseness for a wider class of signals, such as sine waves. LAD criterion results in more accurate adaptation on sudden changes of signal statistics. In this paper, an efficient LAD realization is proposed, in spite of nonexistence of the closed form solution. Combining least squares and LAD criterion, we have achieved unbiased adaptation, robust to noise. Due to its simplicity and acceptable computational speed, the proposed scheme is a good candidate for the real-world applications. In this paper, advantages of the proposed scheme are shown in signal denoising and reconstruction.

Say that a regression method is "unstable" at a data set if a small change in the data can cause a relatively large change in the fitted plane. A well-known example of this is the instability of least squares regression (LS) near (multi)collinear data sets. It is known that least absolute deviation (LAD) and least median of squares (LMS) linear regression can exhibit instability at data sets that are far from collinear. Clear-cut instability occurs at a "singularity"--a data set, arbitrarily small changes to which can substantially change the fit. For example, the collinear data sets are the singularities of LS. One way to measure the extent of instability of a regression method is to measure the size of its "singular set" (set of singularities). The dimension of the singular set is a tractable measure of its size that can be estimated without distributional assumptions or asymptotics. By applying a general theorem on the dimension of singular sets, we find that the singular sets of LAD and LMS are at least as large as that of LS and often much larger. Thus, prima facie, LAD and LMS are frequently unstable. This casts doubt on the trustworthiness of LAD and LMS as exploratory regression tools.

Common econometric estimators such as least squares, least absolute deviations (LAD), instrumental variables, maximum likelihood, and semiparametric estimators are non-robust against data contamination. Despite the known superiority of high-breakdown point (HBP) estimators in such situations, the HBP estimators have rarely been used in economics. This article presents some applications of an HBP estimator called the S-estimator (Rousseeuw and Yohai, Robust and Nonlinear Time Series Analysis (Eds) W.H. Franke and R.D. Martin, Springer-Verlag, NY, pp. 256–72, 1984) to the estimation of a linear regression model and compares the results with those obtained by ordinary least squares (OLS) and LAD methods. It is found that significance of variables as well as signs of coefficient estimates can be quite different under HBP estimation than under OLS and LAD estimation.

Observations at the first opposition are used to determine the orbits of 16 near-Earth asteroids with two or more observed oppositions. The orbits are improved by the differential method. This paper considers two modifications of the improvement technique, which are compared to the classical method based on the principle of the least square method (LSM). The first modification uses the principle of least absolute deviations (LAD). In the second modification, the differences O - C (between the observed and calculated positions) are transformed to fit into a new coordinate system whereby the axes go parallel and perpendicular to the asteroid's apparent path (the modified differential method (MDM)). The orbits determined from one opposition by the classical LSM, LAD, and MDM are compared to a more accurate orbit calculated by the LSM from all the available oppositions. The calculations show that in 13 cases out of 16, the asteroid orbits calculated by LAD are more accurate than those calculated by the classical LSM. The improvement by the modified differential method, which includes the O - C transformation, does not produce any perceptible increase in accuracy when compared to the orbits calculated by the classical method.

In general linear modeling, an alternative to the method of least squares (LS) is the least absolute deviations (LAD) procedure. Although LS is more widely used, the LAD approach yields better estimates in the presence of outliers. In this paper, we examine the performance of LAD estimators for the parameters of the first‐order autoregressive model in the presence of outliers. A simulation study compared these estimates with those given by LS. The general conclusion is that LAD does not deal successfully with additive outliers. A simple procedure is proposed which allows exception reporting when outliers occur.

Nonparametric M-regression with free knot splines

Asymptotics for argmin processes: Convexity arguments

We consider the weighted median problem for a given set of data and analyze its main properties. As an illustration, an efficient method for searching for a weighted Least Absolute Deviations (LAD)-line is given, which is used as the basis for solving various linear and nonlinear LAD-problems occurring in applications. Our method is illustrated by an example of hourly natural gas consumption forecast.

Underdetermined inverse sparse signal reconstruction problems in the presence of non-Gaussian noise interference are often encountered in high-mobility wireless communications and signal processing. These problems can be solved by finding the minimizer of a suitable objective function which consists of a data-fitting term and a regularization term with different mixed-norms. Based on the Gaussian-noise assumption, two mixed norms (i.e. l 2 /l 1 and l ∞ /l 1 ) were confirmed as effective as well as stable algorithms for reconstructing sparse signals. However, the two algorithms are unable to reconstruct signal stable under non-Gaussian noise environments. In this paper, we propose a stable least absolute deviation (LAD) algorithm (i.e., l 1 /l 1 ) for achieving two aspects: exploiting signal sparse structure information as well as mitigating the non-Gaussian noise interference. First of all, regularization parameter of the proposed algorithm is selected via Monte Carlo simulations. Then, experimental results in different non-Gaussian environments are used to demonstrate the effectiveness of the proposed algorithm.

Bootstrap tests for unit roots based on LAD estimation

Nonlinear regression is commonly used as a modeling tool to get a functional form between inputs and response variables when the inputs and the responses have a nonlinear relationship. It should be better to compose the predicted nonlinear models with considering correlation between the responses for multi-response data sets. For this purpose, seemingly unrelated nonlinear regression (SUNR) have been widely used in the literature. The parameter estimation procedure of the SUNR is based on nonlinear least squares (NLS) method, based on L 2-norm. However, it is possible to use different norms for parameter estimation process. The novelty of this study is presenting the applicability of least absolute deviation (LAD) method, defined in L 1-norm, with the NLS method simultaneously for obtaining parameter estimates of the SUNR model in a multi objective perspective. In this study, the proposed multi-objective SUNR model is called MO-SUNR. The optimization of the MO-SUNR model is achieved by using soft computing methods. Two data set examples are given for application purposes of the MO-SUNR model. It is seen from the results that the MO-SUNR provides many alternatively usable compromise parameter estimates through the simultaneous evaluation of the LAD and the NLS methods.