Penalized least absolute deviations estimation for nonlinear model with change-points

Smooth least absolute deviation estimators for outlier-proof identification

In this paper we consider the problem of testing statistical hypotheses for unknown parameters in nonlinear regression models and propose three asymptotically equivalent tests based on regression quantiles estimators, which are Wald test, Lagrange Multiplier test and Likelihood Ratio test. We also derive the asymptotic distributions of the three test statistics both under the null hypotheses and under a sequence of local alternatives and verify that the asymptotic relative efficiency of the proposed test statistics with classical test based on least squares depends on the error distributions of the regression models. We give some examples to illustrate that the test based on the regression quantiles estimators performs better than the test based on the least squares estimators of the least absolute deviation estimators when the disturbance has asymmetric and heavy-tailed distribution.

For autoregressive moving average (ARMA) models with infinite variance innovations, quasi-likelihood-based estimators (such as Whittle estimators) suffer from complex asymptotic distributions depending on unknown tail indices. This makes statistical inference for such models difficult. In contrast, the least absolute deviations estimators (LADE) are more appealing in dealing with heavy tailed processes. In this paper, we propose a weighted least absolute deviations estimator (WLADE) for ARMA models. We show that the proposed WLADE is asymptotically normal, is unbiased, and has the standard root-n convergence rate even when the variance of innovations is infinity. This paves the way for statistical inference based on asymptotic normality for heavy-tailed ARMA processes. For relatively small samples numerical results illustrate that the WLADE with appropriate weight is more accurate than the Whittle estimator, the quasi-maximum-likelihood estimator (QMLE), and the Gauss–Newton estimator when the innovation variance is infinite and that the efficiency loss due to the use of weights in estimation is not substantial.The authors thank the two referees for their valuable suggestions. The work was partially supported by an EPSRC research grant (UK) and the Natural Science Foundation of China (grant 10471005).

This paper considers a local least absolute deviation estimation for unit root processes with generalized autoregressive conditional heteroskedastic (GARCH) errors and derives its asymptotic properties under only finite second-order moment for both errors and innovations. When the innovations are symmetrically distributed, the asymptotic distribution of the estimated unit root is shown to be a functional of a bivariate Brownian motion, and then two unit root tests are derived. The simulation results demonstrate that the tests outperform those based on the Gaussian quasi maximum likelihood estimators with heavy-tailed innovations and those based on the simple least absolute deviation estimators.

We develop analogs of the two classes of weighted empirical minimum distance (m.d.) estimators of the underlying parameters in linear and nonlinear regression models when covariates are observed with Berkson measurement error. One class is based on the integral of the square of symmetrized weighted empirical of residuals while the other is based on a similar integral involving a weighted empirical of residual ranks. The former class requires the regression and measurement errors to be symmetric around zero while the latter class does not need any such assumption. The first class of estimators includes the analogs of the least absolute deviation and Hodges-Lehmann estimators while the second class includes an estimator that is asymptotically more efficient than these two estimators at some error distributions when there is no measurement error. In the case of linear model, no knowledge of the measurement error distribution is needed. We also develop these estimators for nonlinear models when the measurement error distribution is known and when it is unknown but validation data is available.

We study the problem of linear regression where both covariates and responses are potentially (i) heavy-tailed and (ii) adversarially contaminated. Several computationally efficient estimators have been proposed for the simpler setting where the covariates are sub-Gaussian and uncontaminated; however, these estimators may fail when the covariates are either heavy-tailed or contain outliers. In this work, we show how to modify the Huber regression, least trimmed squares, and least absolute deviation estimators to obtain estimators which are simultaneously computationally and statistically efficient in the stronger contamination model. Our approach is quite simple, and consists of applying a filtering algorithm to the covariates, and then applying the classical robust regression estimators to the remaining data. We show that the Huber regression estimator achieves near-optimal error rates in this setting, whereas the least trimmed squares and least absolute deviation estimators can be made to achieve near-optimal error after applying a postprocessing step. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

A genetic method of LAD estimation for models with censored data

The paper presents the calculation method of an asymptotic relative efficiency of the least absolute deviations estimate with respect to the maximum likelihood estimate for the parameter presented in the first order autoregressive model with a stochastic coefficient. The method is based on approximation of its asymptotic relative efficiency by Taylor series. The paper considers an example of the asymptotic relative efficiency calculation for the case where diffusion of the innovation process has an approximate Gaussian distribution (Tukey distribution). It is found out that if the assumptions of the Gaussian process are performed approximately, the maximum likelihood estimate is more efficient than the least absolute deviation estimate

In this article, we consider the estimation of the parameters of a multiple-component elementary chirp model based on weighted least squares estimators (WLSEs). Recently, Mittal et al. (2023) have proposed least squares estimators (LSEs) to estimate the parameters of the elementary chirp model. However, it has been observed that LSEs are quite sensitive to outliers. The least absolute deviation estimators and Huber's M-estimators are robust estimators for estimation in the presence of outliers. However, computing these robust estimators is numerically challenging, and deriving their statistical properties is not straightforward under the same error assumption as LSEs. We prove the strong consistency and asymptotic normality of WLSEs under the same error assumptions as those of LSEs. We have also considered sequential WLSEs to estimate the parameters, which are less computationally involved and have the same asymptotic properties as WLSEs. Based on extensive numerical studies, it has been observed that the behavior of the proposed estimators is better than that of LSEs in the presence of outliers. We also discuss the choice of weight function, as the performance of the proposed estimators depends on the weight function. For illustration, we have presented a simulation study, where outliers are present anywhere in the data, not only in a specific portion. We have also analyzed a synthetic dataset and a real dataset. The performance is quite satisfactory.

This paper empirically investigates the determinants of firms’ performance in the agri-food sector by using recent survey data for Denmark. Treating sales per employee as a proxy for value addition we estimate several bootstrapped regression models to draw conclusions on the marginal effects of potential performance determinants such as the form and nature of ownership, stage of the food chain and commodity sector, new product development, staff quality, firms’ competitive stance, and elements of firms’ strategy. To draw robust inferences we apply, besides the ordinary heteroscedasticity corrected Tobit ML-estimator, a nonparametric least absolute deviations estimator (LAD/CLAD) based on a quantile regression procedure. The results indicate that we cannot reject the hypothesis of no influence of dominant orientation on value added. Rather, firms’ focus on human capital, stage and commodity sector better explains their value addition. We can reject the hypothesis that regional networks have no influence on value added. Differences in location, emphasis on human capital and the negative influence of outsourcing on value added all provide supporting evidence. We reject the hypothesis of no influence of FDI, and moreover propose that FDI has targeted the domestic Danish market as a source of value added.

This article suggests very simple three-step estimators for censored quantile regression models with a separation restriction on the censoring probability. The estimators are theoretically attractive (i.e., asymptotically as efficient as the celebrated Powell's censored least absolute deviation estimator). At the same time, they are conceptually simple and have trivial computational expenses. They are especially useful in samples of small size or models with many regressors, with desirable finite-sample properties and small bias. The separation restriction costs a small reduction of generality relative to the canonical censored regression quantile model, yet its main plausible features remain intact. The estimator can also be used to estimate a large class of traditional models, including the normal Amemiya–Tobin model and many accelerated failure and proportional hazard models. We illustrate the approach with an extramarital affairs example and contrast our findings with those of Fair.

This note considers a three‐step non‐Gaussian quasi‐maximum likelihood estimation (TS‐NGQMLE) of the double autoregressive model with its asymptotics, which improves efficiency of the GQMLE and circumvents inconsistency of the NGQMLE when the innovation is heavy‐tailed. Under mild conditions, the estimator not only can achieve consistency and asymptotic normality regardless of density misspecification of the innovation, but also outperforms the existing estimators, such as the GQMLE and the (weighted) least absolute deviation estimator, when the innovation is indeed heavy‐tailed.

On the robustness of two alternatives to least squares: A Monte Carlo study

In this paper we study the weighted least absolute deviations estimator (WLADE) for Periodic ARMA (PARMA) models with unspecified noises, asymptotic normality of the estimator is derived. The random weighting approach is proposed to estimate the asymptotic covariance matrices of WLADE. Simulation study is carried out to examine the performance of the proposed procedure.

We relate the asymptotic behavior of $M$-estimators of the regression parameter in a linear model in which the dimension of the regression parameter may increase with the sample size to the stochastic equicontinuity of an associated $M$-process. The approach synthesises a number of results for the dimensionally fixed regression model and then extends these results in a direct unified way. The resulting theorems require only mild conditions on the $\psi$-function and the underlying distribution function. In particular, the results do not require $\psi$ to be smooth and hence can be applied to such estimators as the least absolute deviations estimator. We also treat one-step $M$-estimation.