Nonparametric M-regression with free knot splines

ABSTRACTThe paper investigates various nonparametric models including regression, conditional distribution, conditional density and conditional hazard function, when the covariates are infinite dimensional. The main contribution is to prove uniform in bandwidth asymptotic results for kernel estimators of these functional operators. Then, the application issues, involving data-driven bandwidth selection, are discussed.

Extended linear models form a very general framework for statistical modeling. Many practically important contexts fit into this framework, including regression, logistic and Poisson regression, density estimation, spectral density estimation, and conditional density estimation. Moreover, hazard regression, proportional hazard regression, marked point process regression, and diffusion processes, all perhaps with time-dependent covariates, also fit into this framework. Polynomial splines and their tensor products provide a universal tool for constructing maximum likelihood estimates for extended linear models. The theory of rates of convergence for such estimates as it applies both to fixed knot splines and to free knot splines will be surveyed, and the implications of this theory for the development of corresponding methodology will be discussed briefly.

Least absolute deviation (LAD) is a well-known criterion to fit statistical models, but little is known about LAD estimation in structural equation modeling (SEM). To address this gap, the authors use the LAD criterion in SEM by minimizing the sum of the absolute deviations between the observed and the model-implied covariance matrices. Using Monte Carlo simulations, the authors compare the performance of this LAD estimator along several dimensions (bias, efficiency, convergence, frequencies of improper solutions, and absolute percentage deviation) to the full information maximum likelihood (ML) and unweighted least squares (ULS) estimators in structural equation modeling. The results for LAD are mixed: There are special conditions under which the LAD estimator outperforms ML and ULS, but the simulation evidence does not support a general claim that LAD is superior to ML and ULS in small samples.

We consider an L 1 analogue of the least squares estimator for the parameters of stationary, finite-order autoregressions. This estimator, the least absolute deviation (LAD), is shown to be strongly consistent via a result that may have independent interest. The striking feature is that the conditions are so mild as to include processes with infinite variance, notably the stationary, finite autoregressions driven by stable increments in Lα, α > 1. Finally, sampling properties of LAD are compared to those of least squares. Together with a known convergence rate result for least squares, the Monte Carlo study provides evidence for a conjecture on the convergence rate of LAD.

SUMMARY The case-cohort design is a common means of reducing cost in large epidemiological cohort studies. Under this design, covariates are measured only on the cases and a subcohort randomly selected from the entire cohort. In this paper, we demonstrate how to use the case-cohort data to estimate the regression parameter of the additive hazards model, which specifies that the conditional hazard function given a set of covariates is the sum of an arbitrary baseline hazard function and a regression function of the covariates. The proposed estimator is shown to be consistent and asymptotically normal with an easily estimated variance. The subcohort may be selected by independent Bernoulli sampling with arbitrary selection probabilities or by stratified simple random sampling. The efficiencies of various sampling schemes are investigated both analytically and by simulation. A real example is provided.

Anomalies of monthly mean high level, low level, and total cloud cover; outgoing longwave radiation (OLR); and reflected solar radiation were related to sea surface temperature anomalies (SSTAs) using a least absolute deviations (LAD) regression at each grid point over the open ocean for a 6‐year period. Identical LAD analyses were performed (1) on actual observations and (2) on output from the Hamburg general circulation model that was run for the same time period with prescribed observed sea surface temperatures. The slope of the LAD regression line relating SSTA and each variable was used to characterize the relationship between them. A multivariate randomized block permutation test was performed to determine how well the regression line represented the data. Comparisons of the results indicated that near the equator the model's total and high level percent of cloud cover increased in association with a 1°C increase in SSTA at a rate very similar to that observed. The model's rate of change in OLR in association with change in SSTA was also very similar to that obtained from the observed OLR data. However, the model's reflected solar radiation near the equator changed much more than is observed in response to small SSTA changes. In the subtropics and middle latitudes, low level clouds were observed to have a distinct relationship with SSTAs that was not present in the model.

Asymptotics for argmin processes: Convexity arguments

In order to identify the parameters of nonlinear Hammerstein model which are contaminated by colored noise and peak noise, the least absolute deviation (LAD) is selected as the objective function to solve the problem of large residual square when the identification data is disturbed by the impulse noise which obeys symmetrical alpha stable (SαS) distribution. However, LAD cannot meet the need of differentiability required by most algorithms. To improve robustness and to solve the nondifferentiable problem, an approximate least absolute deviation (ALAD) objective function is established by introducing a deterministic function to replace absolute value under certain situations. The proposed method is derived from ALAD criterion and extended stochastic gradient method. Due to the differentiability of the objective function, we can get a recursive identification algorithm which is simple and easy to calculate compared with LAD. The convergence of the proposed identification method is also proved by Lyapunov stability theory, and the simulation experiments show that the proposed method has higher accuracy and stronger robustness than the least square (LS) method in the identification of Hammerstein model with colored noise and impulse noise. The impact of impulse noise can be restrained effectively.

Compact representation of signals and images is a key for many applications. Compactness is often achieved through linear transforms with good energy concentration property. We present an adaptive wavelet filter bank with fixed number of vanishing moments, plus additional local adaptation. Proposed adaptation method is conducted at each sample according to the least absolute deviation (LAD) criterion. Fixed vanishing moments provide for polynomial annihilation. Adaptation is aimed to achieve maximum sparseness for a wider class of signals, such as sine waves. LAD criterion results in more accurate adaptation on sudden changes of signal statistics. In this paper, an efficient LAD realization is proposed, in spite of nonexistence of the closed form solution. Combining least squares and LAD criterion, we have achieved unbiased adaptation, robust to noise. Due to its simplicity and acceptable computational speed, the proposed scheme is a good candidate for the real-world applications. In this paper, advantages of the proposed scheme are shown in signal denoising and reconstruction.

Local Walsh-average regression

We propose novel tensor decomposition methods that advocate both properties of sparsity and robustness to outliers. The sparsity enables us to extract some essential features from a big data that are easily interpretable. The robustness ensures the resistance to outliers that appear commonly in high-dimensional data. We first propose a method that generalizes the ridge regression in M-estimation framework for tensor decompositions. The other approach we propose combines the least absolute deviation (LAD) regression and the least absolute shrinkage operator (LASSO) for the CANDECOMP/PARAFAC (CP) tensor decompositions. We also formulate various robust tensor decomposition methods using different loss functions. The simulation study shows that our robust-sparse methods outperform other general tensor decomposition methods in the presence of outliers.

This paper introduces a least absolute deviation (LAD) regression model suitable for manage interval-valued data. Each example of the data set is described by a feature vector where each feature value is an interval. In the approach, it is fitted two LAD regressions, respectively, on the mid-point and range of the interval values assumed by the variables. The prediction of the lower and upper bound of the interval value of the dependent variable is accomplished from its mid-point and range which are estimated from the fitted LAD regression models applied to the mid-point and range of each interval values of the independent variables. The evaluation of the proposed prediction method is based on the estimation of the average behaviour of root mean squared error and of the correlation coefficient in the framework of a Monte Carlo experience in comparison with the method proposed in [5].KeywordsInterval-Valued DataLAD RegressionSymbolic Data Analysis

This paper considers the problem of robust modeling by using the well-known Least Absolute Deviation (LAD) regression. For that purpose, the approximation function is designed and analyzed, which is based on a certain component weight of the Weighted Median of Data. It is shown that the proposed approximation function is a piecewise constant function with finitely many pieces with respect to the model parameter. Thereby, an investigation of regions of constant values of the approximation function is conducted. It is established that the designed model based on the Component Weighted Median Absolute Deviations estimates a optimal model parameter on a finite set, which describes corresponding regions. Furthermore, the specified restriction of the approximation function is observed and analyzed, in order to examine the observed problem.

Say that a regression method is "unstable" at a data set if a small change in the data can cause a relatively large change in the fitted plane. A well-known example of this is the instability of least squares regression (LS) near (multi)collinear data sets. It is known that least absolute deviation (LAD) and least median of squares (LMS) linear regression can exhibit instability at data sets that are far from collinear. Clear-cut instability occurs at a "singularity"--a data set, arbitrarily small changes to which can substantially change the fit. For example, the collinear data sets are the singularities of LS. One way to measure the extent of instability of a regression method is to measure the size of its "singular set" (set of singularities). The dimension of the singular set is a tractable measure of its size that can be estimated without distributional assumptions or asymptotics. By applying a general theorem on the dimension of singular sets, we find that the singular sets of LAD and LMS are at least as large as that of LS and often much larger. Thus, prima facie, LAD and LMS are frequently unstable. This casts doubt on the trustworthiness of LAD and LMS as exploratory regression tools.

Common econometric estimators such as least squares, least absolute deviations (LAD), instrumental variables, maximum likelihood, and semiparametric estimators are non-robust against data contamination. Despite the known superiority of high-breakdown point (HBP) estimators in such situations, the HBP estimators have rarely been used in economics. This article presents some applications of an HBP estimator called the S-estimator (Rousseeuw and Yohai, Robust and Nonlinear Time Series Analysis (Eds) W.H. Franke and R.D. Martin, Springer-Verlag, NY, pp. 256–72, 1984) to the estimation of a linear regression model and compares the results with those obtained by ordinary least squares (OLS) and LAD methods. It is found that significance of variables as well as signs of coefficient estimates can be quite different under HBP estimation than under OLS and LAD estimation.