Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties

The aim of this paper is to demonstrate the impact of high leverage observations on the performances of prominent and popular Heteroskedasticity-Consistent Covariance Matrix Estimators (HCCMEs) with the help of computer simulation. Firstly, we figure out high leverage observations, then remove them and recalculate the HCCMEs without these observations in order to compare the HCCME performances with and without high leverage points. We identify high leverage observations with the Minimum Covariance Determinant (MCD). We select from among different covariates and disturbance term variances from the related literature in simulation runs in order to compare the percentage difference between the expected value of the HCCME and true covariance matrix as well as the symmetric loss function. Our results revealed that the elimination of high leverage (high MCD distance) observations had improved the HCCME performances considerably and under some settings substantially, depending on the degree of leverage. We hope our theoretical findings will be benefited for practical purposes in applications.

The heteroskedasticity-consistent covariance matrix estimator proposed by White [White, H. A. (1980). Heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 48:817–838], also known as HC0, is commonly used in practical applications and is implemented into a number of statistical software. Cribari–Neto et al. [Cribari–Neto, F., Ferrari, S. L. P., Cordeiro, G. M. (2000). Improved heteroscedasticity–consistent covariance matrix estimators. Biometrika 87:907–918] have developed a bias-adjustment scheme that delivers bias-corrected White estimators. There are several variants of the original White estimator that are also commonly used by practitioners. These include the HC1, HC2, and HC3 estimators, which have proven to have superior small-sample behavior relative to White's estimator. This paper defines a general bias-correction mechamism that can be applied not only to White's estimator, but to variants of this estimator as well, such as HC1, HC2, and HC3. Numerical evidence on the usefulness of the proposed corrections is also presented. Overall, the results favor the sequence of improved HC2 estimators.

We consider the issue of performing testing inferences on the parameters that index the linear regression model under heteroskedasticity of unknown form. Quasi-t test statistics use asymptotically correct standard errors obtained from heteroskedasticity-consistent covariance matrix estimators. An alternative approach involves making an assumption about the functional form of the response variances and jointly modelling mean and dispersion effects. In this paper we compare the accuracy of testing inferences made using the two approaches. We consider several different quasi-t tests and also z tests performed after estimated generalized least squares estimation which was carried out using three different estimation strategies. The numerical evidence shows that some quasi-t tests are typically considerably less size distorted in small samples than the tests carried out after the jointly modelling of mean and dispersion effects. Finally, we present and discuss two empirical applications.

The specification of Smooth Transition Regression models consists of a sequence of tests, which are typically based on the assumption of i.i.d. errors. In this paper we examine the impact of conditional heteroskedasticity and investigate the performance of several heteroskedasticity robust versions. Simulation evidence indicates that conventional tests can frequently result in finding spurious nonlinearity. Conversely, when the true process is nonlinear in mean, the tests appear to have low size adjusted power and can lead to the selection of misspecified models. The above deficiencies also hold for tests based on Heteroskedasticity Consistent Covariance Matrix Estimators but not for the Fixed Design Wild Bootstrap. We highlight the importance of robust inference through empirical applications.

This article compares heteroskedasticity-consistent covariance matrix estimators in finite samples for stochastic regression models where explanatory variables are random and not given. The effect of non independence of explanatory variables and error term and heteroskedasticity is studied on heteroskedasticity-consistent covariance matrix estimators. Under non independence or heteroskedasticity, it is found that a weighted bootstrap estimator, estimators known as HC2 and HC3, and the jackknife work well in estimation of variances while HC3 and the jackknife work well in sizes of quasi t-statistics.

This article considers the issue of performing tests in linear heteroskedastic models when the test statistic employs a consistent variance estimator. Several different estimators are considered, namely: HC0, HC1, HC2, HC3, and their bias-adjusted versions. The numerical evaluation is performed using numerical integration methods; the Imhof algorithm is used to that end. The results show that bias-adjustment of variance estimators used to construct test statistics delivers more reliable tests when they are performed for the HC0 and HC1 estimators, but the same does not hold for the HC3 estimator. Overall, the most reliable test is the HC3-based one.

In this chapter, we discuss large sample theory that can be applied under conditions that are quite likely to be met in large samples even when the Gauss–Markov conditions are broken. There are two reasons for using large sample theory. First, there may be some problems that corrupt our estimators in small samples but tends to diminish down as the sample gets bigger. Thus, if we cannot get a perfect small sample estimator, we will usually want to choose the one that will be best in large samples. Second, in some circumstances, the theory used to derive the properties of estimators in small samples just does not work, and working out the properties of the estimators can be impossible. This makes it very hard to choose between alternative estimators. In these circumstances we judge different estimators on their “large sample properties” because their “small (or finite) sample properties” are unknown.

For testing separate families of hypotheses, the likelihood ratio test does not have the usual asymptotic properties. This paper considers the asymptotic distribution of the ratio of maximized likelihoods (RML) statistic in the special case of testing separate scale or location-scale families of distributions. We derive saddlepoint approximations to the density and tail probabilities of the log of the RML statistic. These approximations are based on the expansion of the log of the RML statistic up to the second order, which is shown not to depend on the location and scale parameters. The resulting approximations are applied in several cases, including normal versus Laplace, normal versus Cauchy, and Weibull versus log-normal. Our results show that the saddlepoint approximations are satisfactory, even for fairly small sample sizes, and are more accurate than normal approximations and Edgeworth approximations, especially for tail probabilities that are the values of main interest in hypothesis testing problems.

Regularization is a solution to solve the problem of unstable estimation of covariance matrix with a small sample set in Gaussian classifier. And multi-regularization parameters estimation is more difficult than single parameter estimation. In this paper, KLIM_L covariance matrix estimation is derived theoretically based on MDL (minimum description length) principle for the small sample problem with high dimension. KLIM_L is a generalization of KLIM (Kullback-Leibler information measure) which considers the local difference in each dimension. Under the framework of MDL principle, multi-regularization parameters are selected by the criterion of minimization the KL divergence and estimated simply and directly by point estimation which is approximated by two-order Taylor expansion. It costs less computation time to estimate the multi-regularization parameters in KLIM_L than in RDA (regularized discriminant analysis) and in LOOC (leave-one-out covariance matrix estimate) where cross validation technique is adopted. And higher classification accuracy is achieved by the proposed KLIM_L estimator in experiment.

Models that allow for autoregressive conditional heteroskedasticity (ARCH) in the error process have recently found widespread application. The purpose of this paper is to evaluate, through Monte Carlo analysis, the small sample properties of an exact Lagrange multiplier test for the presence of ARCH, and to compare the power of this test with that of an asymptotically equivalent TR2 version. The comparison involves first-and higher-order variants of these processes. The results indicate substantial power differentials in favor of the exact LM test, by up to 15% for sample sizes smaller than 100.

Tests for error autocorrelation (AC) are derived under the assumption of independent and identically distributed errors. The tests are not asymptotically valid if the errors are conditionally heteroskedastic. In this article we propose wild bootstrap (WB) Lagrange multiplier tests for error AC in vector autoregressive (VAR) models. We show that the WB tests are asymptotically valid under conditional heteroskedasticity of unknown form. WB tests based on a version of the heteroskedasticity-consistent covariance matrix estimator are found to have the smallest error in rejection probability under the null and high power under the alternative. We apply the tests to VAR models for credit default swap prices and Euribor interest rates. An important result that we find is that the WB tests lead to parsimonious models while the asymptotic tests suggest that a long lag length is required to get white noise residuals.

Journal Article Nonparametric estimation of large covariance matrices of longitudinal data Get access Wei Biao Wu, Wei Biao Wu Search for other works by this author on: Oxford Academic Google Scholar Mohsen Pourahmadi Mohsen Pourahmadi Search for other works by this author on: Oxford Academic Google Scholar Biometrika, Volume 90, Issue 4, December 2003, Pages 831–844, https://doi.org/10.1093/biomet/90.4.831 Published: 01 December 2003

Covariance matrix estimation is one of the most important problems in statistics. To accommodate the complexity of modern datasets, it is desired to have estimation procedures that not only can incorporate the structural assumptions of covariance matrices, but are also robust to outliers from arbitrary sources. In this paper, we define a new concept called matrix depth and then propose a robust covariance matrix estimator by maximizing the empirical depth function. The proposed estimator is shown to achieve minimax optimal rate under Huber’s $\\varepsilon$-contamination model for estimating covariance/scatter matrices with various structures including bandedness and sparsity.

The authors address likelihood ratio statistics used to test simultaneously conditions on mean vectors and patterns on covariance matrices. Tests for conditions on mean vectors, assuming or not a given structure for the covariance matrix, are quite common, since they may be easily implemented. But, on the other hand, the practical use of simultaneous tests for conditions on the mean vectors and a given pattern for the covariance matrix is usually hindered by the nonmanageability of the expressions for their exact distribution functions. The authors show the importance of being able to adequately factorize the c.f. of the logarithm of likelihood ratio statistics in order to obtain sharp and highly manageable near-exact distributions, or even the exact distribution in a highly manageable form. The tests considered are the simultaneous tests of equality or nullity of means and circularity, compound symmetry, or sphericity of the covariance matrix. Numerical studies show the high accuracy of the near-exact distributions and their adequacy for cases with very small samples and/or large number of variables. The exact and near-exact quantiles computed show how the common chi-square asymptotic approximation is highly inadequate for situations with small samples or large number of variables.

A well-known ad-hoc approach to conducting structural equation modeling with missing data is to obtain a saturated maximum likelihood (ML) estimate of the population covariance matrix and then to use this estimate in the complete data ML fitting function to obtain parameter estimates. This 2-stage (TS) approach is appealing because it minimizes a familiar function while being only marginally less efficient than the full information ML (FIML) approach. Additional advantages of the TS approach include that it allows for easy incorporation of auxiliary variables and that it is more stable in smaller samples. The main disadvantage is that the standard errors and test statistics provided by the complete data routine will not be correct. Empirical approaches to finding the right corrections for the TS approach have failed to provide unequivocal solutions. In this article, correct standard errors and test statistics for the TS approach with missing completely at random and missing at random normally distributed data are developed and studied. The new TS approach performs well in all conditions, is only marginally less efficient than the FIML approach (and is sometimes more efficient), and has good coverage. Additionally, the residual-based TS statistic outperforms the FIML test statistic in smaller samples. The TS method is thus a viable alternative to FIML, especially in small samples, and its further study is encouraged.