Heteroskedasticity-Consistent Covariance Matrix Estimators in Small Samples with High Leverage Points

Problem statement: High leverage points are extreme outliers in the X-direction. In regression analysis, the detection of these leverage points becomes important due to their arbitrary large effects on the estimations as well as multicollinearity problems. Mahalanobis Distance (MD) has been used as a diagnostic tool for identification of outliers in multivariate analysis where it finds the distance between normal and abnormal groups of the data. Since the computation of MD relies on non-robust classical estimations, the classical MD can hardly detect outliers accurately. As an alternative, Robust MD (RMD) methods such as Minimum Covariance Determinant (MCD) and Minimum Volume Ellipsoid (MVE) estimators had been used to identify the existence of high leverage points in the data set. However, these methods tended to swamp some low leverage points even though they can identify high leverage points correctly. Since, the detection of leverage points is one of the most important issues in regression analysis, it is imperative to introduce a novel detection method of high leverage points. Approach: In this study, we proposed a relatively new two-step method for detection of high leverage points by utilizing the RMD (MVE) and RMD (MCD) in the first step to identify the suspected outlier points. Then, in the second step the MD was used based on the mean and covariance of the clean data set. We called this method two-step Robust Diagnostic Mahalanobis Distance (RDMDTS) which could identify high leverage points correctly and also swamps less low leverage points. Results: The merit of the newly proposed method was investigated extensively by real data sets and Monte Carlo Simulations study. The results of this study indicated that, for small sample sizes, the best detection method is (RDMDTS) (MVE)-mad while there was not much difference between (RDMDTS) (MVE)-mad and (RDMDTS) (MCD)-mad for large sample sizes. Conclusion/Recommendations: In order to swamp less low leverage as high leverage point, the proposed robust diagnostic methods, (RDMDTS) (MVE)-mad and (RDMDTS) (MCD)-mad were recommended.

Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties

We consider the issue of performing testing inferences on the parameters that index the linear regression model under heteroskedasticity of unknown form. Quasi-t test statistics use asymptotically correct standard errors obtained from heteroskedasticity-consistent covariance matrix estimators. An alternative approach involves making an assumption about the functional form of the response variances and jointly modelling mean and dispersion effects. In this paper we compare the accuracy of testing inferences made using the two approaches. We consider several different quasi-t tests and also z tests performed after estimated generalized least squares estimation which was carried out using three different estimation strategies. The numerical evidence shows that some quasi-t tests are typically considerably less size distorted in small samples than the tests carried out after the jointly modelling of mean and dispersion effects. Finally, we present and discuss two empirical applications.

Regression Quantiles (RQs) (see Koenker and Bassett, 1978) can be found as optimal solutions to a Linear Programming (LP) problem. Also, these optimal solutions correspond to specific elemental regressions (ERs). On the other hand, single case ordinary least squares (OLS) leverage statistics can be expressed as weighted averages of ER ones. Using this three-tier relationship amongst RQs, ERs, and OLS leverage statistics some relationships between single case leverage statistics and ER ones are explored and deduced. We build upon these results and propose a multiple-case RQ weighted predictive leverage statistic, TJ. We do this using an ER view of the well-known leverage relationship, , by summing the ER weighted predictive leverage statistics over all ERs (RQs included) instead of over observations, i.e., . As an ad-hoc cut-off value of this statistic we make use of the analog of the Hoaglin and Welsch (1978) one, i.e., high leverage points have . So in the RQ weighted predictive leverage scenario, the cut-off value becomes , where K is the total number of ERs. We then apply this RQ high leverage diagnostic to well-known data sets in the literature. The cut-off value used generally seems too small. Some proposals of cut-off values based on some analytical bounds and a simulation study are therefore given and shown to be reasonable.

The specification of Smooth Transition Regression models consists of a sequence of tests, which are typically based on the assumption of i.i.d. errors. In this paper we examine the impact of conditional heteroskedasticity and investigate the performance of several heteroskedasticity robust versions. Simulation evidence indicates that conventional tests can frequently result in finding spurious nonlinearity. Conversely, when the true process is nonlinear in mean, the tests appear to have low size adjusted power and can lead to the selection of misspecified models. The above deficiencies also hold for tests based on Heteroskedasticity Consistent Covariance Matrix Estimators but not for the Fixed Design Wild Bootstrap. We highlight the importance of robust inference through empirical applications.

This article considers the issue of performing tests in linear heteroskedastic models when the test statistic employs a consistent variance estimator. Several different estimators are considered, namely: HC0, HC1, HC2, HC3, and their bias-adjusted versions. The numerical evaluation is performed using numerical integration methods; the Imhof algorithm is used to that end. The results show that bias-adjustment of variance estimators used to construct test statistics delivers more reliable tests when they are performed for the HC0 and HC1 estimators, but the same does not hold for the HC3 estimator. Overall, the most reliable test is the HC3-based one.

The heteroskedasticity-consistent covariance matrix estimator proposed by White [White, H. A. (1980). Heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 48:817–838], also known as HC0, is commonly used in practical applications and is implemented into a number of statistical software. Cribari–Neto et al. [Cribari–Neto, F., Ferrari, S. L. P., Cordeiro, G. M. (2000). Improved heteroscedasticity–consistent covariance matrix estimators. Biometrika 87:907–918] have developed a bias-adjustment scheme that delivers bias-corrected White estimators. There are several variants of the original White estimator that are also commonly used by practitioners. These include the HC1, HC2, and HC3 estimators, which have proven to have superior small-sample behavior relative to White's estimator. This paper defines a general bias-correction mechamism that can be applied not only to White's estimator, but to variants of this estimator as well, such as HC1, HC2, and HC3. Numerical evidence on the usefulness of the proposed corrections is also presented. Overall, the results favor the sequence of improved HC2 estimators.

This article compares heteroskedasticity-consistent covariance matrix estimators in finite samples for stochastic regression models where explanatory variables are random and not given. The effect of non independence of explanatory variables and error term and heteroskedasticity is studied on heteroskedasticity-consistent covariance matrix estimators. Under non independence or heteroskedasticity, it is found that a weighted bootstrap estimator, estimators known as HC2 and HC3, and the jackknife work well in estimation of variances while HC3 and the jackknife work well in sizes of quasi t-statistics.

Detection of multiple unusual observations such as outliers, high leverage points and influential observations (IOs) in regression is still a challenging task for statisticians due to the well-known masking and swamping effects. In this paper we introduce a robust influence distance that can identify multiple IOs, and propose a sixfold plotting technique based on the well-known group deletion approach to classify regular observations, outliers, high leverage points and IOs simultaneously in linear regression. Experiments through several well-referred data sets and simulation studies demonstrate that the proposed algorithm performs successfully in the presence of multiple unusual observations and can avoid masking and/or swamping effects.

High leverage points can induce or disrupt multicollinearity patterns in data. Observations responsible for this problem are generally known as collinearity-influential observations. A significant amount of published work on the identification of collinearity-influential observations exists; however, we show in this article that all commonly used detection techniques display greatly reduced sensitivity in the presence of multiple high leverage collinearity-influential observations. We propose a new measure based on a diagnostic robust group deletion approach. Some practical cutoff points for existing and developed diagnostics measures are also introduced. Numerical examples and simulation results show that the proposed measure provides significant improvement over the existing measures.

In this chapter, we discuss large sample theory that can be applied under conditions that are quite likely to be met in large samples even when the Gauss–Markov conditions are broken. There are two reasons for using large sample theory. First, there may be some problems that corrupt our estimators in small samples but tends to diminish down as the sample gets bigger. Thus, if we cannot get a perfect small sample estimator, we will usually want to choose the one that will be best in large samples. Second, in some circumstances, the theory used to derive the properties of estimators in small samples just does not work, and working out the properties of the estimators can be impossible. This makes it very hard to choose between alternative estimators. In these circumstances we judge different estimators on their “large sample properties” because their “small (or finite) sample properties” are unknown.

High leverage points in the data set have massive effect in linear regression. Identifications of high leverage points (HLP) is very important due to their effect of causing wrong conclusion in regression model. The Diagnostic Robust Generalized Potential based on Index Set Equality IDRGP (ISE) is very successful in identifying high leverage points with less running time. However, it still reveals smaller rate of masking and swamping. A fast Improvised method for the identification of High Leverage Point is proposed to reduce the effect of swamping and masking at a faster rate of running time. The results of the simulation study show the merit of our proposed method.

The minimum covariance determinant (MCD) estimator is a highly robust estimator of multivariate location and scatter. It can be computed efficiently with the FAST‐MCD algorithm of Rousseeuw and Van Driessen. Since estimating the covariance matrix is the cornerstone of many multivariate statistical methods, the MCD has also been used to develop robust and computationally efficient multivariate techniques.In this paper, we review the MCD estimator, along with its main properties such as affine equivariance, breakdown value, and influence function. We discuss its computation, and list applications and extensions of the MCD in theoretical and applied multivariate statistics. Copyright © 2009 John Wiley & Sons, Inc.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Robust Methods

The feasible solution algorithm for the minimum covariance determinant estimator in multivariate data

ABSTRACTEmpirical evidence suggests that lack of access to financing is a major constraint to performance of female-owned firms in most countries. Firm performance, financing structure, and constraints have been well explored in the case of firms in developed economies but remain understudied in the case of firms in developing countries, especially in Africa and the Middle East. Largely due to lack of relevant cross-country financial data, existing literature on African firms has presented some survey-based evidence on firm performance and financing structures, while evidence based on detailed financial information remains lacking. This study aims at filling this research gap. We identify female-owned firms operating in the formal sector and examine the impact of ownership structure on financing and firm performance. We use a panel of financial data covering 25,500 companies in the Middle East and Africa for the years 2006 to 2015. Our results reveal a clear, but perhaps surprising, gender-specific pattern. Increased availability of equity and/or debt capital as well as higher leverage has significant positive effects on firm performance, measured by sales, profits and returns on equity. Female ownership per se appears to lower firm performance significantly, according to all measures used even when we control for the levels of available capital and the degrees of leverage. However, when the interplay of female ownership with capital availability is taken into account, we find that this interaction has a positive impact, implying that women are actually more productive when they are equally able to access finance. These results are also confirmed for a subsample including only Sub-Saharan African countries.