A robust hybrid ridge estimation framework using scaled error variance and M-estimation in contaminated linear models

Unemployment is one of the most important issues in macro economics. Unemployment creates many economic and social problems in the economy. The condition and qualification of labor force in a country show economical developments. In the light of these facts, a developing country should overcome the problem of unemployment. In this study, the performance of robust biased Robust Ridge Regression (RRR), Robust Principal Component Regression (RPCR) and RSIMPLS methods are compared with each other and their classical versions known as Ridge Regression (RR), Principal Component Regression (PCR) and Partial Least Squares Regression (PLSR) in terms of predictive ability by using trimmed Root Mean Squared Error (TRMSE) statistic in case of both of multicollinearity and outliers existence in an unemployment data set of Turkey. Analysis results show that RRR model is chosen as the best model for determining unemployment rate in Turkey for the period of 1985-2012. Robust biased RRR method showed that the most important independent variable effecting the unemployment rate is Purchasing Power Parities (PPP). The least important variables effecting the unemployment rate are Import Growth Rate (IMP) and Export Growth Rate (EXP). Hence, any increment in PPP cause an important increment in unemployment rate, however, any increment in IMP causes an unimportant increase in unemployment rate. Any increment in EXP causes an unimportant decrease in unemployment rate.

Finding a new drug which is both safe and efficient is an expensive and time-consuming endeavour. In particular, establishing the ‘dose-effect relationship’ – how beneficial a drug is at different dosages – can be challenging. Predicting this curve requires gathering experimental data by exposing and recording how cells respond to various levels of the drug. However, extreme values are often observed at low and high dosages, potentially introducing errors that are hard to correct in the prediction process. Yet, these extreme observations are sometimes genuine so researchers cannot just ignore them. To improve dose-effect estimation, Zhou, Liu, Fang et al. developed a new general-purpose approach. It uses advanced statistical modelling to account for extremes in lab data. This strategy outperformed other methods when dealing with these observations while also providing higher efficiency in data analysis with more uniform data in experiments. To facilitate implementation, Zhou, Liu, Fang et al. set up a user-friendly tool baptized ‘REAP’; this free online resource allows scientists without advanced statistical experience to harness the new approach and to perform dose-effect analysis more easily and accurately. This could boost research across many different disciplines that examine the effects of chemicals on cells.

The present investigation deals with the use of ridge regression in the presence of high degree multicollinearity, to determine the major factors influencing the agricultural production in Haryana state, India. Time series data for the period (1980–1981 to 2017–2018) on area, production, yield and different factors influencing the production of wheat, gram, paddy and cotton have been used in this study. Time series data was divided into four sub-periods of 10 year interval to measure the variability in area, production and yield of selected crops into different periods. Coefficient of variation around trend was also observed as the stability (or instability) index. Estimates for the factors affecting the agricultural production were observed by using the ordinary least squares technique and ridge regression both. A comparison is made and ridge estimates were found appropriate to measure the effect of different factors on agricultural production in the presence of multicollinearity. Total study period has been separated into training and testing parts and validity of the developed ridge regression model has been done on the basis of adjusted R2, percent deviation of forecast values and RMSE (%).

This paper introduces a new class of efficient and debiased two-step shrinkage estimators for a linear regression model in the presence of multicollinearity. We derive the proposed estimators’ mean square error and define the necessary and sufficient conditions for superiority over the existing estimators. In addition, we develop an algorithm for selecting the shrinkage parameters for the proposed estimators. The comparison of the new estimators versus the traditional ordinary least squares, ridge regression, Liu, and the two-parameter estimators is done by a matrix mean square error criterion. The Monte Carlo simulation results show the superiority of the proposed estimators under certain conditions. In the presence of high but imperfect multicollinearity, the two-step shrinkage estimators’ performance is relatively better. Finally, two real-world chemical data are analyzed to demonstrate the advantages and the empirical relevance of our newly proposed estimators. It is shown that the standard errors and the estimated mean square error decrease substantially for the proposed estimator. Hence, the precision of the estimated parameters is increased, which of course is one of the main objectives of the practitioners.

Multicollinearity arises when two or more regressors are correlated in multiple linear regression model (MLRM) and in most cases, one regressor variable can be predicted from another. Multicollinearity majorly results in inefficient regression model estimates and poor performance of the regression model. However, multicollinearity problem can easily be handled using various methods such as ridge regression, lasso regression, principal components regression, etc. This study compared the effectiveness of two estimators in handling multicollinearity problem in a given dataset. The estimators being compared are ridge estimator (RE) and principal components estimator (PCE). This research uses secondary data obtained from World Bank database, International Monetary Fund (IMF) database, and the Nigerian Debt Management Office to compare the two approaches of handling multicollinearity problem in MLRM. The presence of multicollinearity in the dataset was established using the correlation matrix of predictors and the Variance Inflation Factors (VIF's). Then ridge regression and principal components regression methods were used to fit models to the dataset respectively and their mean squared errors (MSE) were obtained. The MSE was used as performance evaluation measure for the regression models. Both methods addressed the problem multicollinearity in the datasets but the ridge estimator performed better than PCE by having the smallest mean squared error.

This paper introduces the Lasso and Ridge Regression methods, which are two popular regularization approaches. The method they give a penalty to the coefficients differs in both of them. L1 Regularization refers to Lasso linear regression, while L2 Regularization refers to Ridge regression. As we all know, regression models serve two main purposes: explanation and prediction of scientific phenomena. Where prediction accuracy will be optimized by balancing each of the bias and variance of predictions, while explanation will be gained by constructing interpretable regression models by variable selection. The penalized regression method, also known as Lasso regression, adds bias to the model's estimates and reduces variance to enhance prediction. Ridge regression, on the other hand, introduces a minor amount of bias in the data to get long-term predictions. In the presence of multicollinearity, both regression methods have been offered as an alternative to the least square approach (LS). Because they deal with multicollinearity, they have the appropriate properties to reduce numerical instability caused by overfitting. As a result, prediction accuracy can be improved. For this study, the Corona virus disease (Covid-19) dataset was used, which has had a significant impact on global life. Particularly in our region (Kurdistan), where life has altered dramatically and many people have succumbed to this deadly sickness. Our data is utilized to analyze the benefits of each of the two regression methods. The results show that the Lasso approach produces more accurate and dependable or reliable results in the presence of multicollinearity than Ridge and LS methods when compared in terms of accuracy of predictions by using NCSS10, EViews 12 and SPSS 25.

Walsh-average based variable selection for varying coefficient models

The multicollinearity in multiple linear regression models and the existence of influential data points are common problems. These problems exert undesirable effects on the least squares estimators. So, it is very important to introduce some alternative biased estimators of the robust ridge regression to overcome the influence of these problems simultaneously. In this paper, alternative biased robust regression estimator is defined by mixing the ridge estimation technique into the robust least median squares estimation to obtain the Ridge Least Median Squares (RLMS). The efficiency of the combined estimator (RLMS) is compared with some existing regression estimators, which namely, the Ordinary Least Squares (LS); Ridge Regression (RR) and Ridge Least Absolute Deviation (RLAD). The numerical results of this study show that, the RLMS regression estimator is more efficient than other estimators, based on, Bias and mean squared error criteria for many combinations of influential data points and degree of multicollinearity. Keywards: Influential Data Points; Multicollinearity; Ridge regression; Ridge Least Absolute Deviation; Ridge Least Median Squares estimation; Bias and Mean Squared Error criteria

This article introduces a novel method, called Graphical Group Ridge (GG-Ridge), which classifies ridge regression predictors in disjoint groups of conditionally correlated variables and derives different penalties (shrinkage parameters) for these groups of predictors. It combines the ridge regression method with the graphical model for high-dimensional data (i.e. the number of predictors, p, exceeds the number of cases, n) or ill-conditioned data (e.g. in the presence of multicollinearity among predictors). Although ridge regression is very effective with these types of data, its main shortcoming is that it applies the same penalty to all predictors, which can consequently limit the reduction in both the mean square error and the prediction mean square error, and over-shrink some predictors. This issue is addressed by the new method which reduces the mean square errors by assigning different penalties to different groups of predictors. Moreover, it reduces the extent of over-shrinking of predictors as compared to the ridge method, which is a desirable property in many fields such as finance, genetics and climate. The performance of the GG-Ridge method is investigated through two simulation studies and a real data analysis, and the results are compared with those of Ridge regression, and Elastic Net. The results indicate that the GG-Ridge outperforms these two methods in reducing the mean square errors, the prediction mean square error, and the bias of coefficients estimates.

In the multiple linear regression analysis, the ridge regression estimator and the Liu estimator are often used to address multicollinearity. Besides multicollinearity, outliers are also a problem in the multiple linear regression analysis. We propose new biased estimators based on the least trimmed squares (LTS) ridge estimator and the LTS Liu estimator in the case of the presence of both outliers and multicollinearity. For this purpose, a simulation study is conducted in order to see the difference between the robust ridge estimator and the robust Liu estimator in terms of their effectiveness; the mean square error. In our simulations, the behavior of the new biased estimators is examined for types of outliers: X-space outlier, Y-space outlier, and X-and Y-space outlier. The results for a number of different illustrative cases are presented. This paper also provides the results for the robust ridge regression and robust Liu estimators based on a real-life data set combining the problem of multicollinearity and outliers.

Condition-index based new ridge regression estimator for linear regression model with multicollinearity

The violation of the assumptions of no multicollinearity among predictors and homoscedasticity poses a very challenging situation in regression analysis. This situation leads to inflated variance and makes the inference of the coefficient insignificant. The ridge estimators generally involve the error variance obtained using the ordinary least square approach under homoscedasticity. In this study, we have designed a novel idea to use a heteroscedasticity consistent estimator when a joint issue of multicollinearity and heteroscedasticity is present in the data. This suggestion is simple and effective in improving the performance of the ridge estimators. The proposed suggestion of heteroscedastic error variance is implemented on seven popular ridge estimators in the study. However, this can be utilized for any ridge estimator that is a function of error variance. The performance of the new estimators is explored using an extensive simulation study in terms of mean squared error (MSE). The results demonstrate that the proposed estimators consistently yield lower MSE values compared to their classical counterparts, particularly in the presence of both multicollinearity and heteroscedasticity. The heteroscedastic error variance adjustment plays a pivotal role in achieving these improvements. The use of proposed estimators has also been illustrated using real-life applications.

Reconsidering the estimation of costs of phenotypic plasticity using the robust ridge estimator

In Multiple regression analysis, it is assumed that the independent variables are uncorrelated with one another, when such happen, the problem of multicollinearity occurs. Multicollinearity can create inaccurate estimates of the regression coefficients, inflate the standard errors of the regression coefficients, deflate the partial t-tests for the regression coefficients, give false p-values and degrade the predictability of the model. There are several methods to get rid of this problem and one of the most famous one is the ridge regression. The purpose of this research is to study the performance of some popular ridge regression estimators based on the effects of sample sizes and correlation levels on their Average Mean Square Error (AMSE) for Poisson Regression models in the presence of multicollinearity. As performance criteria, average MSE of k was used. A Monte Carlo simulation study was conducted to compare performance of Fifty (50) k estimators under four experimental conditions namely: correlation, Number of explanatory variables, sample size and intercept. From the results of the analysis as summarized in the Tables, the MSE of the estimators performed better in a lower explanatory variables <i>p</i> and an increased intercept value. It was also observed that some estimators performed better on the average at all correlation levels, sample sizes, intercept values and explanatory variables than others.

Ridge regression is an alternative to ordinary least-squares (OLS) regression. It is believed to be superior to least-squares regression in the presence of multicollinearity. The robustness of this method is investigated and comparison is made with the least squares method through simulation studies. Our results show that the system stabilizes in a region of k, where k is a positive quantity less than one and whose values depend on the degree of correlation between the independent variables. The results also illustrate that k is a linear function of the correlation between the independent variables.