A ridge estimation method for the Waring regression model: simulation and application

On the performance of some new ridge parameter estimators in the Poisson-inverse Gaussian ridge regression

The beta regression model (BRM) is used when the dependent variable may take continuous values and be bounded in the interval (0, 1), such as rates, proportions, percentages and fractions. Generally, the parameters of the BRM are estimated by the method of maximum likelihood estimation (MLE). However, the MLE does not offer accurate and reliable estimates when the explanatory variables in the BRM are correlated. To solve this problem, the ridge and Liu estimators for the BRM were proposed by different authors. In the current study, the James Stein Estimator (JSE) for the BRM is proposed. The matrix mean squared error (MSE) and the scalar MSE properties are derived and then compared to the available ridge estimator, Liu estimator and MLE. The performance of the proposed estimator is evaluated by conducting a simulation experiment and analyzing two real-life applications. The MSE of the estimators is considered as a performance evaluation criterion. The findings of the simulation experiment and applications indicate the superiority of the suggested estimator over the competitive estimators for estimating the parameters of the BRM.

Mixed Poisson regression models (MPRMs) are widely used for analyzing overdispersed count data. However, the presence of multicollinearity among explanatory variables poses challenges when estimating regression coefficients using the maximum likelihood estimator (MLE), leading to inflated variances. The Poisson Modification of the Quasi-Lindley regression model (PMQLRM), a recently introduced alternative within MPRMs, faces similar issues. To address this, we propose a Liu-type estimator for the PMQLRM as an effective remedy for multicollinearity. Several existing methods are utilized to estimate the Liu-type parameter, and the theoretical superiority conditions of the proposed estimator over the MLE, ridge regression estimator, and Liu estimator are established using the scalar mean squared error (MSE) criterion. A Monte Carlo simulation study is conducted to compare the performance of different estimators based on the MSE. Additionally, a real-world dataset is analyzed to demonstrate the practical advantages of the proposed method. The findings indicate that the Poisson-modification of the Quasi-Lindley Liu-type estimator outperforms the MLE and other biased estimators when multicollinearity is present, offering a more stable and reliable alternative for parameter estimation in mixed Poisson regression models.

The binary logistic regression model (LRM) is practical in situations when the response variable (RV) is dichotomous. The maximum likelihood estimator (MLE) is generally considered to estimate the LRM parameters. However, in the presence of multicollinearity (MC), the MLE is not the correct choice due to its inflated standard deviation (SD) and standard errors (SE) of the estimates. To combat MC, commonly used biased estimators, i.e., the Ridge estimators (RE) and Liu estimators (LEs), are preferred. However, most of the time, the traditional LE attains a negative value for its Liu parameter (LP), which is considered to be a major drawback. Therefore, to overcome this issue, we proposed a new adjusted LE for the binary LRM. Owing to numerical evaluation purposes, Monte Carlo simulation (MCS) study is performed under different conditions where bias and mean squared error are the performance criteria. Findings showed the superiority of our proposed estimator in comparison with the other estimation methods due to the existence of high but imperfect multicollinearity, which clearly means that it is consistent when the regressors are multicollinear. Furthermore, the findings demonstrated that whenever there is MC, the MLE is not the best choice. Finally, a real application is being considered to be evidence for the advantage of the intended estimator. The MCS and the application findings pointed out that the considered adjusted LE for the binary logistic regression model is a more efficient estimation method whenever the regressors are highly multicollinear.

Beta regression model is used for modeling proportions measured on a continuous scale; its parameters are estimated with the maximum likelihood method. Classical regression models, such as linear regression model and nonlinear regression models like logistic regression are not suitable for such situations. As in linear regression model, the independent variables are assumed to be uncorrelated if this assumption is not met, then the multicollinearity appears. Multicollinearity problem means that there is a near dependency between the independent variables. Biased estimators are commonly used for correcting the multicollinearity problem. In this study, we propose a generalized biased estimator for correcting multicollinearity in beta regression that is generalize beta ridge regression estimator (GBRRE). The performance of the proposed generalized biased estimator is evaluated theoretically via the matrix mean squared errors and the scalar mean squared errors; and practically using a Monte Carlo simulation study. The simulation results show that the optimal shrinkage estimator is K1 and the worst one is K2. Also, the proposed generalized estimator is applied to a real data set of pre-university education students in Egypt during the academic year (2018/2019) and we found the application results agree with the simulation results. Finally based on the results of the simulation study and the application the performance of the suggested generalized biased estimator is better than maximum likelihood estimators.

When product quality follows the Gamma distribution and is related to one or more covariate(s), then Gamma regression model (GRM) profiling will be used. The Gamma profiling is generally based on a maximum likelihood estimator (MLE). In GRM profiling, when two or more covariates are linearly related, the MLE-based GRM profiling is unsuitable. In this situation, an alternative to MLE-based profiling is required. So, this study develops the Gamma ridge regression profiling based on Pearson and deviance residuals. The performance of the proposed profiling is evaluated with the help of a simulation study under different conditions, where average run length is considered as the performance evaluation criterion. The simulation findings show that the Pearson residual based profiling with ridge estimator is better than the deviance residuals-based profiling with MLE as well as ridge estimator. Furthermore, we consider an application to evaluate the performance of the proposed methods.

In this article, a biased estimator is proposed to combat multi-collinearity in the logistic regression model. The proposed estimator is a general estimator which includes other biased estimators, such as the ridge estimator and the Liu estimator as special cases. Necessary and sufficient conditions for the superiority of the new biased estimator over the maximum likelihood estimator, the ridge estimator are obtained and some properties in the mean squared error sense are discussed. Furthermore, a Monte Carlo simulation study is given to illustrate some of the theoretical results.

The inverse Gaussian regression model (IGRM) is applied when the response variable is continuous, positively skewed and well fitted to the inverse Gaussian distribution. In the presence of multicollinearity, the maximum likelihood estimation (MLE) is not a right choice. Therefore, we proposed a new estimator called the principal component ridge estimator for the IGRM which combines the principal component estimator and the ridge estimator. We also consider a two-parameter estimator (TPE) and other biased estimators to see a clear image of our proposed estimator. A Monte Carlo simulation study is also presented to examine the performance of the proposed estimators. Furthermore, we analysed a dataset to assess the superiority of the proposed estimator. Based on the simulation and application results, it is evident that the proposed estimator dominates the classical MLE, and other considered biased estimation methods.

The Poisson regression model (PRM) is the standard statistical method of analyzing count data, and it is estimated by a Poisson maximum likelihood (PML) estimator. Such an estimator is affected by outliers, and some robust Poisson regression estimators have been proposed to solve this problem. PML estimators are also influenced by multicollinearity. Biased Poisson regression estimators have been developed to address this problem, including Poisson ridge regression and Poisson almost unbiased ridge estimators. However, the above mentioned estimators do not deal with outliers and multicollinearity problems together in a PRM. Therefore, we propose two robust ridge estimators to deal with the two problems simultaneously in the PRM, namely, the robust Poisson ridge regression (RPRR) estimator and the robust Poisson almost unbiased ridge (RPAUR) estimator. Theoretical comparisons and Monte‐Carlo simulations are conducted to investigate the performance of the proposed estimators relative to the performance of other approaches to PRM parameter estimation. The simulation results indicate that the RPAUR estimator outperforms the other estimators in all situations where both problems exist. Finally, real data are used to confirm the results of this paper.

This paper studies estimation of the parameters of the generalized Gompertz distribution based on ranked-set sample (RSS). Maximum likelihood (ML) and Bayesian approaches are considered. Approximate confidence intervals for the unknown parameters are constructed using both the normal approximation to the asymptotic distribution of the ML estimators and bootstrapping methods. Bayes estimates and credible intervals of the unknown parameters are obtained using differential evolution Markov chain Monte Carlo and Lindley’s methods. The proposed methods are compared via Monte Carlo simulations studies and an example employing real data. The performance of both ML and Bayes estimates is improved under RSS compared with simple random sample (SRS) regardless of the sample size. Bayes estimates outperform the ML estimates for small samples, while it is the other way around for moderate and large samples.

Robust fitting of mixture regression models

Condition-index based new ridge regression estimator for linear regression model with multicollinearity

Binary logistic semiparametric regression analysis is a commonly used statistical technique when the dependent variable is dichotomous or binary. In this analysis, the relationship between the success probability and certain explanatory variables is assumed to have a linear form, while the relationship to other variables is unknown. Multicollinearity is a serious problem that arises when explanatory variables in logistic semiparametric regression are highly correlated. It is well known that the variance of the maximum likelihood estimator is inflated due to multicollinearity in the semiparametric logistic regression model. Therefore, a novel stochastic restricted iterative weighted ridge estimator for logistic semiparametric regression is introduced, and its statistical properties are extracted asymptotically. Moreover, an extension of the generalized cross validation (GCV) function is introduced and applied for choosing the best values of the ridge parameter and the bandwidth of the kernel smoother. Additionally, some theorems are developed to illustrate the convergence of the GCV mean. Ultimately, the Monte-Carlo simulation studies and an actual real-life data set are conducted to support our theoretical discussion, and the findings indicated that the new estimator outperformed the other estimators under consideration.

The Maximum Likelihood (ML) and Best Linear Unbiased (BLU) estimators of the location and scale parameters of an extreme value distribution (Lawless [1982]) are compared under conditions of small sample sizes and Type I censorship. The comparisons were made in terms of the mean square error criterion. According to this criterion, the ML estimator of σ in the case of very small sample sizes (n < 10) and heavy censorship (low censoring time) proved to be more efficient than the corresponding BLU estimator. However, the BLU estimator for σ attains parity with the corresponding ML estimator when the censoring time increases even for sample sizes as low as 10. The BLU estimator of σ attains equivalence with the ML estimator when the sample size increases above 10, particularly when the censoring time is also increased. The situation is reversed when it came to estimating the location parameter μ, as the BLU estimator was found to be consistently more efficient than the ML estimator despite the improved performance of the ML estimator when the sample size increases. However, computational ease and convenience favor the ML estimators. Keywords: maximum likelihood estimatorsbest linear unbiased estimatorslife-testingreliability

Logistic regression is a widely used method to model categorical response data, and maximum likelihood (ML) estimation has widespread use in logistic regression. Although ML method is the most used method to estimate the regression coefficients in logistic regression model, multicollinearity seriously affects the ML estimator. To remedy the undesirable effects of multicollinearity, estimators alternative to ML are proposed. Drawing on the similarities between the multiple linear and logistic regressions, ridge, Liu and two parameter estimators are proposed which are based on the ML estimator. On the other hand, first-order approximated ridge estimator is proposed for use in logistic regression. This study will present further solutions to the problem in the form of alternative estimators which reduce the effect of collinearity. Owing to this, first-order approximated Liu, iterative Liu and iterative two parameter estimators are proposed. A simulation study as well as real life application are carried out to ascertain the effect of sample size and degree of multicollinearity, in which the ML based, first-order approximated and iterative biased estimators are compared. Graphical representations are presented which support the effect of the shrinkage parameter on the mean square error and prediction mean square error of the biased estimators.