Iterative algorithms of biased estimation methods in binary logistic regression

The Maximum Likelihood (ML) and Best Linear Unbiased (BLU) estimators of the location and scale parameters of an extreme value distribution (Lawless [1982]) are compared under conditions of small sample sizes and Type I censorship. The comparisons were made in terms of the mean square error criterion. According to this criterion, the ML estimator of σ in the case of very small sample sizes (n < 10) and heavy censorship (low censoring time) proved to be more efficient than the corresponding BLU estimator. However, the BLU estimator for σ attains parity with the corresponding ML estimator when the censoring time increases even for sample sizes as low as 10. The BLU estimator of σ attains equivalence with the ML estimator when the sample size increases above 10, particularly when the censoring time is also increased. The situation is reversed when it came to estimating the location parameter μ, as the BLU estimator was found to be consistently more efficient than the ML estimator despite the improved performance of the ML estimator when the sample size increases. However, computational ease and convenience favor the ML estimators. Keywords: maximum likelihood estimatorsbest linear unbiased estimatorslife-testingreliability

The beta regression is a widely known statistical model when the response (or the dependent) variable has the form of fractions or percentages. In most of the situations in beta regression, the explanatory variables are related to each other which is commonly known as the multicollinearity problem. It is well-known that the multicollinearity problem affects severely the variance of maximum likelihood (ML) estimates. In this article, we developed a new biased estimator (called a two-parameter estimator) for the beta regression model to handle this problem and decrease the variance of the estimation. The properties of the proposed estimator are derived. Furthermore, the performance of the proposed estimator is compared with the ML estimator and other common biased (ridge, Liu, and Liu-type) estimators depending on the mean squared error criterion by making a Monte Carlo simulation study and through two real data applications. The results of the simulation and applications indicated that the proposed estimator outperformed ML, ridge, Liu, and Liu-type estimators.

Many clinical or prevention studies involve missing or censored outcomes. Maximum likelihood (ML) methods provide a conceptually straightforward approach to estimation when the outcome is partially missing. Methods of implementing ML methods range from the simple to the complex, depending on the type of data and the missing-data mechanism. Simple ML methods for ignorable missing-data mechanisms (when data are missing at random) include complete-case analysis, complete-case analysis with covariate adjustment, survival analysis with covariate adjustment, and analysis via propensity-to-be-missing scores. More complex ML methods for ignorable missing-data mechanisms include the analysis of longitudinal dropouts via a marginal model for continuous data or a conditional model for categorical data. A moderately complex ML method for categorical data with a saturated model and either ignorable or nonignorable missing-data mechanisms is a perfect fit analysis, an algebraic method involving closed-form estimates and variances. A complex and flexible ML method with categorical data and either ignorable or nonignorable missing-data mechanisms is the method of composite linear models, a matrix method requiring specialized software. Except for the method of composite linear models, which can involve challenging matrix specifications, the implementation of these ML methods ranges in difficulty from easy to moderate.

Examining the robustness properties of maximum likelihood (ML) estimators of parameters in exponential power and generalized t distributions has been considered together. The well-known asymptotic properties of ML estimators of location, scale and added skewness parameters in these distributions are studied. The ML estimators for location, scale and scale variant (skewness) parameters are represented as an iterative reweighting algorithm (IRA) to compute the estimates of these parameters simultaneously. The artificial data are generated to examine performance of IRA for ML estimators of parameters simultaneously. We make a comparison between these two distributions to test the fitting performance on real data sets. The goodness of fit test and information criteria approve that robustness and fitting performance should be considered together as a key for modeling issue to have the best information from real data sets.

Association mapping uses statistical analyses to test for relationships between genomic markers called single nucleotide polymorphisms (SNPs) and traits. This research focuses on the use of logistic regression to assess the additive, dominance, and epistatic effects when investigating associations between SNPs and binary traits, such as disease status. A very specific phenomenon that results in infinite maximum likelihood estimates (MLEs) of logistic regression parameters, called quasi-separation of points (QSP), is investigated. We provide a solution that relies on the use of Firth’s MLE to estimate logistic regression parameters. Simulated and real data are utilized to investigate the use of Firth’s MLE in a QSP setting.

The reverberation power spectral density (PSD) is often required for dereverberation and noise reduction algorithms. In this work, we compare two maximum likelihood (ML) estimators of the reverberation PSD in a noisy environment. In the first estimator, the direct path is first blocked. Then, the ML criterion for estimating the reverberation PSD is stated according to the probability density function of the blocking matrix (BM) outputs. In the second estimator, the speech component is not blocked. Instead, the ML criterion for estimating the speech and reverberation PSD is stated according to the probability density function of the microphone signals. To compare the expected mean square error (MSE) between the two ML estimators of the reverberation PSD, the Cramer–Rao Bounds (CRBs) for the two ML estimators are derived. We show that the CRB for the joint reverberation and speech PSD estimator is lower than the CRB for estimating the reverberation PSD from the BM outputs. Experimental results show that the MSE of the two estimators indeed obeys the CRB curves. Experimental results of multimicrophone dereverberation and noise reduction algorithm show the benefits of using the ML estimators in comparison with another baseline estimators.

Weibull distribution is widely used in various areas such as life tables, failure rates, and definition of wind speed distribution. Therefore, parameter estimation for the Weibull distribution is an important problem in many real data applications. The least square (LS), the weighted least square (WLS) and the maximum likelihood (ML) are the most popular methods for the parameter estimation in the Weibull distribution. In this study, based on the LS, WLS and ML estimation methods, a multi-objective programming approach is proposed for the parameter estimation of two-parameter Weibull distribution. This new approach evaluates together LS, WLS and ML methods in the estimation process. NSGA-II method, which is a multi-objective heuristic optimization method, is used to solve the proposed multi-objective estimation model. To evaluate the applicability and performance of the proposed approach, a detailed Monte Carlo simulation study based on deficiency criteria and a real data application are designed. The results illustrated that the proposed multi-objective programming approach provides quite accurate parameter estimates for the two parameter Weibull distribution with respect to deficiency criteria.

Researchers using latent class (LC) analysis often proceed using the following three steps: (1) an LC model is built for a set of response variables, (2) subjects are assigned to LCs based on their posterior class membership probabilities, and (3) the association between the assigned class membership and external variables is investigated using simple cross-tabulations or multinomial logistic regression analysis. Bolck, Croon, and Hagenaars (2004) demonstrated that such a three-step approach underestimates the associations between covariates and class membership. They proposed resolving this problem by means of a specific correction method that involves modifying the third step. In this article, I extend the correction method of Bolck, Croon, and Hagenaars by showing that it involves maximizing a weighted log-likelihood function for clustered data. This conceptualization makes it possible to apply the method not only with categorical but also with continuous explanatory variables, to obtain correct tests using complex sampling variance estimation methods, and to implement it in standard software for logistic regression analysis. In addition, a new maximum likelihood (ML)—based correction method is proposed, which is more direct in the sense that it does not require analyzing weighted data. This new three-step ML method can be easily implemented in software for LC analysis. The reported simulation study shows that both correction methods perform very well in the sense that their parameter estimates and their SEs can be trusted, except for situations with very poorly separated classes. The main advantage of the ML method compared with the Bolck, Croon, and Hagenaars approach is that it is much more efficient and almost as efficient as one-step ML estimation.

Beta regression model is used for modeling proportions measured on a continuous scale; its parameters are estimated with the maximum likelihood method. Classical regression models, such as linear regression model and nonlinear regression models like logistic regression are not suitable for such situations. As in linear regression model, the independent variables are assumed to be uncorrelated if this assumption is not met, then the multicollinearity appears. Multicollinearity problem means that there is a near dependency between the independent variables. Biased estimators are commonly used for correcting the multicollinearity problem. In this study, we propose a generalized biased estimator for correcting multicollinearity in beta regression that is generalize beta ridge regression estimator (GBRRE). The performance of the proposed generalized biased estimator is evaluated theoretically via the matrix mean squared errors and the scalar mean squared errors; and practically using a Monte Carlo simulation study. The simulation results show that the optimal shrinkage estimator is K1 and the worst one is K2. Also, the proposed generalized estimator is applied to a real data set of pre-university education students in Egypt during the academic year (2018/2019) and we found the application results agree with the simulation results. Finally based on the results of the simulation study and the application the performance of the suggested generalized biased estimator is better than maximum likelihood estimators.

A Poisson ridge regression estimator

. We propose a two-stage generalized randomized response (GRR) design, incorporating the first-stage design of Huang’s two-stage RR design as the first-stage design and Christofides’ GRR design as the second-stage design. Our design allows estimation of the proportion of the population with a sensitive attribute and the population proportion of sensitive attribute respondents providing a truthful response to a directly asked sensitive question in the first stage. We derive the large-sample properties of the maximum likelihood (ML) estimators for logistic regression parameters and assess the finite-sample performance of the ML estimators via Monte Carlo experiments. The practical application of our methodology is illustrated by using the 2016 survey data on the sexuality of local freshmen of Feng Chia University in Taiwan.

The Multiple Comparison Procedures with Modeling Techniques (MCP-Mod) framework has been recently approved by the U.S. Food, Administration, and European Medicines Agency as fit-for-purpose for phase II studies. Nonetheless, this approach relies on the asymptotic properties of Maximum Likelihood (ML) estimators, which might not be reasonable for small sample sizes. In this paper, we derived improved ML estimators and correction for their covariance matrices in the censored Weibull regression model based on the corrective and preventive approaches. We performed two simulation studies to evaluate ML and improved ML estimators with their covariance matrices in (i) a regression framework (ii) the Multiple Comparison Procedures with Modeling Techniques framework. We have shown that improved ML estimators are less biased than ML estimators yielding Wald-type statistics that controls type I error without loss of power in both frameworks. Therefore, we recommend the use of improved ML estimators in the MCP-Mod approach to control type I error at nominal value for sample sizes ranging from 5 to 25 subjects per dose.

The beta regression model is commonly used when analyzing data that come in the form of rates or percentages. However, a problem that may encounter when analyzing these kinds of data that has not been investigated for this model is the multicollinearity problem. It is well known that the maximum likelihood (ML) method is very sensitive to high inter-correlation among the explanatory variables. Therefore, this paper proposes some ridge estimators for the beta regression model to remedy the problem of instability of the traditional ML method and increase the efficiency of estimation. The performance of ridge estimators is compared to the ML estimator through the mean squared error (MSE) and the mean absolute error (MAE) criteria by conducting a Monte-Carlo simulation study and through an empirical application. According to the simulation and application results, the proposed estimators outperform the ML estimator in terms of MSE and MAE.

We simulated single-generation data for a fitness trait in mutation-accumulation (MA) experiments, and we compared three methods of analysis. Bateman-Mukai (BM) and maximum likelihood (ML) need information on both the MA lines and control lines, while minimum distance (MD) can be applied with or without the control. Both MD and ML assume gamma-distributed mutational effects. ML estimates of the rate of deleterious mutation had larger mean square error (MSE) than MD or BM had due to large outliers. MD estimates obtained by ignoring the mean decline observed from comparison to a control are often better than those obtained using that information. When effects are simulated using the gamma distribution, reducing the precision with which the trait is assayed increases the probability of obtaining no ML or MD estimates but causes no appreciable increase of the MSE. When the residual errors for the means of the simulated lines are sampled from the empirical distribution in a MA experiment, instead of from a normal one, the MSEs of BM, ML, and MD are practically unaffected. When the simulated gamma distribution accounts for a high rate of mild deleterious mutation, BM detects only approximately 30% of the true deleterious mutation rate, while MD or ML detects substantially larger fractions. To test the robustness of the methods, we also added a high rate of common contaminant mutations with constant mild deleterious effect to a low rate of mutations with gamma-distributed deleterious effects and moderate average. In that case, BM detects roughly the same fraction as before, regardless of the precision of the assay, while ML fails to provide estimates. However, MD estimates are obtained by ignoring the control information, detecting approximately 70% of the total mutation rate when the mean of the lines is assayed with good precision, but only 15% for low-precision assays. Contaminant mutations with only tiny deleterious effects could not be detected with acceptable accuracy by any of the above methods.

Two-parameter Weibull distribution is the most widely adopted lifetime model for power transformers. An appropriate parameter estimation method is essential to guarantee the accuracy of a derived Weibull lifetime model. Six popular parameter estimation methods (i.e. the maximum likelihood estimation method, two median rank regression methods including the one regressing X on Y and the other one regressing Y on X, the Kaplan-Meier method, the method based on cumulative hazard plot, and the Li's method) are reviewed and compared in order to find the optimal one that suits transformer's Weibull lifetime modelling. The comparison took several different scenarios into consideration: 10000 sets of lifetime data, each of which had a sampling size of 40 1000 and a censoring rate of 90\%, were obtained by Monte-Carlo simulations for each scienario. Scale and shape parameters of Weibull distribution estimated by the six methods, as well as their mean value, median value and 90\% confidence band are obtained. The cross comparison of these results reveals that, among the six methods, the maximum likelihood method is the best one, since it could provide the most accurate Weibull parameters, i.e. parameters having the smallest bias in both mean and median values, as well as the shortest length of the 90\% confidence band. The maximum likelihood method is therefore recommended to be used over the other methods in transformer Weibull lifetime modelling.