Coverage-based performance of confidence intervals for linear regression coefficients under multicollinearity: Simulation and application

Finding a new drug which is both safe and efficient is an expensive and time-consuming endeavour. In particular, establishing the ‘dose-effect relationship’ – how beneficial a drug is at different dosages – can be challenging. Predicting this curve requires gathering experimental data by exposing and recording how cells respond to various levels of the drug. However, extreme values are often observed at low and high dosages, potentially introducing errors that are hard to correct in the prediction process. Yet, these extreme observations are sometimes genuine so researchers cannot just ignore them. To improve dose-effect estimation, Zhou, Liu, Fang et al. developed a new general-purpose approach. It uses advanced statistical modelling to account for extremes in lab data. This strategy outperformed other methods when dealing with these observations while also providing higher efficiency in data analysis with more uniform data in experiments. To facilitate implementation, Zhou, Liu, Fang et al. set up a user-friendly tool baptized ‘REAP’; this free online resource allows scientists without advanced statistical experience to harness the new approach and to perform dose-effect analysis more easily and accurately. This could boost research across many different disciplines that examine the effects of chemicals on cells.

Based on the fiscal revenue and other relevant economic index data of Beijing from 1995 to 2020, this study uses the research methods of Ridge regression and Lasso regression to explore the influencing factors of Beijing’s fiscal revenue. Considering that the traditional linear regression model will produce strong multicollinearity among many variables. Therefore, ridge regression and Lasso regression model were firstly used to reduce the influence of multicollinearity between variables, and then variable selection was carried out. Finally, the two models were compared according to the analysis results, and the optimal analysis model was selected. The results show that compared with ridge regression model, lasso regression model has better goodness of fit, smaller error and better model. The added value of the second industry, power generation, resident population, urban per capita disposable income and total retail sales of social consumer goods has a positive impact on fiscal revenue, and the whole social fixed assets investment, employment in cities and towns, per capita consumption expenditure of urban households is has a certain negative impact on fiscal income level.

Introduction: Confidence intervals (CIs) provide a more precise evaluation of outcomes, especially when the risk of an event is influenced by cluster size. While confidence intervals are commonly used to assess uncertainty in future data, in this study, we focus on their role in quantifying variability within currently observed outcomes. Specifically, the width of the predicted confidence interval serves as an indicator of existing intra-cluster heterogeneity, highlighting the extent of variability across different cluster sizes. This study introduces a novel method for evaluating observed outcomes of dichotomous random variables in datasets with unequal binary cluster sizes. By employing a robust neighborhood confidence interval width, this approach ensures a more reliable and adaptive estimation of intra-cluster variability, allowing for a more accurate interpretation of current data distributions. Methods: We introduce a novel algorithm for constructing an intra-cluster robust neighborhood confidence interval and its corresponding width for each cluster. This method enables the ranking of clusters based on confidence interval width, from the narrowest to the widest, providing a systematic approach to quantifying intra-cluster variability. By evaluating observed values within these ranked intervals, the algorithm offers a more precise assessment of data heterogeneity. To illustrate the effectiveness of this method, we present both a simulated example assessing its finite-sample performance and a real-world application in the context of antimicrobial resistance data with unequal binary cluster sizes. Results: The robust neighborhood intra-cluster confidence interval (CI) width was successfully derived for interpreting binary outcome data with unequal cluster sizes. The analysis showed that narrow confidence intervals indicate minimal random variation, suggesting higher reliability in the observed results, whereas wider intervals highlight increased intracluster variability. Conclusion: The intra-cluster robust neighborhood CI and its corresponding width provide a valuable tool for analyzing binary outcome data with unequal cluster sizes. This method enhances the interpretation of observed results by systematically quantifying variability within clusters, allowing for more reliable intra-cluster comparisons.

By constructing a correlation network between global sea surface temperature anomalies (SSTAs) and summer precipitation anomalies in the Yunnan–Guizhou Plateau, key SST regions influencing summer precipitation anomalies in the Yunnan–Guizhou Plateau were selected. It was found that spring SSTAs in the Bay of Bengal, southwestern Atlantic, and eastern Pacific are crucial for influencing summer precipitation anomalies in the Yunnan–Guizhou Plateau. Setting SSTAs from these three regions as predictor variables 3 months in advance, we constructed multiple linear regression (MLR), ridge regression (RR), and lasso regression (LR) models to predict summer precipitation anomalies over the Yunnan–Guizhou region. The training phase involved data spanning from 1961 to 2005, which aimed to predict precipitation anomalies in the Yunnan–Guizhou Plateau for the period extending from 2006 to 2022. Based on MLR, RR, and LR models, the correlations between predicted values and observed summer precipitation anomalies in Yunnan–Guizhou were 0.48, 0.46, and 0.46, respectively. These values were all higher than the correlation coefficients of the NCC_CSM model’s predicted and observed values. Additionally, its performance in predicting summer precipitation anomalies over the Yunnan–Guizhou region, based on key SST regions, was assessed using performance metrics such as anomaly correlation coefficient (ACC), anomaly sign consistency rate (PC), and trend anomaly comprehensive score (PS score). The average ACC of MLR, RR, and LR models was higher than that of the NCC_CSM model’s predictions. For MLR, RR, LR, and NCC_CSM models, the PCs exceeding 50% of the year were 14, 14, 11, and 10, respectively. Furthermore, the average PS score for predicting summer precipitation anomalies over the Yunnan–Guizhou region using MLR, RR, and LR was approximately 73 points; 8 higher than the average PS score of the NCC_CSM model. Therefore, predicting summer precipitation anomalies over the Yunnan–Guizhou region based on key SST regions is of great significance for improving the prediction skills of precipitation anomalies in this region.

Methods of estimating the pooled effect size under meta-analysis: A comparative appraisal

A recent developed trapping web sampling design (Anderson et al. 1983) and a series of four density estimation models were evaluated using known field populations of ground—dwelling darkling beetles (Tenebrionidae: Eleodes spp.) in a shrub—steppe ecosystem in southwestern Wyoming. Known densities (range 794—11 793 beetles/ha) of marked beetles were established annually in two circular enclosures (415.5 m2 each) from 1984 to 1987. After releasing the beetles, we sampled the marked populations in each enclosure, using 141 pitfall traps arranged in a web design. The resulting data were analyzed with four density estimation models (Fourier series, exponential power series, negative exponential, and half normal) using the program TRANSECT. We then evaluated each model's ability to estimate the true beetle density (D). We found that the negative exponential model was most reliable, estimating D "correctly" (i.e., the 95% confidence interval of D included D) in 100% of the trials (n = 11), and having a mean coefficient of variation (CV) of 20% and no detectable bias. The exponential power series and Fourier series also performed well, providing confidence interval coverage of D in 91 and 73% of the trials, respectively. However, the precision of the exponential power series model was extremely low, resulting in very wide confidence intervals. The half normal model consistently underestimated the true densities, and provided only 18% confidence interval coverage. Tests of the sampling design's assumptions (regarding animal capture probabilities and movements) using actual field data showed that, in most cases, all assumptions were satisfied; even when assumptions were violated, the there successful models continued to function well. An improved procedure for setting confidence intervals is also provided. Based on these analytical results, and considering the logistical and theoretical advantages of trapping webs over trapping grids, we recommend the use of the web design and distance measure analyses for estimating population densities of mobile small animals.

Abstract. Accurate quantification of long-term trends in stratospheric ozone can be challenging due to their sensitivity to natural variability, the quality of the observational datasets, and non-linear changes in forcing processes as well as the statistical methodologies. Multivariate linear regression (MLR) is the most commonly used tool for ozone trend analysis; however, the complex coupling in many atmospheric processes can make it prone to the issue of over-fitting when using the conventional ordinary-least-squares (OLS) approach. To overcome this issue, here we adopt a regularized (ridge) regression method to estimate ozone trends and quantify the influence of individual processes. We use the Stratospheric Water and OzOne Satellite Homogenized (SWOOSH) merged dataset (v2.7) to derive stratospheric ozone profile trends for the period 1984–2020. Besides SWOOSH, we also analyse a machine-learning-based satellite-corrected gap-free global stratospheric ozone profile dataset from a chemical transport model (ML-TOMCAT) and output from a chemical transport model (TOMCAT) simulation forced with European Centre for Medium-Range Weather Forecasts (ECMWF) ERA5 reanalysis. For 1984–1997, we observe smaller negative trends in the SWOOSH stratospheric ozone profile using ridge regression compared to OLS. Except for the tropical lower stratosphere, the largest differences arise in the mid-latitude lowermost stratosphere (>4 % per decade difference at 100 hPa). From 1998 and the onset of ozone recovery in the upper stratosphere, the positive trends estimated using the ridge regression model (∼1 % per decade near 2 hPa) are smaller than those using OLS (∼2 % per decade). In the lower stratosphere, post-1998 negative trends with large uncertainties are observed and ridge-based trend estimates are somewhat smaller and less variable in magnitude compared to the OLS regression. Aside from the tropical lower stratosphere, the largest difference is around 2 % per decade at 100 hPa (with ∼3 % per decade uncertainties for individual trends) in northern mid-latitudes. For both time periods the SWOOSH data produce large negative trends in the tropical lower stratosphere with a correspondingly large difference between the two trend methods. In both cases the ridge method produces a smaller trend. The regression coefficients from both OLS and ridge models, which represent ozone variations associated with natural processes (e.g. the quasi-biennial oscillation, solar variability, El Niño–Southern Oscillation, Arctic Oscillation, Antarctic Oscillation, and Eliassen–Palm flux), highlight the dominance of dynamical processes in controlling lower-stratospheric ozone concentrations. Ridge regression generally yields smaller regression coefficients due to correlated explanatory variables, and care must be exercised when comparing fit coefficients and their statistical significance across different regression methods. Comparing the ML-TOMCAT-based trend estimates with the ERA5-forced model simulation, we find ML-TOMCAT shows significant improvements with much better consistency with the SWOOSH dataset, despite the ML-TOMCAT training period overlapping with SWOOSH only for the Microwave Limb Sounder (MLS) measurement period. The largest inconsistencies with respect to SWOOSH-based trends post-1998 appear in the lower stratosphere where the ERA5-forced model simulation shows positive trends for both the tropics and the mid-latitudes. The large differences between satellite-based data and the ERA5-forced model simulation confirm significant uncertainties in ozone trend estimates, especially in the lower stratosphere, underscoring the need for caution when interpreting results obtained with different regression methods and datasets.

Does simulation really increase gynecologic surgical skill?

SummaryThis paper considers a linear regression model with regression parameter vector β. The parameter of interest is θ= aTβ where a is specified. When, as a first step, a data‐based variable selection (e.g. minimum Akaike information criterion) is used to select a model, it is common statistical practice to then carry out inference about θ, using the same data, based on the (false) assumption that the selected model had been provided a priori. The paper considers a confidence interval for θ with nominal coverage 1 ‐ α constructed on this (false) assumption, and calls this the naive 1 ‐ α confidence interval. The minimum coverage probability of this confidence interval can be calculated for simple variable selection procedures involving only a single variable. However, the kinds of variable selection procedures used in practice are typically much more complicated. For the real‐life data presented in this paper, there are 20 variables each of which is to be either included or not, leading to 220 different models. The coverage probability at any given value of the parameters provides an upper bound on the minimum coverage probability of the naive confidence interval. This paper derives a new Monte Carlo simulation estimator of the coverage probability, which uses conditioning for variance reduction. For these real‐life data, the gain in efficiency of this Monte Carlo simulation due to conditioning ranged from 2 to 6. The paper also presents a simple one‐dimensional search strategy for parameter values at which the coverage probability is relatively small. For these real‐life data, this search leads to parameter values for which the coverage probability of the naive 0.95 confidence interval is 0.79 for variable selection using the Akaike information criterion and 0.70 for variable selection using Bayes information criterion, showing that these confidence intervals are completely inadequate.

BackgroundClinical registries, which capture information about the health and healthcare use of patients with a health condition or treatment, often contain patient-reported outcomes (PROs) that provide insights about the patient’s perspectives on their health. Missing data can affect the value of PRO data for healthcare decision-making. We compared the precision and bias of several missing data methods when estimating longitudinal change in PRO scores.MethodsThis research conducted analyses of clinical registry data and simulated data. Registry data were from a population-based regional joint replacement registry for Manitoba, Canada; the study cohort consisted of 5631 patients having total knee arthroplasty between 2009 and 2015. PROs were measured using the 12-item Short Form Survey, version 2 (SF-12v2) at pre- and post-operative occasions. The simulation cohort was a subset of 3000 patients from the study cohort with complete PRO information at both pre- and post-operative occasions. Linear mixed-effects models based on complete case analysis (CCA), maximum likelihood (ML) and multiple imputation (MI) without and with an auxiliary variable (MI-Aux) were used to estimate longitudinal change in PRO scores. In the simulated data, bias, root mean squared error (RMSE), and 95% confidence interval (CI) coverage and width were estimated under varying amounts and types of missing data.ResultsThree thousand two hundred thirty (57.4%) patients in the study cohort had complete data on the SF-12v2 at both occasions. In this cohort, mixed-effects models based on CCA resulted in substantially wider 95% CIs than models based on ML and MI methods. The latter two methods produced similar estimates and 95% CI widths. In the simulation cohort, when 50% of the data were missing, the MI-Aux method, in which a single hypothetical auxiliary variable was strongly correlated (i.e., 0.8) with the outcome, reduced the 95% CI width by up to 14% and bias and RMSE by up to 50 and 45%, respectively, when compared with the MI method.ConclusionsMissing data can substantially affect the precision of estimated change in PRO scores from clinical registry data. Inclusion of auxiliary information in MI models can increase precision and reduce bias, but identifying the optimal auxiliary variable(s) may be challenging.

Accurate energy consumption prediction is a critical resource allocation, reducing operational costs, and supporting sustainable energy management. This study presents a machine learning-based energy consumption prediction system, comparing the performance of Linear Regression, Ridge Regression, Lasso Regression, and Decision tree models. Using a historical dataset of consumer energy usage, key features were extracted and preprocessed to enhance predictive accuracy. Model evaluation employed Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and Coefficient of Determination metrics. Results indicate that while Linear Regression offered competitive baseline performance, Ridge and Lasso Regressions achieved improved stability against multicollinearity, and the Decision Tree Model delivered the highest predictive accuracy, albeit with potential overfitting risks. The findings contribute to developing scalable, data-driven forecasting frameworks applicable to smart grids, policy planning, and demand-side management.

Skart is an automated sequential batch-means procedure for constructing a skewness- and autoregression-adjusted confidence interval (CI) for the steady-state mean of a simulation output process either in discrete time (i.e., using observation-based statistics), or in continuous time (i.e., using time-persistent statistics). Skart delivers a CI designed to satisfy user-specified requirements concerning both the CI's coverage probability and its absolute or relative precision. Skart exploits separate adjustments to the classical batch-means CI to account for the effects on the distribution of the underlying Student's t-statistic arising from skewness and autocorrelation of the batch means. The skewness adjustment is based on a Cornish–Fisher expansion for the classical batch-means t-statistic, and the autocorrelation adjustment is based on a first-order autoregressive approximation to the batch-means autocorrelation function. Skart also delivers a point estimator for the steady-state mean that is approximately free of initialization bias. The associated warm-up period is based on iteratively applying Von Neumann's randomness test to spaced batch means with increasing sizes for each batch and its preceding spacer. In extensive experimentation, Skart compared favorably with its competitors. [Supplementary material is available for this article. Go to the publisher's online edition of IIE Transactions for additional discussion, detailed proofs, etc.]

Disease risk estimation plays an important role in disease prevention. Many studies have found that the ability to predict risk improves as the number of risk single-nucleotide polymorphisms (SNPs) in the risk model increases. However, the width of the confidence interval of the risk estimate is often not considered in the evaluation of the risk model. Here, we explore how the risk and the confidence interval width change as more SNPs are added to the model in the order of decreasing effect size, using both simulated data and real data from studies of abdominal aortic aneurysms and age-related macular degeneration. Our results show that confidence interval width is positively correlated with model size and the majority of the bigger models have wider confidence interval widths than smaller models. Once the model size is bigger than a certain level, the risk does not shift markedly, as 100% of the risk estimates of the one-SNP-bigger models lie inside the confidence interval of the one-SNP-smaller models. We also created a confidence interval-augmented reclassification table. It shows that both more effective SNPs with larger odds ratios and less effective SNPs with smaller odds ratios contribute to the correct decision of whom to screen. The best screening strategy is selected and evaluated by the net benefit quantity and the reclassification rate. We suggest that individuals whose upper bound of their risk confidence interval is above the screening threshold, which corresponds to the population prevalence of the disease, should be screened.

We propose new approaches for choosing the shrinkage parameter in ridge regression, a penalized likelihood method for regularizing linear regression coefficients, when the number of observations is small relative to the number of parameters. Existing methods may lead to extreme choices of this parameter, which will either not shrink the coefficients enough or shrink them by too much. Within this "small-n, large-p" context, we suggest a correction to the common generalized cross-validation (GCV) method that preserves the asymptotic optimality of the original GCV. We also introduce the notion of a "hyperpenalty", which shrinks the shrinkage parameter itself, and make a specific recommendation regarding the choice of hyperpenalty that empirically works well in a broad range of scenarios. A simple algorithm jointly estimates the shrinkage parameter and regression coefficients in the hyperpenalized likelihood. In a comprehensive simulation study of small-sample scenarios, our proposed approaches offer superior prediction over nine other existing methods.

Sequest is a fully sequential procedure that delivers improved point and confidence-interval (CI) estimators for a designated steady-state quantile by exploiting a combination of ideas from batching and sectioning. Sequest incorporates effective methods to do the following: (a) eliminate bias in the sectioning-based point estimator that is caused by initialization of the simulation or an inadequate simulation run length (sample size); and (b) adjust the CI half-length for the effects of skewness or correlation in the batching-based point estimators of the designated quantile. Sequest delivers a CI designed to satisfy user-specified requirements concerning both the CI's coverage probability and its absolute or relative precision. We found that Sequest exhibited good small- and large-sample properties in a preliminary evaluation of the procedure's performance on a suite of problems that includes some problems designed to stress test the procedure.