Model averaging procedure for partially linear single-index models

A central point of disagreement, in certain long-standing discussions about a particular whaling dataset in the Scientific Committee of the International Whaling Commission, has directly involved model selection issues for linear mixed effect models. The biological question under discussion is associated with a clearly defined parameter of primary interest, a focus parameter, which makes model selection with the Focused Information Criterion (FIC) more appropriate than other selection methods. Since the existing FIC methodology has not covered the case of linear mixed effects models, this article sets up the required framework and develops the necessary formulae for the relevant FIC. Our new criterion requires the asymptotic distribution of estimators derived for a given candidate linear mixed model but with behaviour examined under a wider linear mixed model. These results, needed here to build our FIC, also have independent interest.

Bayesian Model Averaging and Weighted Average Least Squares: Equivariance, Stability, and Numerical Issues

In this paper, we propose the model averaging estimation method for multiplicative error model and construct the corresponding weight choosing criterion based on the Kullback–Leibler divergence with a hyperparameter to avoid the problem of overfitting. The resulting model average estimator is proved to be asymptotically optimal. It is shown that the Kullback–Leibler model averaging (KLMA) estimator asymptotically minimizes the in‐sample Kullback–Leibler divergence and improves the forecast accuracy of out‐of‐sample even under different loss functions. In simulations, we show that the KLMA estimator compares favorably with smooth‐AIC estimator (SAIC), smooth‐BIC estimator (SBIC), and Mallows model averaging estimator (MMA), especially when some nonlinear noise is added to the data generation process. The empirical applications in the daily range of S&P500 and price duration of IBM show that the out‐of‐sample forecasting capacity of the KLMA estimator is better than that of other methods.

This study investigates the performance of various model selection criteria for binary logistic regression models in diverse data settings. The research compares traditional criteria (AIC, AICc, BIC, FIC) and proposes criteria (pAIC, pAICc, pBIC) designed to improve predictive ability and prevent overfitting. A simulation study systematically manipulates factors like imbalanced ratios, collinearity, number of observations, and number of variables to evaluate the effectiveness of these criteria across various scenarios. The performance is assessed using four key metrics: F1-score, false positive rate, false negative rate, and Area Under the Curve (AUC). The findings reveal a complex interplay between data characteristics and criterion performance. While no single criterion dominated, data characteristics significantly impacted performance. Traditional criteria excelled in F1-score and lowest false negative rate with balanced data and abundant observations, but struggled with limited data. BIC performed well in balanced, uncorrelated settings, but suffered with data complexity. The proposed criteria effectively controlled false positives and maintained AUC across various scenarios, making them suitable for imbalanced or limited data. The Focused Information Criterion (FIC) appears to be less competitive compared to other criteria for binary logistic regression model selection across diverse data scenarios.

Optimization of chemometric approaches for the extraction of isorhamnetin-3-O-rutinoside from Calendula officinalis L.

The focused information criterion for model selection is constructed to select the model that best estimates a particular quantity of interest, the focus, in terms of mean squared error. We extend this focused selection process to the high‐dimensional regression setting with potentially a larger number of parameters than the size of the sample. We distinguish two cases: (i) the case where the considered submodel is of low dimension and (ii) the case where it is of high dimension. In the former case, we obtain an alternative expression of the low‐dimensional focused information criterion that can directly be applied. In the latter case, we use a desparsified estimator that allows us to derive the mean squared error of the focus estimator. We illustrate the performance of the high‐dimensional focused information criterion with a numerical study and a real dataset.

Focused Information Criterion and Model Averaging for Large Panels with a Multifactor Error Structure

SummaryOptimum variance component estimation methods that yield uniformly minimum variance quadratic unbiased estimators for a full dataset are often difficult or impossible to implement. In this paper we propose an estimator which is near optimal under some distributional assumptions that can be made without specifying an exact functional form. This estimator has an exact closed form expression. An average optimum estimator, which can be used when optimum estimators exist for subsets of the data, is also proposed. Performance comparisons of the proposed estimators are made individually with other popular estimators using simulated data. A performance comparison of the average optimum estimator is made under four constraints on the variance components. A real dataset is analysed using the proposed estimators. The robustness properties of the proposed estimators, in comparison with the other estimators, including the method of moments, are also investigated, using data simulated from a skew normal distribution. The average optimum estimator is strongly robust and far superior for estimating one of the variance components, as demonstrated by making comparisons with other methods. These comparisons are based on bias, mean squared error, and mean absolute deviation. The average estimator is moderately robust in respect of the estimation of the other variance components.

Model uncertainty is a prominent feature in many applied settings. This is certainty true in the efficiency analysis realm where concerns over the proper distributional specification of the error components of a stochastic frontier model is, generally, still open along with which variables influence inefficiency. Given the concern over the impact that model uncertainty is likely to have on the stochastic frontier model in practice, the present research proposes two distinct model averaging estimators, one which averages over nested classes of inefficiency distributions and another that has the ability to average over distinct distributions of inefficiency. Both of these estimators are shown to produce optimal weights when the aim is to uncover conditional inefficiency at the firm level. We study the finite-sample performance of the model average estimator via Monte Carlo experiments and compare with traditional model averaging estimators based on weights constructed from model selection criteria and present a short empirical application.

Variable selection in regression models has been well studied in the literature, with many non‐Bayesian and Bayesian methods available in this regard. An important class of regression models is generalized linear models, which involve situations where the response variable is discrete. To add more flexibility, generalized additive partial linear models can be considered, where some predictors can have a non‐linear effect while some predictors have a strictly linear effect. We consider Bayesian variable selection in these models. The functions in the non‐parametric additive part of the model are expanded in a B‐spline basis and multivariate Laplace prior put on the coefficients with point mass at zero. The coefficients corresponding to the strictly linear components are assigned a univariate Laplace prior with point mass at zero. The prior times the likelihood is mathematically intractable, but we find an approximation by expansion around the posterior mode, which is the group lasso solution in generalized linear model setting for the choice of prior. We thus completely avoid Markov chain Monte Carlo methods, which are extremely slow and unreliable in high‐dimensional models. We evaluate the performance of the Bayesian method by conducting simulation studies and real data analyses. Copyright © 2014 John Wiley & Sons, Ltd.

We illustrate a recently proposed two-step bootstrap model averaging (bootstrap MA) approach to cope with model selection uncertainty. The predictive performance is investigated in an example and in a simulation study. Results are compared to those derived from other model selection methods. In the framework of the linear regression model we use the two-step bootstrap MA, which consists of a screening step to eliminate covariates thought to have no influence on the response, and a model-averaging step. We also apply the full model, variable selection using backward elimination based on Akaike's Information Criterion (AIC), the Bayes Information Criterion (BIC) and the bagging approach. The predictive performance is measured by the mean squared error (MSE) and the coverage of confidence intervals for the true response. We obtained similar results for all approaches in the example. In the simulation the MSE was reduced by all approaches in comparison to the full model. The smallest values are obtained for bootstrap MA. Only the bootstrap MA and the full model correctly estimated the nominal coverage. The backward elimination procedures led to substantial underestimation and bagging to an overestimation of the true coverage. The screening step of bootstrap MA eliminates most of the unimportant factors. The new bootstrap MA approach shows promising results for predictive performance. It increases practical usefulness by eliminating unimportant factors in the screening step.

Regression models based on zero inflated distributions are oftenly used in exploratory data analysis having excess zeroes. The difficulty faced by many researchers is regarding the selection of covariates to be included in the model. Following the idea of focused information criterion, observed focused information criterion is proposed for model selection. The motivation for this has its roots in the concept of observed fisher information. Using this criterion, a forward selection procedure is proposed for selection of variables in regression models based on inflated distributions. The procedure is illustrated using a dataset on DMFT index using the modified observed focused information criterion.

In this article, we describe the estimation of linear regression models with uncertainty about the choice of the explanatory variables. We introduce the Stata commands bma and wals, which implement, respectively, the exact Bayesian model-averaging estimator and the weighted-average least-squares estimator developed by Magnus, Powell, and Pr¨ufer (2010, Journal of Econometrics 154: 139–153). Unlike standard pretest estimators that are based on some preliminary diagnostic test, these model-averaging estimators provide a coherent way of making inference on the regression parameters of interest by taking into account the uncertainty due to both the estimation and the model selection steps. Special emphasis is given to several practical issues that users are likely to face in applied work: equivariance to certain transformations of the explanatory variables, stability, accuracy, computing speed, and out-of-memory problems. Performances of our bma and wals commands are illustrated using simulated data and empirical applications from the literature on model-averaging estimation.

BACKGROUND This study investigates whether dietary patterns, substantiated by biomarkers, are associated with semen quality. METHODS In 161 men of subfertile couples undergoing in vitro fertilization treatment in a tertiary referral clinic in Rotterdam, the Netherlands, we assessed nutrient intakes and performed principal component factor analysis to identify dietary patterns. Total homocysteine (tHcy), folate, vitamin B12 and B6 were measured in blood and seminal plasma. Semen quality was assessed by sperm volume, concentration, motility, morphology and DNA fragmentation index (DFI). Linear regression models analyzed associations between dietary patterns, biomarkers and sperm parameters, adjusted for age, body mass index (BMI), smoking, vitamins and varicocele. RESULTS The 'Health Conscious' dietary pattern shows high intakes of fruits, vegetables, fish and whole grains. The 'Traditional Dutch' dietary pattern is characterized by high intakes of meat, potatoes and whole grains and low intakes of beverages and sweets. The 'Health Conscious' diet was inversely correlated with tHcy in blood (beta = -0.07, P = 0.02) and seminal plasma (beta = -1.34, P = 0.02) and positively with vitamin B6 in blood (beta = 0.217, P = 0.01). An inverse association was demonstrated between the 'Health Conscious' diet and DFI (beta = -2.81, P = 0.05). The 'Traditional Dutch' diet was positively correlated with red blood cell folate (beta = 0.06, P = 0.04) and sperm concentration (beta = 13.25, P = 0.01). CONCLUSIONS The 'Health Conscious' and 'Traditional Dutch' dietary pattern seem to be associated with semen quality in men of subfertile couples.

Introduction Clomiphene citrate (CC), a selective estrogen receptor modulator, and anastrozole, a selective aromatase inhibitor, are common empiric medical therapies for men with idiopathic infertility. The effects of jointly administering these agents are not well understood. Objective The objective of this study is to compare semen parameters in men treated with clomiphene citrate and anastrozole (CC-A) combination therapy to those in men taking anastrozole alone. Methods We performed a multi-center, retrospective cohort study of men with idiopathic infertility prescribed anastrozole at two tertiary referral institutions. Demographic, laboratory, and prescription data were captured. The subset of patients taking CC at the time of anastrozole initiation were identified. Patients with pre-treatment normozoospermia, recent or current exogenous testosterone or human chorionic gonadotropin use, previous orchiectomy, and scrotal surgery during the treatment period were excluded. Outcome measures included changes in World Health Organization sperm concentration category (WHO-SCC) following initiation of anastrozole therapy. Linear and logistic regression models were constructed to evaluate the association of combination therapy with these outcomes. Results 72 men were included in the analysis cohort, including 54 men taking anastrozole alone and 18 men on CC therapy who subsequently started taking anastrozole (CC-A). Men on CC-A relative to anastrozole monotherapy were older (40 vs. 35 years, p=0.03) and had a lower BMI (30 vs. 34 kg/m^2, p=0.05), respectively. Pre-treatment LH levels (5.6 vs. 7.6 IU/L, p=0.25) and testosterone (336 vs 263 ng/dL, p=0.32) were similar. Baseline semen parameters including sperm concentration (3.7 vs. 2.3 M/mL, p=0.26) and motility (41 vs. 36%, p= 0.17) were also comparable. However, post-treatment median sperm concentration (6.9 vs. 3.4M/mL, p=0.02) and the change in mean sperm concentration (17.5 vs. 6.2 M/mL, p=0.03) were significantly higher in the CC-A combination group. Univariate linear regression also revealed an association between the use of CCA combination therapy and increase in sperm concentration (p=0.05). Despite these differences, WHO-SCC upgrade rates between groups were comparable (56 vs. 39%, p=0.22) in the setting of similar post-treatment levels of LH (7.4 vs. 8.6 IU/L, p=0.5), FSH (8.6 vs. 9.4 IU/mL, p=0.19), and testosterone (537 vs. 415 ng/dL, p=0.09). Conclusions Combination therapy with clomiphene citrate and anastrozole was associated with modest benefits in post-treatment sperm concentration when compared to anastrozole monotherapy. Combination empiric medical therapy using clomiphene citrate and anastrozole offers a small, but significant, clinical benefit over anastrozole alone for infertile men Disclosure No