Likelihood Inference of Non-Constant Diversification Rates with Incomplete Taxon Sampling

Several total-evidence dating studies under the fossilized birth-death (FBD) model have produced very old age estimates, which are not supported by the fossil record. This phenomenon has been termed "deep root attraction (DRA)." For two specific data sets, involving divergence time estimation for the early radiations of ants, bees, and wasps (Hymenoptera) and of placental mammals (Eutheria), it has been shown that the DRA effect can be greatly reduced by accommodating the fact that extant species in these trees have been sampled to maximize diversity, so-called diversified sampling. Unfortunately, current methods to accommodate diversified sampling only consider the extreme case where it is possible to identify a cut-off time such that all splits occurring before this time are represented in the sampled tree but none of the younger splits. In reality, the sampling bias is rarely this extreme and may be difficult to model properly. Similar modeling challenges apply to the sampling of the fossil record. This raises the question of whether it is possible to find dating methods that are more robust to sampling biases. Here, we show that the skyline FBD (SFBD) process, where the diversification and fossil-sampling rates can vary over time in a piecewise fashion, provides age estimates that are more robust to inadequacies in the modeling of the sampling process and less sensitive to DRA effects. In the SFBD model we consider, rates in different time intervals are either considered to be independent and identically distributed or assumed to be autocorrelated following an Ornstein-Uhlenbeck (OU) process. Through simulations and reanalyses of Hymenoptera and Eutheria data, we show that both variants of the SFBD model unify age estimates under random and diversified sampling assumptions. The SFBD model can resolve DRA by absorbing the deviations from the sampling assumptions into the inferred dynamics of the diversification process over time. Although this means that the inferred diversification dynamics must be interpreted with caution, taking sampling biases into account, we conclude that the SFBD model represents the most robust approach currently available for addressing DRA in total-evidence dating.

The birth-death process is widely used in phylogenetics to model speciation and extinction. Recent studies have shown that the inferred rates are sensitive to assumptions about the sampling probability of lineages. Here, we examine the effect of the method used to sample lineages. Whereas previous studies have assumed random sampling (RS), we consider two extreme cases of biased sampling: "diversified sampling" (DS), where tips are selected to maximize diversity and "cluster sampling (CS)," where sample diversity is minimized. DS appears to be standard practice, for example, in analyses of higher taxa, whereas CS may occur under special circumstances, for example, in studies of geographically defined floras or faunas. Using both simulations and analyses of empirical data, we show that inferred rates may be heavily biased if the sampling strategy is not modeled correctly. In particular, when a diversified sample is treated as if it were a random or complete sample, the extinction rate is severely underestimated, often close to 0. Such dramatic errors may lead to serious consequences, for example, if estimated rates are used in assessing the vulnerability of threatened species to extinction. Using Bayesian model testing across 18 empirical data sets, we show that DS is commonly a better fit to the data than complete, random, or cluster sampling (CS). Inappropriate modeling of the sampling method may at least partly explain anomalous results that have previously been attributed to variation over time in birth and death rates.

This paper introduces a SAS/IML program to select among the multivariate model candidates based on a few well-known multivariate model selection criteria. Stepwise regression and all-possible-regression are considered. The program is user friendly and requires the user to paste or read the data at the beginning of the module, include the names of the dependent and independent variables (the y's and the x's), and then run the module. The program produces the multivariate candidate models based on the following criteria: Forward Selection, Forward Stepwise Regression, Backward Elimination, Mean Square Error, Coefficient of Multiple Determination, Adjusted Coefficient of Multiple Determination, Akaike's Information Criterion, the Corrected Form of Akaike's Information Criterion, Hannan and Quinn Information Criterion, the Corrected Form of Hannan and Quinn (HQc) Information Criterion, Schwarz's Criterion, and Mallow's PC. The output also constitutes detailed as well as summarized results.

The shape of the dose-response relation between particulate matter air pollution and mortality is crucial for public health assessment, and departures of this relation from linearity could have important regulatory consequences. A number of investigators have studied the shape of the particulate matter-mortality dose-response relation and concluded that the relation could be adequately described by a linear model. Some of these researchers examined the hypothesis of linearity by comparing Akaike's Information Criterion (AIC) values obtained under linear, piecewise linear, and spline alternative models. However, at the current time, the efficacy of the AIC in this context has not been assessed. The authors investigated AIC as a means of comparing competing dose-response models, using data from Cook County, Illinois, for the period 1987-2000. They found that if nonlinearities exist, the AIC is not always successful in detecting them. In a number of the scenarios considered, AIC was equivocal, picking the correct simulated dose-response model about half of the time. These findings suggest that further research into the shape of the dose-response relation using alternative model selection criteria may be warranted.

Capture-recapture methods estimate the size of hidden populations by leveraging the proportion of overlap of the population on independent lists. Log-linear modeling relaxes the assumption of list independence, but best model selection criteria remain uncertain. Incorrect model selection can deliver incorrect and even implausible size estimates. We used simulations to model when capture-recapture methods deliver biased or unbiased estimates and compare model selection criteria. Simulations included five scenarios for list dependence among three incomplete lists of population of interest. We compared metrics of log-linear model selection, accuracy, and precision. We also compared log-linear model performance to the decomposable graph approach (a Bayesian model average), the sparse multiple systems estimation (SparseMSE) approach that accounts for zero or low cell counts, and the Sample Coverage approach. Log-linear models selected by Akaike's information criterion (AIC) calculated accurate population size estimates in all scenarios except for those with sparse or zero cell counts. In these scenarios, the decomposable graph and the Sample Coverage models produced more accurate size estimates. Conventional capture-recapture model selection fails with sparse cell counts. This naïve approach to model selection should be replaced with the implementation of multiple different models in order triangulate the truth in real-world applications.

The literature on the ecology of marine assemblages includes frequent examples of data analysis with no well-defined alternative hypotheses for the definition of environmental variables (independent matrix for multivariate methods). Alternative models, whereby spatial or temporal patterns are investigated, should be explicitly assumed. We present a parsimonious procedure for model selection in multivariate data combined with canonical correspondence analysis to determine the measure of explained variance for each tested model, using Akaike's information criterion (AIC) for model selection. The AIC procedure is an effective tool for model selection and, in contrast to other conventional procedures that use only 1 implicit model and ignore other community patterns, it provides a framework for ranking hierarchical patterns that adds an alternative non-disjunctive per- spective to assemblage analysis. Hierarchical patterns are revealed as layers in a scale-dependent framework.

The analysis of ecological data has changed in two important ways over the last 15years. The development and easy availability of Bayesian computational methods has allowed and encouraged the fitting of complex hierarchical models. At the same time, there has been increasing emphasis on acknowledging and accounting for model uncertainty. Unfortunately, the ability to fit complex models has outstripped the development of tools for model selection and model evaluation: familiar model selection tools such as Akaike's information criterion and the deviance information criterion are widely known to be inadequate for hierarchical models. In addition, little attention has been paid to the evaluation of model adequacy in context of hierarchical modeling, i.e., to the evaluation of fit for a single model. In this paper, we describe Bayesian cross-validation, which provides tools for model selection and evaluation. We describe the Bayesian predictive information criterion and a Bayesian approximation to the BPIC known as the Watanabe-Akaike information criterion. We illustrate the use of these tools for model selection, and the use of Bayesian cross-validation as a tool for model evaluation, using three large data sets from the North American Breeding Bird Survey.

In this paper a model selection algorithm for a nonlinear system identification method is proposed to study functional magnetic resonance imaging (fMRI) effective connectivity. Unlike most other methods, this method does not need a pre-defined structure/model for effective connectivity analysis. Instead, it relies on selecting significant nonlinear or linear covariates for the differential equations to describe the mapping relationship between brain output (fMRI response) and input (experiment design). These covariates, as well as their coefficients, are estimated based on a least angle regression (LARS) method. In the implementation of the LARS method, Akaike's information criterion corrected (AICc) algorithm and the leave-one-out (LOO) cross-validation method were employed and compared for model selection. Simulation comparison between the dynamic causal model (DCM), nonlinear identification method, and model selection method for modelling the single-input-single-output (SISO) and multiple-input multiple-output (MIMO) systems were conducted. Results show that the LARS model selection method is faster than DCM and achieves a compact and economic nonlinear model simultaneously. To verify the efficacy of the proposed approach, an analysis of the dorsal and ventral visual pathway networks was carried out based on three real datasets. The results show that LARS can be used for model selection in an fMRI effective connectivity study with phase-encoded, standard block, and random block designs. It is also shown that the LOO cross-validation method for nonlinear model selection has less residual sum squares than the AICc algorithm for the study.

Background: Metastatic regional lymph nodes (LN) is a strong predictor of worse long-term outcome after resection of breast cancer. This study aimed to compare the prognostic performance of American Joint Committee on Cancer (AJCC) N stage relative to lymph node ratio (LNR), log odds of metastatic lymph nodes (LODDS), number of removed lymph nodes (NRLNs), and number of negative lymph nodes (NNLNs) in breast cancer patients. Methods: All of the breast cancer patients who underwent surgery between 2004 and 2012 were identified from the Surveillance, Epidemiology, and End Results database. Restricted cubic spline functions were used to characterize the association between continuous variables and the risk of death and determine the optimal cut-off points. The Cox proportional hazards models were constructed, and the relative discriminative abilities of the different LN staging systems were assessed using the Akaike's Information Criterion (AIC) and the Harrell's concordance index (C-index). Results: A total of 264,096 breast cancer patients were enrolled, and 177,598 (67.2%) had no lymph node metastasis, whereas 86,498 (32.8%) had lymph node metastasis. 187,785 (71.1%) patients had a limited number of LNs harvested (NRLN <10). The median follow-up time was 73 months, and the 8-year overall survival (OS) and breast cancer-specific survival (BCSS) were 82.6% and 90.4%, respectively. LNR, LODDS, NRLNs, and NNLNs were all nonlinearly associated with OS and BCSS. Patients with metastatic LN had an increased risk of OS (hazards ratio: 2.32, 95% confidence interval: 2.27–2.37; P < 0.001) and BCSS (hazards ratio: 4.53, 95% confidence interval: 4.40–4.66; P < 0.001). When LNR was equal to 0 or 1, there was a heterogeneity of outcomes, and LODDS still yielded informative values compared to LNR. Among the entire cohort, LNR modeled as a continuous variable had a somewhat better prognostic performance (AIC: 923231.4 and C-index: 0.722 for OS; AIC: 482962.3 and C-index: 0.817 for BCSS) than any of other LN staging systems. However, a model with AJCC N stage showed the best fit in patients with a limited number of LNs harvested (AIC: 501321.8 and C-index: 0.699 for OS; AIC: 212605.6 and C-index: 0.809 for BCSS). When assessed among patients with metastatic LN, LODDS outperformed other staging systems including AJCC N stage, LNR, NRLNs and NNLNs, whenever assessed using continuous (AIC: 428626.2 and C index: 0.728 for OS; AIC: 296886.8 and C index: 0.770 for BCSS) or categorical (AIC: 429527.5 and C index: 0.722 for OS; AIC: 297796.6 and C index: 0.762 for BCSS) cutoff values. Conclusions: Although LNR assessed as a continuous variable was the most potent method to stratify patients regardless of LN status, the prognostic superiority of LNR is confounded by a limited LN harvest. LODDS was a better and more powerful predictor of survival when patients were LN positive, especially among those patients with either very low or high LNR. Citation Format: Gong Y, Ji P, Jiang Y-Z, Hu X, Shao Z-M. Comparing prognostic performance of different lymph node staging systems among patients with breast cancer [abstract]. In: Proceedings of the 2018 San Antonio Breast Cancer Symposium; 2018 Dec 4-8; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2019;79(4 Suppl):Abstract nr P2-08-36.

Both Akaike information criterion and cross-validation are important tools in model selection. Stone [(1977), ‘An Asymptotic Equivalence of Choice of Model by Cross-Validation and Akaikes Criterion’, Journal of the Royal Statistical Society, Series B, 39, 44–47] established the equivalence of these two criteria for parametric models. In this paper, we build a similar equivalence for a large semiparametric family.

Prognostic indices combining several clinical and laboratory parameters have been proposed for prognostication in chronic lymphocytic leukemia (CLL). Recently, international consortium on CLL proposed an international prognostic index (CLL-IPI) integrating clinical, molecular, and genetic parameters. The present study was designed to evaluate the reproducibility of CLL-IPI in Indian CLL cohort. The prognostic ability of CLL-IPI in terms of overall survival (OS) and time to first treatment (TTFT) was investigated in treatment-naive CLL patients and also compared with other existing prognostic scores. For assigning scores, clinical and laboratory details were obtained from medical records, and IGHV gene mutation status, β2-microglobulin levels, and copy number variations were determined using c-DNA, ELISA, and multiplex ligation-dependent probe amplification (MLPA), respectively. The scores were generated as per the weighted grades assigned to each variable involved in score categorization. The predictive value of prognostic models was assessed and compared using Harrell's C-index and Akaike's information criterion (AIC). Stratification of patients according to CLL-IPI yielded significant differences in terms of OS and TTFT (p < 0.001). Comparative assessment of scores for OS suggested better performance of CLL-IPI (C = 0.64, AIC = 740) followed by Barcelona-Brno (C = 0.61, AIC = 754) and MDACC score (C = 0.59, AIC = 759). Comparison of predictive value of prognostic scores for TTFT illustrated better performance of CLL-IPI (C = 0.72, AIC = 726) followed by Barcelona-Brno (C = 0.68, AIC = 743), modified GCLLSG (C = 0.66, AIC = 744), and O-CLL1 index (C = 0.55, AIC = 773). The results suggest better performance of CLL-IPI in terms of both OS and TTFT as compared to other available scores in our cohort.

Divergence times, historical biogeography, and shifts in speciation rates of Myrtales

In order to have confidence in model-based phylogenetic analysis, the model of nucleotide substitution adopted must be selected in a statistically rigorous manner. Several model-selection methods are applicable to maximum likelihood (ML) analysis, including the hierarchical likelihood-ratio test (hLRT), Akaike information criterion (AIC), Bayesian information criterion (BIC), and decision theory (DT), but their performance relative to empirical data has not been investigated thoroughly. In this study, we use 250 phylogenetic data sets obtained from TreeBASE to examine the effects that choice in model selection has on ML estimation of phylogeny, with an emphasis on optimal topology, bootstrap support, and hypothesis testing. We show that the use of different methods leads to the selection of two or more models for approximately 80% of the data sets and that the AIC typically selects more complex models than alternative approaches. Although ML estimation with different best-fit models results in incongruent tree topologies approximately 50% of the time, these differences are primarily attributable to alternative resolutions of poorly supported nodes. Furthermore, topologies and bootstrap values estimated with ML using alternative statistically supported models are more similar to each other than to topologies and bootstrap values estimated with ML under the Kimura two-parameter (K2P) model or maximum parsimony (MP). In addition, Swofford-Olsen-Waddell-Hillis (SOWH) tests indicate that ML trees estimated with alternative best-fit models are usually not significantly different from each other when evaluated with the same model. However, ML trees estimated with statistically supported models are often significantly suboptimal to ML trees made with the K2P model when both are evaluated with K2P, indicating that not all models perform in an equivalent manner. Nevertheless, the use of alternative statistically supported models generally does not affect tests of monophyletic relationships under either the Shimodaira-Hasegawa (S-H) or SOWH methods. Our results suggest that although choice in model selection has a strong impact on optimal tree topology, it rarely affects evolutionary inferences drawn from the data because differences are mainly confined to poorly supported nodes. Moreover, since ML with alternative best-fit models tends to produce more similar estimates of phylogeny than ML under the K2P model or MP, the use of any statistically based model-selection method is vastly preferable to forgoing the model-selection process altogether.

본 논문은 불균형 자료에서 선형혼합모형에 적용되는 Akaike Information Criterion(AIC)의 효율에 대한 연구이다. Vaida와 Balanchard (2005)에 의해 제안된 cAIC(conditional AIC)는 mAIC(marginal AIC)가 임의효과의 예측에 대한 불확실성을 모형선택에서 반영하지 못하는 단점을 극복할 수 있는 방법이다. cAIC에 대한 이론적인 성질과 확장은 Liang 등 (2008)과 Greven과 Kneib (2010)에 의하여 연구되었다. cAIC의 형태는 자료의 구조에 영향을 받지는 않지만 선형혼합모형에서 모수의 추정 효율은 자료의 불균형의 정도에 따라 많은 영향을 받는 것이 알려져 있다. 기존의 연구에서 실시한 모든 모의실험이 자료가 균형인 경우에만 실행되어 자료의 불균형이 AIC에 근거한 혼합모형 선택 방법의 효율에 어떤 영향을 미치는지 알려져 있지 않다. 본 논문은 자료의 불균형이 모형선택 방법의 효율에 미치는 영향을 모의실험을 통하여 알아보았다. 자료의 불균형이 심해짐에 따라 AIC에 근거한 모형선택방법은 복잡한 모형을 선택하는 경향이 낮아짐을 보였다. This article consider a performance model selection based on AIC under unbalanced deign in linear mixed effect models. Vaida and Balanchard (2005) proposed conditional AIC for model selection in linear mixed effect models when the prediction of random effects is of primary interest. Theoretical properties of cAIC and related criteria have been investigated by Liang et al. (2008) and Greven and Kneib (2010). However, all of the simulation studies were performed under a balanced design. Even though functional form of AIC remain same even under the unbalanced deign, it is worthwhile to investigate performance of AIC based model selection criteria under the unbalanced design. The simulation study in this article shows how unbalancedness affects model selection in linear mixed effect models.

Most model selection procedures use a fixed penalty penalizing an increase in the size of a model. These nonadaptive selection procedures perform well only in one type of situation. For instance, Bayesian information criterion (BIC) with a large penalty performs well for “small” models and poorly for “large” models, and Akaike's information criterion (AIC) does just the opposite. This article proposes an adaptive model selection procedure that uses a data-adaptive complexity penalty based on a concept of generalized degrees of freedom. The proposed procedure, combining the benefit of a class of nonadaptive procedures, approximates the best performance of this class of procedures across a variety of different situations. This class includes many well-known procedures, such as AIC, BIC, Mallows's Cp, and risk inflation criterion (RIC). The proposed procedure is applied to wavelet thresholding in nonparametric regression and variable selection in least squares regression. Simulation results and an asymptotic analysis support the effectiveness of the proposed procedure.