The likelihood ratio test for hidden Markov models in two-sample problems

This paper studies the applications of extreme value theory on analysis for closing price data of the Dow-Jones industrial index and Danish fire insurance claims data. The generalized extreme value (GEV) distribution is considered in analyzing the real data, and the hypothesis testing problem for the shape parameter of GEV distribution is investigated based on a new test statistic—the $$L_q$$ -likelihood ratio ( $$L_q$$ R) statistic. The $$L_q$$ R statistic can be treated as a generalized form of the classical likelihood ratio (LR) statistic. We show that the asymptotic behavior of proposed statistic is characterized by the degree of distortion $$q$$ . For small and modest sample sizes, the $$L_q$$ R statistic is still available when $$q$$ is properly chosen. By simulation studies, the proposed statistic not only performs the asymptotic properties, but also outperforms the classical LR statistic as the sample sizes are modest or even small. Meanwhile, the test power based on the new statistic is also simulated by Monte Carlo methods. At last, the models are diagnosed by graphical methods.

Few studies have considered the impacts of sample size and sample ratio of presence and absence points on the results of random forest (RF) testing. We applied this technique for the prediction of the spatial distribution of snail habitats based on a total of 15,000 sample points (5,000 presence samples and 10,000 control points). RF models were built using seven different sample ratios (1:1, 1:2, 1:3, 1:4, 2:1, 3:1, and 4:1) and the optimal ratio was identified via the Area Under the Curve (AUC) statistic. The impact of sample size was compared by RF models under the optimal ratio and the optimal sample size. When the sample size was small, the sampling ratios of 1:1, 1:2 and 1:3 were significantly better than the sample ratios of 4:1 and 3:1 at all four levels of sample sizes (p<0.01) and there was no significant difference among the ratios of 1:1, 1:2 and 1:3 (p>0.05). The sample ratio of 1:2 appeared to be optimal for a relatively large sample size with the lowest quartile deviation. In addition, increasing the sample size produced a higher AUC and a smaller slope and the most suitable sample size found in this study was 2400 (AUC=0.96). This study provides a feasible idea to select an appropriate sample size and sample ratio for ecological niche modelling (ENM) and also provides a scientific basis for the selection of samples to accurately identify and predict snail habitat distributions.

In some survival studies part of the population may be no longer subject to the event of interest. The called cure rate models take this fact into account. They have been extensively studied for several authors who have proposed extensions and applications in real lifetime data. Classic large sample tests are usually considered in these applications, especially the likelihood ratio. Recently a new test called \textit{gradient test} has been proposed. The gradient statistic shares the same asymptotic properties with the classic likelihood ratio and does not involve knowledge of the information matrix, which can be an advantage in survival models. Some simulation studies have been carried out to explore the behavior of the gradient test in finite samples and compare it with the classic tests in different models. However little is known about the properties of these large sample tests in finite sample for cure rate models. In this work we performed a simulation study based on the promotion time model with Weibull distribution, to assess the performance of likelihood ratio and gradient tests in finite samples. An application is presented to illustrate the results.

Summary The mixed linear model is a popular method for analysing unbalanced repeated measurement data. The classical statistical tests for parameters in this model are based on asymptotic theory that is unreliable in the small samples that are often encountered in practice. For testing a given fixed effect parameter with a small sample, we develop and investigate refined likelihood ratio (LR) tests. The refinements considered are the Bartlett correction and use of the Cox–Reid adjusted likelihood; these are examined separately and in combination. We illustrate the various LR tests on an actual data set and compare them in two simulation studies. The conventional LR test yields type I error rates that are higher than nominal. The adjusted LR test yields rates that are lower than nominal, with absolute accuracy similar to that of the conventional LR test in the first simulation study and better in the second. The Bartlett correction substantially improves the accuracy of the type I error rates with either the conventional or the adjusted LR test. In many cases, error rates that are very close to nominal are achieved with the refined methods.

In the present paper, a two-sided test in a family of multivariate distribution according to the Mahalanobis distance with mean vector and positive definite matrix is considered. First, a family of multivariate distribution is introduced, then using the likelihood ratio method a test statistic is computed. The distribution of the test statistic is proposed for different sample sizes and fixed dimension. We study the distribution approximation computed using the likelihood ratio test and an efficient algorithm to compute the density functions can be derived according to Witkovsk´y, J. Stat. Plan. Inference. 94 (2001), 1–13. Also, a simulation study is presented on the sample sizes and powers to compare the performance of tests and show that the proposed distribution approximation is better than the classical distribution approximation.

BackgroundThere has been a considerable increase in studies investigating rates of diversification and character evolution, with one of the promising techniques being the BiSSE method (binary state speciation and extinction). This study uses simulations under a variety of different sample sizes (number of tips) and asymmetries of rate (speciation, extinction, character change) to determine BiSSE’s ability to test hypotheses, and investigate whether the method is susceptible to confounding effects.ResultsWe found that the power of the BiSSE method is severely affected by both sample size and high tip ratio bias (one character state dominates among observed tips). Sample size and high tip ratio bias also reduced accuracy and precision of parameter estimation, and resulted in the inability to infer which rate asymmetry caused the excess of a character state. In low tip ratio bias scenarios with appropriate tip sample size, BiSSE accurately estimated the rate asymmetry causing character state excess, avoiding the issue of confounding effects.ConclusionsBased on our findings, we recommend that future studies utilizing BiSSE that have fewer than 300 terminals and/or have datasets where high tip ratio bias is observed (i.e., fewer than 10% of species are of one character state) should be extremely cautious with the interpretation of hypothesis testing results.

Degisim katsayisi, bir rastgele degiskenin gozlemlerinin tutarliligini ya da tekduzeligini olcer. Degisim katsayisi, ortalama basina standart sapma oldugundan goreceli degiskenligin bir olcusudur. Bu sebepten dolayi guruplar arasindaki degiskenligi karsilastirmak icin kullanilmaktadir. Literaturde k sayida normal yiginin DK'larini karsilastirmak icin bazi testler onerilmistir. Bu calismada k normal yiginin degisim katsayilarinin esitliginin test edilmesinde kullanilan duzenlenmis Bennett testi, skor testi, merkezi olmayan t testi, olabilirlik oran testi, genellestirilmis p yaklasimi incelenmistir. Ayrica bu testler, farkli ornek capi ve gurup sayisi durumlari altinda deneysel I.tip hata oranlari ve gucleri bakimindan simulasyon yoluyla karsilastirilmis ve sonuclari yorumlanmistir. Simulasyon calismasindan elde edilen sonuclara gore, ozellikle ornek caplari esit ve grup sayisi kucuk iken duzenlenmis Bennett testi, grup sayisi arttikca ise skor testi diger testlere gore daha iyi oldugu gozlenmistir.

In this paper, we examine by Monte Carlo experiments the small sample properties of the W (Wald), LM (Lagrange Multiplier) and LR (Likelihood Ratio) tests for equality between sets of coefficients in two linear regressions under heteroscedasticity. The small sample properties of the size-corrected W, LM and LR tests proposed by Rothenberg (1984) are also examined and it is shown that the performances of the size-corrected W and LM tests are very good. Further, we examine the two-stage test which consists of a test for homoscedasticity followed by the Chow (1960) test if homoscedasticity is indicated or one of the W, LM or LR tests if heteroscedasticity should be assumed. It is shown that the pretest does not reduce much the bias in the size when the sizecorrected citical values are used in the W, LM and LR tests.

The new estimators, based on Hotelling's (Hotelling, 1953) adjusted Z statistic (Fisher, 1921), of a common corrlation coefficient ρ from k≧2 independent random samples drawn from bivariate normal population are developed. By using a simulation study these estimators are compared with three estimators studied by Donner and Rosner (Applied Statistics, 1980). For testing the significance of the common ρ we derive the C(α) test statistic and the likelihood ratio statistic. By using the same simulation study we compare the performance of these statistics with a few others based on estimators of the common ρ. An estimator based on Hotelling's adjusted Z-statistic, applicable to both equal and unequal sample sizes, perform best for ρ < 5 whereas an estimator based on Fisher's Z statistic performs best for ρ≧.5. For testing significance of the common correlation the C(α) test statistic (Neyman, 1959) performs best with respect to both size and power and also it has a remarkably simple form.

4 - A moments-based procedure for evaluating risk forecasting models

Scientists often interpret P-values as measures of the relative strength of statistical findings. This is common practice in large-scale genomic studies where P-values are used to choose which of numerous hypothesis test results should be pursued in subsequent research. In this study, we examine P-value variability to assess the degree of certainty P-values provide. We develop prediction intervals for the P-value in a replication study given the P-value observed in an initial study. The intervals depend on the initial value of P and the ratio of sample sizes between the initial and replication studies, but not on the underlying effect size or initial sample size. The intervals are valid for most large-sample statistical tests in any context, and can be used in the presence of single or multiple tests. While P-values are highly variable, future P-value variability can be explicitly predicted based on a P-value from an initial study. The relative size of the replication and initial study is an important predictor of the P-value in a subsequent replication study. We provide a handy calculator implementing these results and apply them to a study of Alzheimer's disease and recent findings of the Cross-Disorder Group of the Psychiatric Genomics Consortium. This study suggests that overinterpretation of very significant, but highly variable, P-values is an important factor contributing to the unexpectedly high incidence of non-replication. Formal prediction intervals can also provide realistic interpretations and comparisons of P-values associated with different estimated effect sizes and sample sizes.

When using maximum likelihood methods to estimate genetic and environmental components of (co)variance, it is common to test hypotheses using likelihood ratio tests, since such tests have desirable asymptotic properties. In particular, the standard likelihood ratio test statistic is assumed asymptotically to follow a χ2distribution with degrees of freedom equal to the number of parameters tested. Using the relationship between least squares and maximum likelihood estimators for balanced designs, it is shown why the asymptotic distribution of the likelihood ratio test for variance components does not follow a χ2distribution with degrees of freedom equal to the number of parameters tested when the null hypothesis is true. Instead, the distribution of the likelihood ratio test is a mixture of χ2distributions with different degrees of freedom. Implications for testing variance components in twin designs and for quantitative trait loci mapping are discussed. The appropriate distribution of the likelihood ratio test statistic should be used in hypothesis testing and model selection.

A simple iterative procedure is found for the exact null and alternative distributions of likelihood ratio, cumulative sum and related statistics for testing for a change in probability of a sequence of independent binomial random variables. It is concluded that the likelihood ratio test is slightly less powerful than the cumulative sum test near the middle of the sequence, but that the likelihood ratio test is much more powerful near the ends. An example from epidemiology is used to illustrate the results. Under certain conditions the cumulative sum test is identical to a two-sample Kolmogorov-Smirnov test and so the iterative procedure can be used to find the null distribution of Kolmogorov-Smirnov and related statistics.

Multivariate linear regressions are widely used statistical tools in many applications to model the associations between multiple related responses and a set of predictors. To infer such associations, it is often of interest to test the structure of the regression coefficients matrix, and the likelihood ratio test (LRT) is one of the most popular approaches in practice. Despite its popularity, it is known that the classical $\chi^2$ approximations for LRTs often fail in high-dimensional settings, where the dimensions of responses and predictors $(m,p)$ are allowed to grow with the sample size $n$. Though various corrected LRTs and other test statistics have been proposed in the literature, the fundamental question of when the classic LRT starts to fail is less studied, an answer to which would provide insights for practitioners, especially when analyzing data with $m/n$ and $p/n$ small but not negligible. Moreover, the power performance of the LRT in high-dimensional data analysis remains underexplored. To address these issues, the first part of this work gives the asymptotic boundary where the classical LRT fails and develops the corrected limiting distribution of the LRT for a general asymptotic regime. The second part of this work further studies the test power of the LRT in the high-dimensional setting. The result not only advances the current understanding of asymptotic behavior of the LRT under alternative hypothesis, but also motivates the development of a power-enhanced LRT. The third part of this work considers the setting with $p>n$, where the LRT is not well-defined. We propose a two-step testing procedure by first performing dimension reduction and then applying the proposed LRT. Theoretical properties are developed to ensure the validity of the proposed method. Numerical studies are also presented to demonstrate its good performance.

High throughput RNA sequencing (RNA-seq) technology is increasingly used in disease-related biomarker studies. A negative binomial distribution has become the popular choice for modeling read counts of genes in RNA-seq data due to over-dispersed read counts. In this study, we propose two explicit sample size calculation methods for RNA-seq data using a negative binomial regression model. To derive these new sample size formulas, the common dispersion parameter and the size factor as an offset via a natural logarithm link function are incorporated. A two-sided Wald test statistic derived from the coefficient parameter is used for testing a single gene at a nominal significance level 0.05 and multiple genes at a false discovery rate 0.05. The variance for the Wald test is computed from the variance-covariance matrix with the parameters estimated from the maximum likelihood estimates under the unrestricted and constrained scenarios. The performance and a side-by-side comparison of our new formulas with three existing methods with a Wald test, a likelihood ratio test or an exact test are evaluated via simulation studies. Since other methods are much computationally extensive, we recommend our M1 method for quick and direct estimation of sample sizes in an experimental design. Finally, we illustrate sample sizes estimation using an existing breast cancer RNA-seq data.