- Research Article
- 10.1097/01720610-200804000-00006
Considerations of sample size in medical research
- Apr 01, 2008
- Journal of the American Academy of Physician Assistants
- John W Waterbor + 1 more +1
Considerations of sample size in medical research
An evaluation framework for statistical tests on microarray data
Considerations of sample size in medical research
Considerations of sample size in medical research
Design and analysis of a 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria: Small sample considerations for cluster-randomized trials with count data.
Cluster-randomized trials allow for the evaluation of a community-level or group-/cluster-level intervention. For studies that require a cluster-randomized trial design to evaluate cluster-level interventions aimed at controlling vector-borne diseases, it may be difficult to assess a large number of clusters while performing the additional work needed to monitor participants, vectors, and environmental factors associated with the disease. One such example of a cluster-randomized trial with few clusters was the "efficacy and risk of harms of repeated ivermectin mass drug administrations for control of malaria" trial. Although previous work has provided recommendations for analyzing trials like repeated ivermectin mass drug administrations for control of malaria, additional evaluation of the multiple approaches for analysis is needed for study designs with count outcomes. Using a simulation study, we applied three analysis frameworks to three cluster-randomized trial designs (single-year, 2-year parallel, and 2-year crossover) in the context of a 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria. Mixed-effects models, generalized estimating equations, and cluster-level analyses were evaluated. Additional 2-year parallel designs with different numbers of clusters and different cluster correlations were also explored. Mixed-effects models with a small sample correction and unweighted cluster-level summaries yielded both high power and control of the Type I error rate. Generalized estimating equation approaches that utilized small sample corrections controlled the Type I error rate but did not confer greater power when compared to a mixed model approach with small sample correction. The crossover design generally yielded higher power relative to the parallel equivalent. Differences in power between analysis methods became less pronounced as the number of clusters increased. The strength of within-cluster correlation impacted the relative differences in power. Regardless of study design, cluster-level analyses as well as individual-level analyses like mixed-effects models or generalized estimating equations with small sample size corrections can both provide reliable results in small cluster settings. For 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria, we recommend a mixed-effects model with a pseudo-likelihood approximation method and Kenward-Roger correction. Similarly designed studies with small sample sizes and count outcomes should consider adjustments for small sample sizes when using a mixed-effects model or generalized estimating equation for analysis. Although the 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria is already underway as a parallel trial, applying the simulation parameters to a crossover design yielded improved power, suggesting that crossover designs may be valuable in settings where the number of available clusters is limited. Finally, the sensitivity of the analysis approach to the strength of within-cluster correlation should be carefully considered when selecting the primary analysis for a cluster-randomized trial.
Read moreAssessing the impact of restricted follow-up and small sample sizes on survival estimations in prostate cancer using registry data.
294 Background: Economic evaluations in oncology aim to assess the value of new therapies in the long term based on clinical trial data that often have restricted follow-up times (< 5 years) and small sample sizes (< 500 patients). This requires the use of extrapolation assumptions on long-term survival that go beyond the observed data. In this analysis, differences in survival extrapolation methods are tested in samples of sizes and follow-up reflecting typical clinical trials against a background of known survival in prostate cancer from a US based cancer registry. Methods: Data from the National Cancer Institute's Surveillance Epidemiology and End Results (SEER) registry on long-term survival in patients with stage IV prostate cancer were employed. The data set comprised those patients diagnosed between 1988 and 2003, with follow-up data available until 2012. Additional survival for those who received surgery (compared to those who did not), was estimated based on extrapolations using standard parametric statistical models (exponential, Weibull, log-logistic, log-normal, Gamma) fitted to the observed data. Survival analyses were run for 5 sample size scenarios (n = 27,670, 1000, 500, 200, 50) and 6 follow-up scenarios (follow-up years = 25, 20, 10, 5, 2, 1) yielding 30 combination scenarios. Performance of the methods was tested relative to the maximum follow-up, maximum sample size scenario (i.e. reference case) from the SEER registry. Results: Log-logistic and log-normal models were associated with flat tails which led to inflated survival estimations. For scenarios with smaller sizes, gamma models often did not converge. Exponential models were the most frequently reported as best model fit (in approximately 50% of scenarios). Also, gains in OS were consistent when exponential models were selected, and closely matched gain in OS from the reference case. Conclusions: Since clinical trials in oncology are often associated with small patient sample sizes and restricted follow-up, selecting an exponential model may lead to the most consistent and stable results based on the experiment constructed here. Further research should confirm these results for other types of cancer.
Read moreAssociation test for rare variants using the hamming distance
The recent development of DNA sequencing technology has given rise to many statistical methods for Rare Variant Association Studies (RVASs), such as burden and sequence kernel association tests. However, these methods, which usually require large samples, can lose power in association studies with small samples. In this study, we propose two statistical approaches applicable for RVASs when the sample size is not large. Our approaches are based on the Hamming distance, which compares the dissimilarity of Single Nucleotide Polymorphisms (SNPs) components between cases and controls. Existing Hamming distance-based methods mainly analyse common variants. For rare variant data with a small sample size, we extended two existing methods by using the weight based on minor allele frequency. Through simulation studies, we show that our proposed approaches control type 1 error rates and are more powerful even when given very small sample sizes. They also work well regardless of the direction of causal SNP effects. Applying these methods to real data, we confirmed that they identified true causal genes well. Based on the results of this study, we firmly believe that our proposed methods are powerful for small sample data.
Read moreOversampling and replacement strategies in propensity score matching: a critical review focused on small sample size in clinical settings
BackgroundPropensity score matching is a statistical method that is often used to make inferences on the treatment effects in observational studies. In recent years, there has been widespread use of the technique in the cardiothoracic surgery literature to evaluate to potential benefits of new surgical therapies or procedures. However, the small sample size and the strong dependence of the treatment assignment on the baseline covariates that often characterize these studies make such an evaluation challenging from a statistical point of view. In such settings, the use of propensity score matching in combination with oversampling and replacement may provide a solution to these issues by increasing the initial sample size of the study and thus improving the statistical power that is needed to detect the effect of interest. In this study, we review the use of propensity score matching in combination with oversampling and replacement in small sample size settings.MethodsWe performed a series of Monte Carlo simulations to evaluate how the sample size, the proportion of treated, and the assignment mechanism affect the performances of the proposed approaches. We assessed the performances with overall balance, relative bias, root mean squared error and nominal coverage. Moreover, we illustrate the methods using a real case study from the cardiac surgery literature.ResultsMatching without replacement produced estimates with lower bias and better nominal coverage than matching with replacement when 1:1 matching was considered. In contrast to that, matching with replacement showed better balance, relative bias, and root mean squared error than matching without replacement for increasing levels of oversampling. The best nominal coverage was obtained by using the estimator that accounts for uncertainty in the matching procedure on sets of units obtained after matching with replacement.ConclusionsThe use of replacement provides the most reliable treatment effect estimates and that no more than 1 or 2 units from the control group should be matched to each treated observation. Moreover, the variance estimator that accounts for the uncertainty in the matching procedure should be used to estimate the treatment effect.
Read moreMapping evergreen broad-leaved species spatial cover in Italian forests from Sentinel-2 time series using Deep Learning
Climate change-induced shifts, such as prolonged growing seasons and milder winters, coupled with land-use alterations like forest management abandonment, are reshaping species composition across European forests. A significant spread of evergreen broad-leaved species (EVEs) was observed in southern European forests, driven by global change dynamics. However, large-scale spatio-temporal analysis of these changes are lacking and emphasizes the necessity of mapping these dynamics. The TRACEVE projects (&#8220;Tracing the evergreen broad-leaved species and their spread&#8221;) primary goal is therefore&#160;tracking&#160; EVEs' cover, spread and diversity on a national-scale in Italy. As part of the project, this study focuses on satellite remote sensing, Deep Learning, and forest mapping, aiming at creating seamless maps quantifying the current degree of EVEs cover within forests in Italy.Challenges arise in the transitional zones between evergreen and deciduous forests, where EVEs initially spread in the understory of a deciduous canopy. Tracking EVEs at the edge of their range, where abundance is rare and in mixed forests is therefore&#160;difficult. Leveraging Sentinel-2 remote sensing time series covering the full annual phenological cycle addresses this challenge, utilizing leaf-on and leaf-off canopy conditions. Values of species cover derived from ad-hoc&#160;forest plot observations in Italian protected areas&#160;across a latitudinal gradient serve as initial training data, although the small sampling size (~1000 plots) poses challenges for the generalizability of time series extrinsic regression models, particularly when employing state-of-the-art Deep Learning architectures.The main aim of the study is therefore the development of a robust, Sentinel-2 based mapping procedure to track EVEs within Italian forests on a national-scale. A remote sensing time series extrinsic regression model based on a Deep Learning architecture for cover degree mapping will be developed. Sentinel-2 annual time series, along with derived indices&#160;serve as predictors for the models, while target cover will be derived from the plot observations. The study is built around exploring selected strategies&#160;to address the issue of large-scale model generalizability, given a small training sample size, that is recorded within small representative areas scattered across Italy.&#160;This entails assessing the efficacy of self-supervised pretraining methodologies applied to remote sensing time series within forested regions, pretraining on a more extensive forest database, and evaluating the viability of training data augmentation techniques.To validate and evaluate results on a national scale, an independent forest vegetation database containing around 17,000 forest plots sampled across Italy is employed. This extensive dataset enhances the understanding of EVEs' distribution within Italy's diverse forest ecosystems and can further enhance understanding of complex model results.In conclusion, the study combines advanced satellite remote sensing technologies, Deep Learning methodologies coupled with vegetation plot datasets to map current distribution of EVEs in Italian forests. The findings contribute to the TRACEVE project's future objectives but also offer insights into the challenges and opportunities of Deep Learning models in large-scale forest mapping applications.AcknowledgementsThis research has been conducted within the project &#8220;TRACEVE - Tracing the evergreen broad-leaved species and their spread&#8221; (I 6452-B) funded by the Austrian Science Fund (FWF).
Read moreA comparative study on model selection and multiple model fusion
There exist quite a few criteria for penalty-based model selection. Although they have various justifications for large sample problems, their performance under small or moderate sample size is unclear which hinders the development of model combination methods using the appropriate penalty term. In this paper, we assess the performance of seven model selection criteria based on linear regression models with unknown noise variance. We set the true data generation mechanism to be within the model set as well as outside the model set. In the latter case, soft model selection through multiple model fusion is proposed and its difference from Bayesian model averaging is highlighted. The penalty term used in each model selection criterion provides a natural link to estimate the model probability without assuming any prior knowledge of the unknown parameter. An important question is whether the estimated model probabilities are consistent when multiple models are fused for prediction or interpolation. We argue that strong consistency only holds under large sample regime while soft model selection can still be better than choosing a single model with small sample size. Our numerical results using different model selection criteria for polynomial fitting indicate that the conditional model estimator (CME) has the best performance in selecting the correct model order and fusing multiple models for prediction and interpolation. The minimum description length (MDL) based criteria are next to CME and outperform Bayesian information criterion (BIC) and Akaike information criterion (AIC) significantly.
Read moreSimilarity-Principle-Based Machine Learning Method for Clinical Trials and Beyond
With recent success in supervised learning, artificial intelligence (AI) and machine learning (ML) can play a vital role in precision medicine. Deep learning neural networks have been used in drug discovery when larger data is available. However, applications of machine learning in clinical trials with small sample size (around a few hundreds) are limited. We propose a Similarity-Principle-Based Machine Learning (SBML) method, which is applicable for small and large sample size problems. In SBML, the attribute-scaling factors are introduced to objectively determine the relative importance of each attribute (predictor). The gradient method is used in learning (training), that is, updating the attribute-scaling factors. We evaluate SBML when the sample size is small and investigate the effects of tuning parameters. Simulations show that SBML achieves better predictions in terms of mean squared errors for various complicated nonlinear situations than full linear models, optimal and ridge regressions, mixed effect models, support vector machine and decision tree methods.
Read moreAutoregressive Prediction with Rolling Mechanism for Time Series Forecasting with Small Sample Size
Reasonable prediction makes significant practical sense to stochastic and unstable time series analysis with small or limited sample size. Motivated by the rolling idea in grey theory and the practical relevance of very short‐term forecasting or 1‐step‐ahead prediction, a novel autoregressive (AR) prediction approach with rolling mechanism is proposed. In the modeling procedure, a new developed AR equation, which can be used to model nonstationary time series, is constructed in each prediction step. Meanwhile, the data window, for the next step ahead forecasting, rolls on by adding the most recent derived prediction result while deleting the first value of the former used sample data set. This rolling mechanism is an efficient technique for its advantages of improved forecasting accuracy, applicability in the case of limited and unstable data situations, and requirement of little computational effort. The general performance, influence of sample size, nonlinearity dynamic mechanism, and significance of the observed trends, as well as innovation variance, are illustrated and verified with Monte Carlo simulations. The proposed methodology is then applied to several practical data sets, including multiple building settlement sequences and two economic series.
Read moreA Practical Solution to the Small Sample Size Bias and Uncertainty Problems of Model Selection Criteria in Two-Input Process Multiple Response Surface Methodology Datasets
Multiple response surface methodology (MRSM) most often involves the analysis of small sample size datasets which have associated inherent statistical modeling problems. Firstly, classical model selection criteria in use are very inefficient with small sample size datasets. Secondly, classical model selection criteria have an acknowledged selection uncertainty problem. Finally, there is a credibility problem associated with modeling small sample sizes of the order of most MRSM datasets. This work focuses on determination of a solution to these identified problems. The small sample model selection uncertainty problem is analysed using sixteen model selection criteria and a typical two-input MRSM dataset. Selection of candidate models, for the responses in consideration, is done based on response surface conformity to expectation to deliberately avoid selection of models using the problematic classical model selection criteria. A set of permutations of combinations of response models with conforming response surfaces is determined. Each combination is optimised and results are obtained using overlaying of data matrices. The permutation of results is then averaged to obtain credible results. Thus, a transparent multiple model approach is used to obtain the solution which gives some credibility to the small sample size results of the typical MRSM dataset. The conclusion is that, for a two-input process MRSM problem, conformity of response surfaces can be effectively used to select candidate models and thus the use of the problematic model selection criteria is avoidable.
Read more범주형 반복측정자료를 위한 일반화 추정방정식의 소표본 특성
Liang과 Zeger는 이산형 혹은 연속형 반복측정자료를 분석하기 위한 일반화 추정방정식 (GEE)을 제안하였다 GEE모형은 범주형 반복측정자료의 모형으로 확장될 수 있으며, 이 GEE추정량은 대표본인 경우 다변량 정규분포를 따른다. 그러나 GEE는 대표본근사이론에 기초한다. 본 논문에서는 소표본인 경우 반복 측정된 순서자료에 대한 GEE추정량의 성질을 연구한다. 우리는 두가지 방법을 사용하여 두그룹의 반복 측정된 순서자료를 생성하며 모의실험을 통하여 소표본인 경우 여러 개 범주를 갖는 순서반응 자료에 대하여 GEE추정량의 1종 오류율, 검정력, 상대효율, 두 그룹의 표본크기가 다를 경우 효과, 그리고 분산 추정량의 성질등을 연구한다. Liang and Zeger proposed generalized estimating equations(GEE) for analyzing repeated data which is discrete or continuous. GEE model can be extended to model for repeated categorical data and its estimator has asymptotic multivariate normal distribution in large sample sizes. But GEE is based on large sample asymptotic theory. In this paper, we study the properties of GEE estimators for repeated ordinal data in small sample sizes. We generate ordinal repeated measurements for two groups using two methods. Through Monte Carlo simulation studies we investigate the empirical type 1 error rates, powers, relative efficiencies of the GEE estimators, the effect of unequal sample size of two groups, and the performance of variance estimators for polytomous ordinal response variables, especially in small sample sizes.
Read moreRegularized Exploratory Bifactor Analysis With Small Sample Sizes.
Several methods of factor extraction have recently gained popularity as a procedure for dealing with estimation problems associated with small sample sizes, which can be found in the various behavioral science disciplines, such as comparative psychology and behavior genetics. Two popular approaches for particularly small samples (below 50) include unweighted least squares factor analysis (ULS-FA) and regularized exploratory factor analysis (REFA). However, it is unclear how well each of the approaches performs with small samples in the context of exploratory bifactor modeling. In the current study, a comprehensive simulation study was conducted to evaluate the small sample behavior of the two approaches in terms of bifactor structure recovery under different sample size, factor loading, number of variables per factor, number of factors, and factor correlation experimental conditions. The results show that REFA is recommended for use over ULS-FA, particularly in the conditions involving low factor loadings, few group factors, or a small number of variables per factor.
Read moreA Monte Carlo Comparison of Seven ε-Adjustment Procedures in Repeated Measures Designs With Small Sample Sizes
The purpose of this study was to evaluate seven univariate procedures for testing omnibus null hypotheses for data gathered from repeated measures designs. Five alternate approaches are compared to the two more traditional adjustment procedures (Geisser and Greenhouse’s ε̂ and Huynh and Feldt’s ε̃), neither of which may be entirely adequate when sample sizes are small and the number of levels of the repeated factors is large. Empirical Type I error rates and power levels were obtained by simulation for conditions where small samples occur in combination with many levels of the repeated factor. Results suggested that alternate univariate approaches were improvements to the traditional approaches. One alternate approach in particular was found to be most effective in controlling Type I error rates without unduly sacrificing power.
Read moreEstudos de expressão gênica utilizando-se microarrays: delineamento, análise, e aplicações na pesquisa zootécnica
A tecnologia de microarrays, ou microarranjos de DNA, possibilita a avaliação simultânea da expressão de milhares de genes em diferentes tecidos em determinado organismo, em diferentes estágios de desenvolvimento ou condições ambientais. Microarrays são bastante utilizados em experimentos de genômica funcional com diversas espécies animais e vegetais, e têm sido gradativamente incorporados em diferentes áreas da pesquisa zootécnica, como crescimento e metabolismo, resposta imune a doenças, reprodução e resposta a fatores de estresse não-infecciosos (restrição alimentar, exposição a elementos tóxicos e outras condições ambientais desfavoráveis), bem como melhoramento genético animal. Tais experimentos, entretanto, são ainda consideravelmente caros, como consequência, geralmente são conduzidos com tamanhos amostrais relativamente pequenos. Por outro lado, a realização dos experimentos com microarrays, desde a coleta das amostras, até a obtenção das imagens para análise, envolve uma série de procedimentos laboratoriais de alta complexidade, que frequentemente introduzem variações adicionais aos resultados obtidos. Desta maneira, a condução de ensaios com microarrays requer cuidadoso delineamento experimental e análise estatística dos dados. Nesta apresentação são discutidos princípios básicos do planejamento de ensaios com microarrays, bem como as ferramentas estatísticas e computacionais mais comuns para a análise dos mesmos. São também discutidos alguns exemplos de aplicação de experimentos com microarrays em zootecnia e, numa última seção, são traçadas algumas considerações finais envolvendo os tópicos gerais abordados.
Read moreRobust GMM Tests for Structural Breaks
Robust GMM Tests for Structural Breaks