Autocorrelation and estimates of treatment effect size for single‐case experimental design data

REX B. KLINE Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research Washington, DC: American Psychological Association, 2004, 336 pages (ISBN 1-59147-118-4, US$49.95 Hardcover) In 1999, a blue-ribbon task force assembled by the American Psychological Association published their findings with regards to the long-standing controversy pertaining to null hypothesis testing (NHST). The task force dictated effect sizes and confidence intervals be reported, and p values and dichotomous accept-reject decisions be given less weight. Editorial policies in a number of journals came to reflect the views of the task force as did a subsequent revision to the American Psychological Association Publication Manual. Rex B. Kline wrote Beyond Significance Testing. Reforming Data Analysis Methods in Behavioral Research as a follow-up to both the task force recommendations and the revision to the publication manual. Kline's 1998 book Principles and Practice of Structural Equation Modeling (Guilford Press) was well received and a second edition is being published this fall. In Beyond Significance Testing, Kline reviews the controversy regarding testing, offers methods for effect size and confidence interval estimation, and suggests some methodologies. There is an accompanying website that includes resources for instructors and students. Part I of the book is a review of fundamental concepts and the debate regarding testing. Part II provides statistics for effect size and confidence interval estimation for parametric and nonparametric two-group, oneway, and factorial designs. Part III examines metaanalysis, resampling, and Bayesian estimation procedures. In the first chapter, Kline provides a scholarly summary of the null hypothesis testing debate concluding with the APA task force findings and what Kline regards as ambiguous recommendations in the publication manual. Kline predicts the future will see a smaller role for traditional statistical testing (p values) in psychology. This change will take time and may not occur until the next generation of researchers are trained, but Kline anticipates the social sciences will then become more like the natural sciences in that we will report the directions and magnitudes of our effects, determine whether they replicate, and evaluate them for their theoretical, clinical, or practical significance (p. 15). Chapter 2 is a review of fundamental concepts of research design, including sampling and estimation, the logic of statistical testing, and t, F, and chi-square tests. The problems with statistical tests are revisited in Chapter 3. What follows is a long list of errors in interpretation of p values and conclusions made after null hypothesis testing. The emphasis on null hypothesis testing in psychology is also argued to inhibit advancement of the discipline. To be fair, Kline recognizes there is yet no magical alternative to statistical tests and that such tests are appropriate in some circumstances when applied correctly. Nonetheless, Kline envisions a future where effect sizes and confidence intervals are reported, substantive rather than statistical predominates, and NHST-Centric thinking has diminished. Part II covers effect size and confidence interval calculations. Chapter 4 is a presentation of parametric effect size indexes. Independent and dependent sample statistics are covered separately. The textbook's website has a supplementary chapter on twogroup multivariate designs. Group difference indexes such as d are distinguished from measures of association such as r. Case level analyses of group differences are also reviewed. Sections not relevant to a reader's needs can be skipped without loss of continuity. Interpretive guidelines for effect size magnitude and how one might be fooled by effect size estimation are sections that should not be passed over. …

Quantitative synthesis across studies requires consistent measures of effect size among studies. In community ecology, these measures of effect size will often be some measure of the strength of interactions between taxa. However, indices of interaction strength vary greatly among both theoretical and empirical studies, and the connection between hypotheses about interaction strength and the metrics that are used to test these hypotheses are often not explicit. We describe criteria for choosing appropriate metrics and methods for comparing them among studies at three stages of designing a meta-analysis to test hypotheses about variation in interaction intensity: (1) the choice of response variable; (2) how effect size is calculated using the response in two treatments; and (3) whether there is a consistent quantitative effect across all taxa and systems studied or only qualitatively similar effects within each taxon–system combination. The consequences of different choices at each of these stages are illustrated with a meta-analysis to examine the relationship between competition/facilitation intensity and productivity in plants. The analysis used a database of 296 cases in 14 studies. The results were unexpected and largely inconsistent with existing theory: competition intensity often significantly declined (rather than increased) with productivity, and facilitation was sometimes restricted to more productive (rather than less productive) sites. However, there was considerable variation in the pattern among response variables and measures of effect size. For example, on average, competitive effects on final biomass and survival decreased with standing crop, but competitive effects on growth rate did not. On the other hand, facilitative interactions were more common at low standing crop for final biomass and growth rate, but more common at high standing crop for survival. Results were more likely to be significant using the log response ratio (ln[removal/control]) as the effect size than using the relative competition intensity ([removal − control]/removal), although the trends for these conceptually similar indices did not differ. When all studies were grouped in a single meta-regression of interaction intensity on standing crop to test quantitative similarity among studies, survival showed the clearest negative relationship. However, when the same regressions were done for each unique combination of taxon and site within each study to test for qualitative similarity among studies, the slopes averaged over studies tended to be negative for biomass and growth rate, but not different from zero for survival. These results are subject to a number of caveats because of the limitations of the available data—most notably, the extension of effects of interactions on individual growth or survival to effects on population distribution and abundance or community structure is highly problematic. Nevertheless, the fact that none of the meta-analyses demonstrated a significant positive relationship between competition and standing crop but that we frequently found negative relationships is an important pattern that has not been apparent from qualitative surveys of individual studies, and it demonstrates the potential power of meta-analysis in ecology. We conclude with recommendations to overcome some of the limitations of the currently available data and meta-analytical procedures.

Many empirical studies estimating effective population size apply the temporal method that provides an estimate of the variance effective size through the amount of temporal allele frequency change under the assumption that the study population is completely isolated. This assumption is frequently violated, and the magnitude of the resulting bias is generally unknown. We studied how gene flow affects estimates of effective size obtained by the temporal method when sampling from a population system and provide analytical expressions for the expected estimate under an island model of migration. We show that the temporal method tends to systematically underestimate both local and global effective size when populations are connected by gene flow, and the bias is sometimes dramatic. The problem is particularly likely to occur when sampling from a subdivided population where high levels of gene flow obscure identification of subpopulation boundaries. In such situations, sampling in a manner that prevents biased estimates can be difficult. This phenomenon might partially explain the frequently reported unexpectedly low effective population sizes of marine populations that have raised concern regarding the genetic vulnerability of even exceptionally large populations.

Effect size measures were computed for a sample of 100 research studies published in four leading rehabilitation journals. The effect size measures were categorized as small, medium, or large according to established criteria. Analysis of the effect size measures revealed that the average treatment effect was in the medium range (0.69). Further data analysis exposed a statistically significant difference in mean effect size values across articles contained in the four rehabilitation journals. A regression analysis indicated that characteristics of the study design were not significant predictors of effect size values in the 100 studies. The interpretation of effect size measures in relation to statistical significance testing is discussed and the argument made that reporting measures of effect size will assist researchers in determining the sensitivity of experimental manipulations in rehabilitation research.

Small dams represent one of the most widespread human influences on riverscapes. Greater understanding of how these structures affect aquatic organisms is needed to ensure that decisions regarding their construction and removal strike an appropriate balance between components of human and ecosystem services. Within the basin of the Laurentian Great Lakes, the effects that in-stream barriers (dams) used to control the non-native, parasitic sea lamprey (Petromyzon marinus) on the diversity of non-target fishes is a significant concern for fishery managers. A previous study indicated that upstream changes in the species richness of non-target fishes observed in 24 streams with a sea lamprey barrier relative to paired reference streams (a measure of effect size) was variable across the basin. We examined the degree to which the variance in effect size could be attributed to imprecision in the field sampling protocol used to estimate effect sizes, differences in catchment-scale landscape attributes between barrier and reference streams within pairs, and differences in landscape attributes at different spatial scales among barrier streams. Simulation modeling and analyses of repeated field measurements made for a subset of streams demonstrated that a large variance in effect size is expected for the field sampling design and that estimates of effect size measured for individual barrier streams are imprecise. Regression models and multimodel inference methods based on Akaike's Information Criterion provided less support for hypotheses linking effect size to landscape attributes. Mean effect size, adjusted for the influences of landscape characteristics within and across stream pairs, provides the most reliable and least biased estimate of the effect of sea lamprey barriers on the richness of nontarget fish species. With the information currently available, landscape characteristics of catchments cannot be used to help decision makers anticipate effects sizes for candidate streams being considered for future barrier construction. Our findings will help fishery managers in the Laurentian Great Lakes make more informed decisions regarding the use and placement of sea lamprey barriers and achieve their objective of delivering an integrated pest management plan for sea lamprey control that is environmentally and economically sound and socially acceptable.

INTRODUCTION: Preeclampsia (PE)is a pregnancy-associated, multi-organ, life-threatening disease that appears after the 20th week of gestation. The aim of this study was to perform a systematic review and meta-analysis to determine whether women with PE have disrupted miRNA expression compared to women who do not have PE. METHODS: We conducted a systematic review and meta-analysis of studies that reported miRNAs expression levels in placenta or peripheral blood of pregnant women with vs. without PE. Studies published before October 29, 2021 were identified through PubMed, EMBASE and Web of Science. Two reviewers used predefined forms and protocols to evaluate independently the eligibility of studies based on titles and abstracts and to perform full-text screening, data abstraction and quality assessment. Standardized mean difference (SMD) was used as a measure of effect size. RESULTS: 229 publications were included in the systematic review and 53 in the meta-analysis. The expression levels in placenta were significantly higher in women with PE compared to women without PE for miRNA-16 (SMD=1.51,95%CI=0.55-2.46), miRNA-20b (SMD=0.89, 95%CI=0.33-1.45), miRNA-23a (SMD=2.02, 95%CI=1.25-2.78),miRNA-29b(SMD=1.37, 95%CI=0.36-2.37), miRNA-155 (SMD=2.99, 95%CI=0.83-5.14) and miRNA-210(SMD=1.62, 95%CI=0.78-2.46), and significantly lower for miRNA-376c (SMD=-4.86, 95%CI=-9.51 to -0.20).An increased level of miRNK-155expression was found in peripheral blood of women with PE (SMD=2.06, 95%CI=0.35-3.76), while the expression level of miRNA-16 was significantly lower in peripheral blood of PE women (SMD=-0.47, 95%CI=-0.91 to -0.03).The functional roles of the presented miRNAs include control of trophoblast proliferation, migration, invasion, apoptosis, differentiation, cellular metabolism and angiogenesis. CONCLUSION: miRNAs play an important role in the pathophysiology of PE. The identification of differentially expressed miRNAs in maternal blood creates an opportunity to define an easily accessible biomarker of PE.

A meta-analysis of the analgesic effects of omega-3 polyunsaturated fatty acid supplementation for inflammatory joint pain

In this published article, 1 our reporting of longitudinal BP ND results in the caudate, putamen and globus include incorrect data from a participant with early premanifest HD. Due to an unusually steep drop in BP ND values in all three regions, this participant was originally excluded from our estimations of effect and sample sizes as an outlier. The corrected data show that the participant meets criteria for inclusion, and his/her data has now been incorporated into our effect and sample size analyses.

Null hypothesis significance testing (NHST) is the dominant statistical approach in biology, although it has many, frequently unappreciated, problems. Most importantly, NHST does not provide us with two crucial pieces of information: (1) the magnitude of an effect of interest, and (2) the precision of the estimate of the magnitude of that effect. All biologists should be ultimately interested in biological importance, which may be assessed using the magnitude of an effect, but not its statistical significance. Therefore, we advocate presentation of measures of the magnitude of effects (i.e. effect size statistics) and their confidence intervals (CIs) in all biological journals. Combined use of an effect size and its CIs enables one to assess the relationships within data more effectively than the use of p values, regardless of statistical significance. In addition, routine presentation of effect sizes will encourage researchers to view their results in the context of previous research and facilitate the incorporation of results into future meta-analysis, which has been increasingly used as the standard method of quantitative review in biology. In this article, we extensively discuss two dimensionless (and thus standardised) classes of effect size statistics: d statistics (standardised mean difference) and r statistics (correlation coefficient), because these can be calculated from almost all study designs and also because their calculations are essential for meta-analysis. However, our focus on these standardised effect size statistics does not mean unstandardised effect size statistics (e.g. mean difference and regression coefficient) are less important. We provide potential solutions for four main technical problems researchers may encounter when calculating effect size and CIs: (1) when covariates exist, (2) when bias in estimating effect size is possible, (3) when data have non-normal error structure and/or variances, and (4) when data are non-independent. Although interpretations of effect sizes are often difficult, we provide some pointers to help researchers. This paper serves both as a beginner's instruction manual and a stimulus for changing statistical practice for the better in the biological sciences.

Background and AimsNon-alcoholic fatty liver disease (NAFLD) is considered a part of the metabolic syndrome. Two drugs, saroglitazar (S) and pioglitazone (P) are known to have beneficial effects on the...

Due to practical difficulties in obtaining direct genetic estimates of effective sizes, conservation biologists have to rely on so-called 'demographic models' which combine life-history and mating-system parameters with F-statistics in order to produce indirect estimates of effective sizes. However, for the same practical reasons that prevent direct genetic estimates, the accuracy of demographic models is difficult to evaluate. Here we use individual-based, genetically explicit computer simulations in order to investigate the accuracy of two such demographic models aimed at investigating the hierarchical structure of populations. We show that, by and large, these models provide good estimates under a wide range of mating systems and dispersal patterns. However, one of the models should be avoided whenever the focal species' breeding system approaches monogamy with no sex bias in dispersal or when a substructure within social groups is suspected because effective sizes may then be strongly overestimated. The timing during the life cycle at which F-statistics are evaluated is also of crucial importance and attention should be paid to it when designing field sampling since different demographic models assume different timings. Our study shows that individual-based, genetically explicit models provide a promising way of evaluating the accuracy of demographic models of effective size and delineate their field of applicability.

It is good scientific practice to the report an appropriate estimate of effect size and a confidence interval (CI) to indicate the precision with which a population effect was estimated. For comparisons of 2 independent groups, a probability-based effect size estimator (A) that is equal to the area under a receiver operating characteristic curve and closely related to the popular Wilcoxon-Mann-Whitney nonparametric statistical tests has many appealing properties (e.g., easy to understand, robust to violations of parametric assumptions, insensitive to outliers). We performed a simulation study to compare 9 analytic and 3 empirical (bootstrap) methods for constructing a CI for A that can yield very different CIs for the same data. The experimental design crossed 6 factors to yield a total of 324 cells representing challenging but realistic data conditions. Results were examined using several criteria, with emphasis placed on the extent to which observed CI coverage probabilities approximated nominal levels. Based on the simulation study results, the bias-corrected and accelerated bootstrap method is recommended for constructing a CI for the A statistic; bootstrap methods also provided the least biased and most accurate standard error of A. An empirical illustration examining score differences on a citation-based index of scholarly impact across faculty at low-ranked versus high-ranked research universities underscores the importance of choosing an appropriate CI method.

To investigate the use of an enriched study design on the estimates of treatment effect in analgesic trials for chronic musculoskeletal pain. Database searches were conducted from 2004 to 2014. We included randomized placebo-controlled trials evaluating pain medications for chronic musculoskeletal pain. Methodological quality was assessed using the PEDro scale. The estimates of treatment effect on pain and adverse events were compared between enriched and nonenriched designs. Metaregression was used to assess the association between the effect size estimate and the study design controlling for analgesic dose and methodological quality. We included 108 trials, of which 99 were included in the meta-analysis (n=44 171). There were no overall differences in effect sizes between enriched and nonenriched designs for pain intensity. There was a significant difference for a reduction in any adverse events favouring enriched designs for opioids, but not for other analgesics or the outcome serious adverse events. There was an association between effect size and methodological quality, with failure to blind the outcome assessor and failure to use intention-to-treat analysis being associated with larger effect sizes. There is no evidence that the use of an enriched study design changes the treatment effect size estimate for pain. There is some evidence that clinical trials that employ enriched designs report a reduced risk of adverse events in trials for chronic musculoskeletal pain, but it is unclear whether enriched designs influence estimates of serious adverse events. Features of trial design and study quality were associated with treatment effect estimates.

In this note, we recollect some formulas and facts about linear mixed models in relation to Cohen’s effect size measure $$f^2$$ f 2 . It is shown how an estimate of the variance-covariance matrix for the estimated fixed effects parameter vector may serve to compute $$f^2$$ f 2 in the presence of random effects. The advantage of the purported approach lies in the fact that any variance-covariance estimate already available may be applied. This also illuminates the circumstance that an actual computed effect size value necessarily depends on the employed linear mixed model estimation procedures. To demonstrate possible applications, it is shown how $$f^2$$ f 2 can be computed with the statistical software environment using . Based on an artificially generated data set, the possible impact of the inclusion of fixed and/or random effects is exemplarily discussed. In addition, the application to several random variables is demonstrated with a publicly available dat set.

It is increasingly common for group studies in neuropsychology to report effect sizes. In contrast this is rarely done in single-case studies (at least in those studies that employ a case-controls design). The present paper sets out the advantages of reporting effect sizes, derives suitable effect size indexes for use in single-case studies, and develops methods of supplementing point estimates of effect sizes with interval estimates. Computer programs that implement existing classical and Bayesian inferential methods for the single case (as developed by Crawford, Garthwaite, Howell, and colleagues) are upgraded to provide these point and interval estimates. The upgraded programs can be downloaded from www.abdn.ac.uk/~psy086/dept/Single_Case_Effect_Sizes.htm