Accuracy in parameter estimation and simulation approaches for sample-size planning accounting for item effects

Focuses on open science practices in suicide research. Specifically, the authors use examples from their own and others' work to demonstrate the opportunities for future-proofing research by implementing open science practices, and they discuss some of the challenges and their potential solutions. They cover implementing open science practices in new, ongoing, and concluded studies, and discuss practices in order of being 'low' to 'high' threshold to implement (based on Kathawalla et al., 2021). Space constraints preclude an extensive presentation of all open science practices although they briefly describe the strategies of providing preprints, sharing study materials and codes, preregistration of studies and registered reports, and sharing research data. To highlight the open science work of as many researchers as possible, examples are provided in Electronic Supplementary Material 1 (ESM 1) rather than in the text. The authors ask for readers' help in adding to these examples via their 'living' reading list (https://osf.io/v6y3t/).

Methods for planning sample size (SS) for the standardized mean difference so that a narrow confidence interval (CI) can be obtained via the accuracy in parameter estimation (AIPE) approach are developed. One method plans SS so that the expected width of the CI is sufficiently narrow. A modification adjusts the SS so that the obtained CI is no wider than desired with some specified degree of certainty (e.g., 99% certain the 95% CI will be no wider than omega). The rationale of the AIPE approach to SS planning is given, as is a discussion of the analytic approach to CI formation for the population standardized mean difference. Tables with values of necessary SS are provided. The freely available Methods for the Behavioral, Educational, and Social Sciences (K. Kelley, 2006a) R (R Development Core Team, 2006) software package easily implements the methods discussed.

REX B. KLINE Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research Washington, DC: American Psychological Association, 2004, 336 pages (ISBN 1-59147-118-4, US$49.95 Hardcover) In 1999, a blue-ribbon task force assembled by the American Psychological Association published their findings with regards to the long-standing controversy pertaining to null hypothesis testing (NHST). The task force dictated effect sizes and confidence intervals be reported, and p values and dichotomous accept-reject decisions be given less weight. Editorial policies in a number of journals came to reflect the views of the task force as did a subsequent revision to the American Psychological Association Publication Manual. Rex B. Kline wrote Beyond Significance Testing. Reforming Data Analysis Methods in Behavioral Research as a follow-up to both the task force recommendations and the revision to the publication manual. Kline's 1998 book Principles and Practice of Structural Equation Modeling (Guilford Press) was well received and a second edition is being published this fall. In Beyond Significance Testing, Kline reviews the controversy regarding testing, offers methods for effect size and confidence interval estimation, and suggests some methodologies. There is an accompanying website that includes resources for instructors and students. Part I of the book is a review of fundamental concepts and the debate regarding testing. Part II provides statistics for effect size and confidence interval estimation for parametric and nonparametric two-group, oneway, and factorial designs. Part III examines metaanalysis, resampling, and Bayesian estimation procedures. In the first chapter, Kline provides a scholarly summary of the null hypothesis testing debate concluding with the APA task force findings and what Kline regards as ambiguous recommendations in the publication manual. Kline predicts the future will see a smaller role for traditional statistical testing (p values) in psychology. This change will take time and may not occur until the next generation of researchers are trained, but Kline anticipates the social sciences will then become more like the natural sciences in that we will report the directions and magnitudes of our effects, determine whether they replicate, and evaluate them for their theoretical, clinical, or practical significance (p. 15). Chapter 2 is a review of fundamental concepts of research design, including sampling and estimation, the logic of statistical testing, and t, F, and chi-square tests. The problems with statistical tests are revisited in Chapter 3. What follows is a long list of errors in interpretation of p values and conclusions made after null hypothesis testing. The emphasis on null hypothesis testing in psychology is also argued to inhibit advancement of the discipline. To be fair, Kline recognizes there is yet no magical alternative to statistical tests and that such tests are appropriate in some circumstances when applied correctly. Nonetheless, Kline envisions a future where effect sizes and confidence intervals are reported, substantive rather than statistical predominates, and NHST-Centric thinking has diminished. Part II covers effect size and confidence interval calculations. Chapter 4 is a presentation of parametric effect size indexes. Independent and dependent sample statistics are covered separately. The textbook's website has a supplementary chapter on twogroup multivariate designs. Group difference indexes such as d are distinguished from measures of association such as r. Case level analyses of group differences are also reviewed. Sections not relevant to a reader's needs can be skipped without loss of continuity. Interpretive guidelines for effect size magnitude and how one might be fooled by effect size estimation are sections that should not be passed over. …

Cognitive diagnostic models (CDMs) are increasingly being used in various assessment contexts to identify cognitive processes and provide tailored feedback. However, the most commonly used estimation method for CDMs, marginal maximum likelihood estimation with Expectation–Maximization (MMLE-EM), can present difficulties when sample sizes are small. This study compares the results of different estimation methods for CDMs under varying sample sizes using simulated and empirical data. The methods compared include MMLE-EM, Bayes modal, Markov chain Monte Carlo, a non-parametric method, and a parsimonious parametric model such as Restricted DINA. We varied the sample size, and assessed the bias in the estimation of item parameters, the precision in attribute classification, the bias in the reliability estimate, and computational cost. The findings suggest that alternative estimation methods are preferred over MMLE-EM under low sample-size conditions, whereas comparable results are obtained under large sample-size conditions. Practitioners should consider using alternative estimation methods when working with small samples to obtain more accurate estimates of CDM parameters. This study aims to maximize the potential of CDMs by providing guidance on the estimation of the parameters.

The geometric chart has proven more effective than Shewhart p or np charts to monitor the proportion nonconforming in high-quality processes. Implementing a geometric chart commonly requires the assumption that the in-control proportion nonconforming is known or accurately estimated. However, accurate parameter estimation is very difficult and may require a larger sample size than that available in practice in high-quality process where the proportion of nonconforming items is very small. Thus, the error in the parameter estimation increases and may lead to deterioration in the performance of the control chart if a sample size is inadequate. We suggest adjusting the control limits in order to improve the performance when a sample size is insufficient to estimate the parameter. We propose a linear function for the adjustment constant, which is a function of the sample size, the number of nonconforming items in a sample, and the false alarm rate. We also compare the performance of the geometric charts without and with adjustment using the expected value of the average run length (ARL) and the standard deviation of the ARL (SDARL).

We examine the accuracy and precision of parameter estimates for both the exponential and normal distributions when using only a collection of sample extremes. That is, we consider a collection of random variables, where each of the random variables is either the minimum or maximum of a sample of nj independent, identically distributed random variables drawn from a normal or exponential distribution with unknown parameters. Previous work derived estimators for the population parameters assuming the nj sample sizes are constant. Since sample sizes are often not constant in applications, we derive new unbiased estimators that take into account the varying sample sizes. We also perform simulations to assess how the previously derived estimators perform when the constant sample size is simply replaced with the average sample size. We explore how varying the mean, standard deviation, and probability distribution of the sample sizes affects the estimation error. Overall, our results demonstrate that using the average sample size in place of the constant sample size still results in reliable estimates for the population parameters, especially when the average sample size is large. Our estimation framework is applied to a biological example involving plant pollination.

PET analysis is conventionally performed as a two-stage process of quantification followed by analysis. We recently introduced SiMBA (Simultaneous Multifactor Bayesian Analysis), a hierarchical model that performs quantification and analysis for all brain regions of all individuals at once, and in so doing improves both the accuracy of parameter estimation and inferential efficiency. However until now, SiMBA has only been implemented for the two-tissue compartment model. We have now extended this general approach to also allow a non-invasive reference tissue implementation that includes both the full reference tissue model (FRTM) and the simplified reference tissue model (SRTM). Although the FRTM did not successfully converge and gave rise to unlikely parameter estimates for this application, the SRTM described the data well. In simulated data, SiMBA improves quantitative parameter estimation accuracy, reducing error by, on average, 57% for binding potential () using SRTM. In considerations of statistical power, our simulation studies indicate that the efficiency of SiMBA modeling approximately corresponds to improvements that would require doubling the sample size if using conventional methods, with no increase in the false-positive rate. We applied the model to PET data measured with [11C]AZ10419369, which binds selectively to the serotonin 1B receptor, in datasets collected at three different PET centres (n = 139, n = 44 and n = 39). We show that SiMBA yields replicable inferences by comparing associations between PET parameters and age in the different datasets. Moreover, we show that time activity curve data from different centres can be combined in a single SiMBA model using covariates to control between-centre parameter differences, in order to harmonise data between centres. In summary, we present a novel approach for non-invasive quantification and analysis of PET time activity curve data which improves quantification and inferences, enables effective between-centre data harmonisation, and also yields replicable outcomes. This method has the potential to significantly expand the range of research questions which can be meaningfully tested using conventional sample sizes with PET imaging.

Multilevel analysis is widely used for hierarchical data (e.g., individuals within clusters), allowing researchers to examine how individual- and cluster-level (i.e., level 1 and level 2) attributes relate to individual outcomes. A well-known issue concerns conflated fixed slopes, which arise when a level 1 predictor with systematic variance across clusters is modeled with a single fixed effect at the individual level, conflating its within- and between-cluster slopes. Less recognized, however, is the problem of conflated random slopes that occurs when the sources of heterogeneity introduced by the within- and between-cluster components of a level 1 predictor are not separated (i.e., the fixed effects of the predictor are separated across levels, but not their random effects). A conflated random slope is a blend of slope heterogeneity and intercept heteroscedasticity, producing misleading conclusions about between-cluster differences. Proper parameterization of the random slopes is necessary to avoid this issue, either using cluster-mean-centering (CMC) or constant-centering (CC). Despite its importance, the practical consequences of conflated random slopes remain rarely discussed in psychology and educational research. This study addresses that gap using a Monte Carlo simulation comparing three models that estimate unconflated random slopes and one widely used model that estimates conflated random slopes. The models differed in their parameterization of the random slope for a level 1 predictor: The within-only model had a random slope for the CMC level 1 predictor only; the within–between model had random slopes for both the CMC level 1 predictor and the CC level 2 predictor; the within–contextual model had random slopes for the CC level 1 predictor and the CC level 2 predictor; and the random-conflated model had a random slope for the CC level 1 predictor only. The simulation factors were level 1 sample size (5 and 30), level 2 sample size (25 and 50), random slope reliability (0.20 and 0.60), and the ratio of intercept heteroscedasticity to slope heterogeneity (0.5, 1.0, 1.5). After examining model convergence, model performance was assessed in terms of bias in fixed and random effects, power to detect random slopes, and accuracy of standard errors. Results showed clear trade-offs. The random-conflated model had very high convergence (> .97) even in small samples, but it consistently underestimated level 1 slope variance when within- and between-cluster variances differed, yielding downwardly biased variance estimates. The within-only model correctly specified the random slope but imposed homogeneity in intercept variance across cluster means, leading to slightly biased SEs for level 2 fixed effects. By contrast, the within–between and within–contextual models—theoretically correct specifications—had substantially lower convergence rates (.38–.73) in small samples but generally outperformed the simpler alternatives in parameter estimation and SE accuracy in larger sample sizes. Regarding power, all four models showed high capability to detect non-zero level 1 random slope variance. However, power to detect level 2 random slope variance in the within–between and within–contextual models was modest (.39 and .58, respectively), especially when sample sizes at level 1 and level 2 were small. This highlights the tension in practice between model complexity and the available information for the model from the data. Overall, our results show that estimating conflated random slopes distorts both fixed and random effect estimates, threatening valid inferences about between-cluster differences. When sample sizes permit, researchers may specify the within–between model, particularly if intercept heteroscedasticity matters. When data are limited or intercept heteroscedasticity is not of interest, the within-only model is a reasonable alternative to avoid random conflation, though with the trade-off of slightly biased SEs for level 2 fixed effects.

Aguado y García, (2020). Apuntes desde la perspectiva de género a la historia de la Psicología de México. Revista Electrónica De Psicología Iztacala, 23(4). Apud, I. y Romaní, O. (2016). La encrucijada de la adicción. Distintos modelos en el estudio de la drogodependencia. Salud y drogas, 16(2), 115-125. https://doi.org/10.21134/haaj.v16i2.267 Arostegui Santamaría, E. y Urbano Aljama, A. (2004). La mujer drogodependiente: especificidad de género y factores asociados. Universidad de Deusto; Deustuko Unibertsitatea. Arteta, D. (2020). Violencia de género y adicciones en las mujeres. Cendoc Bogani (Ajuntament de València). Barberá Heredia, E. y Cala Carrillo, M. J. (2008). Desarrollo histórico en la investigación psicológica del enfoque de género. Revista de Historia de la Psicología, 29(3/4), 25-33. Becoña Iglesias, E. y Oblitas Guadalupe, L. A. (2003). Psicología de la salud y adicciones: perspectiva terapéutica. Revista de Psicología de la PUCP, 21(1), 72-106. https://doi.org/10.18800/psico.200301.003 Beorlegui, D. (2019). Historia oral e historias de vida: subjetividad, memoria y feminismo(s) en Universidad del País Vasco (Ed.), Otras formas de (des)aprender. Investigación feminista en tiempos de violencia, resistencias y decolonialidad (pp. 129-146). Bohan, J. S. (1995). Re-placing women in psychology: readings toward a more inclusive history. Kendall Hunt Publishing. Bordería Mondejar, C. (1997). Subjetividad y cambio social en las historias de vida de mujeres: notas sobre el método biográfico. Arenal: Revista de historia de las mujeres, 4(2), 177-195. Cano, L. y Mayor, L. (1998). La intervención psicológica en el marco de los Servicios Sociales: el caso de las drogodependencias. Informació Psicològica, 66, 10-18. Cano, L. (2006). Adicciones y salud. Estado actual de la investigación en España [Tesis doctoral, Universitat de València]. Castañeda, M. (2008). Metodología de la investigación feminista. CEIIHC, UNAM. Ferrer Pérez, V. y Bosch Fiol, E. (2012). Del gènere de la Psicologia social a la Psicologia social de gènere: Un segle de trobades i desavinences. Anuari de Psicologia de la Societat Valenciana de Psicologia, 14(1), 81-93. García Colmenares, C. (2011). Las primeras psicólogas españolas. Itinerarios vitales y profesionales. Universidad de Granada. García Dauder, S. (2003). Fertilización cruzada entre la psicología social de la ciencia y los estudios feministas de la ciencia. Revista Athenea Digital, 4, 109-150. García Dauder, S. (2010). El olvido de las mujeres pioneras en la historia de la Psicología. Revista de Historia de la Psicología, 31(4), 9-2. García Dauder, S. (2019). La perspectiva de género en psicología. FOCAD: Consejo General de la Psicología de España. Giménez Segura, M. C. (2007). Las mujeres en la historia de la Psicología. Revista de Historia de la Psicología, 28(2/3), 281-290. Gómez Moya, J. (2006). El alcoholismo femenino, una verdad oculta. Trastornos adictivos, 8(4), 251-260. https://doi.org/10.1016/S1575-0973(06)75131-5 González Sala, F., Osca-Lluch, J. y Ferragut Domingo, C. (2022). Análisis de la producción y colaboración interinstitucional en la psicología española desde una perspectiva de género entre 1970 y 1989. Revista de Historia de la Psicología, 43(1), 2-10. https://doi.org/10.5093/rhp2022a1 Gruber, J., Mendel, J., Lindquist, K. A., Schmader, T., Clark, L. A., Bliss-Moreau, E., Akinola, M., Atlas, L., Barch, D. M., Barrett, L. F., Borelli, J. L., Brannon, T. N., Bunge, S.A., Campos B., Cantlon, J., Carter, R., Carter-Sowellm A. R., Chen, S., Craske, M. G.,…Williams, L. A. (2021). The Future of Women in Psychological Science. Perspectives on Psychological Science, 16(3), 483-516. https://doi.org/10.1177/1745691620952789 Harding, S. (1987). ¿Existe un método feminista? (trad.). Is There a Feminist Method en Sandra Harding (ed.). Feminism and Methodology, Bloomington/Indianapolis. Indiana University Press. Hernández Micó, J.J. (2001). Modelo de intervención en drogodependencias en la Comunidad Valenciana: organización, competencias y ordenación de los recursos. Salud y Drogas, 1(1), 97-110. https://doi.org/10.21134/haaj.v1i1.183 Magallón Portolés, C. (1998). Pioneras españolas en las ciencias. Las mujeres del instituto nacional de física y química. Consejo Superior de investigación Científica. Magaña, I., Wolff, X. y Winkler, M.I. (1998). Historia de la psicología y género: percepción de los estudiantes de psicología, Revista chilena de psicología, 19(2). Meneses Falcón, C. (2001). Mujer y heroína: un estudio antropológico de la heroinomanía femenina [Tesis doctoral, Universidad de Granada]. Ortiz García, P. y Clavero Mira, E. (2014). Estilos de consumo de sustancias adictivas en función del género. Una aproximación desde el análisis del discurso. Acta Sociológica, 63, 121-144. https://doi.org/10.1016/S0186-6028(14)70485-2 Pujadas, J. J. (2000). El método biográfico y los géneros de la memoria. Revista de Antropología Social, 9, 127-158. Quintero, M.L., Velázquez, E.B y Padilla, S. (2018). Adicciones y género en la sociedad moderna. Miguel Ángel Porrúa. Romo Avilés, N. (2010). La mirada de género en el abordaje de los usos y abusos de drogas. Revista española de drogodependencias, 35(3), 269-272. Russo, N. y Denmark, F. L. (1987). Contributions of women to psychology. Annual Review of Psychology, 38, 279-298. https://doi.org/10.1146/annurev.ps.38.020187.001431 Sos Peña, R. (2018). La presència de dones pioneres europees en el camp psicològic. Anuari de Psicologia, 13-33. Taylor, S. y Bogdan, R. (1987). Introducción a los métodos cualitativos de investigación. Paidós. Valls-Llobet, C. (2016). La ciència de la diferència. La invisibilitat de les dones en les ciències de la salut. Mètode Science Studies Journal, (91), 73- 77. Valls-Llobet, C. (2020). Mujeres invisibles para la medicina. Desvelando nuestra salud. Capitan Swing. Winkler, M.I., Magaña, I. y Wolff, X. (2001). Mujeres en la historia de la psicología: autorías y paradojas. Revista Latinoamericana de Psicología, 33(1), 23-37.

Some Thoughts About Data Type, Distribution, and Statistical Significance

Considerations of sample size in medical research

In this article, we consider the probability of improved accuracy in parameter estimation due to increases in sample size. For a range of scenarios we quantify this probability and develop a simple sample size formula to guarantee a certain level of improvement with a given probability. Our results lead to insights into sample size requirements and how they are related to the parameters governing the sampling process.

PET analysis is conventionally performed as a two-stage process of quantification followed by analysis. We recently introduced SiMBA (Simultaneous Multifactor Bayesian Analysis), a hierarchical model that performs quantification and analysis for all brain regions of all individuals at once, and in so doing improves both the accuracy of parameter estimation as well as inferential efficiency. However until now, SiMBA has only been implemented for the two-tissue compartment model. We have now extended this general approach to also allow a non-invasive reference tissue implementation that includes both the full reference tissue model and the simplified reference tissue model. In simulated data, SiMBA improves quantitative parameter estimation accuracy, reducing error by, on average, 57% for binding potential (). In considerations of statistical power, our simulation studies indicate that the efficiency of SiMBA modeling approximately corresponds to improvements that would require doubling the sample size if using conventional methods, with no increase in the false positive rate. We applied the model to PET data measured with [11C]AZ10419369, which binds selectively to the serotonin 1B receptor, in datasets collected at three different PET centres (n=139, n=44 and n=39). We show that SiMBA yields replicable inferences by comparing associations between PET parameters and age in the different datasets. Moreover, we show that time activity curve data from different centres can be combined in a single SiMBA model using covariates to control between-centre parameter differences, in order to harmonise data between centres. In summary, we present a novel approach for noninvasive quantification and analysis of PET time activity curve data which improves quantification and inferences, enables effective between-centre data harmonisation, and also yields replicable outcomes. This method has the potential to significantly expand the range of research questions which can be meaningfully tested using conventional sample sizes with PET imaging.

The interplay of the following factors: population design (PDN), the cost function in terms of maximum cost (Max. C) (i.e., maximum number of samples/sample size), sample size, and intersubject variability [restricted (30%) to moderate (60%)] on the estimation of pharmacokinetic parameters from population pharmacokinetic data sets obtained using mixed designs was investigated in a simulation study. A two compartment model with multiple bolus intravenous inputs was assumed, and the residual variability was set at 15%. The sample size (N) investigated ranged from 30 to 200 with the associated cost function varying accordingly with the five individual and sixteen population designs studied. Accurate and precise estimates of structural model parameters were obtained for N > or = 50 (Max. C > or = 150) irrespective of the intersubject variability (ITV) and PDN investigated. When ITV was 30%, all structural model parameters were well estimated irrespective of the PDN. Robust estimates of clearance and its variability were obtained for all N at all levels of ITV with Max. C > or = 90 (PDN > or = 4). Imprecise estimates of ITV in V1, V2, and Q were obtained at 60% ITV irrespective of N, PDN, or Max. C. Positive bias was associated with the estimation of variability in V1, V2, and Q with PDN < or = 4 (Max. C < or = 150). This was due in part to a greater proportion of subjects sampled only once. Correspondingly, residual variability was underestimated. It is of utmost importance to avoid this artifact by ensuring that at least a moderate subset of subjects contributing data to a population pharmacokinetic study contribute data more than once. Given a sample size and ITV, the cost function must be considered in designing a population pharmacokinetic study using mixed designs.

The control chart based on the geometric distribution (geometric chart) has been shown to be competitive with p- or np-charts for monitoring the proportion nonconforming, especially for applications in high quality manufacturing environments. However, implementing a geometric chart is often based on the assumption that the in-control proportion nonconforming is known or accurately estimated. For a high quality process, an accurate parameter estimate may require a very large sample size that is seldom available. In this paper we investigate the sample size effect when the proportion nonconforming is estimated. An analytical approximation is derived to compute shift detection probabilities and run length distributions. It is found that the effect on the alarm probability can be significant even with sample sizes as large as 10,000. However, the average run length is only affected mildly unless the sample size is small and there is a large process improvement. In practice, the quantitative results of the paper can be used to determine the minimum number of items required for estimating the control limits of a geometric chart so that certain average run length requirements are met.