While regression discontinuity designs (RDDs) offer credible identification of treatment effects near cutoff points, their external validity remains inherently limited. To address this, a conditional independence assumption may be required. However, in multiple-score RDDs, this requires joint independence between all running variables and both potential outcomes and treatment assignments—a restriction often too strong to hold in practice. We relax this by assuming only mean independence between potential outcomes and the interaction of running variables, conditional on covariates. This weaker assumption allows us to link treatment effects at boundary and non-boundary points. We propose a non-parametric testing procedure and illustrate our method using simulation experiments and an empirical application in Colombian education policy, respectively.

Regression discontinuity (RD) designs are a popular approach to estimating a treatment effect of cutoff-based interventions. Two current estimation approaches dominate the literature. One fits separate regressions on either side of the cutoff, and the other performs finite sample inference based on a local randomization assumption. Recent developments of these approaches have often focused on asymptotic properties and large sample sizes. Educational applications often contain relatively small samples or sparsity near the cutoff, making estimation more difficult. As an alternative to the aforementioned approaches, we develop a partial linear estimator for RD designs. We show in simulations that our estimator outperforms certain leading estimators in several realistic, small-sample scenarios. We apply our estimator to school accountability scores in Indiana.

Understanding how effect heterogeneity across subgroups is mediated by multiple mediators is important yet under-studied. Despite the growing popularity of causal mediation analysis, existing methods rarely address the mediation of moderated treatment effects, particularly when multiple mediators are of interest. This study develops a causal inference approach for mediated moderation analysis with multiple mediators, decomposing a moderated treatment effect into mediated moderation effects and remaining moderation. We present causal estimands and extend a multiply robust estimator that can incorporate machine learning techniques to relax modeling assumptions. Simulations were conducted to evaluate the method’s performance. An empirical example about adolescent mental health illustrates the application. We hope this study provides a novel causal inference-based approach to understanding multiple mediating mechanisms underlying subgroup heterogeneity in treatment effects.

This study proposes an empirical Bayesian approach with loss functions for testing the fairness of an automated scoring algorithm. The proposed method outperforms traditional approaches by (1) being robust to small samples, (2) incorporating parameter uncertainty, and (3) balancing the loss of keeping versus flagging items. The impact of flagging potentially unfair items on the classification of examinees is investigated. The effectiveness of the proposed method is illustrated through simulations and an application to language assessment data.

Item response theory models are widely used and play an important role in psychological measurement and educational assessment. In its practical application, missing data is unavoidable, especially the not-reached and omitted items. For such missingness, if not handled properly, a biased parameter estimation would be derived, leading to an inaccurate estimation of examinees. To address this issue, this paper develops a joint model allowing for missing responses resulting from not-reached and omitted items. The proposed model simultaneously incorporates responses, response time, and omission time, using a survival analysis framework. Under the proposed model, the missing mechanism of not-reached items (NRIs) is interpreted by the right censoring of potential response times for examinees’ first NRIs. Furthermore, the missing mechanism of omitted items is explained by the competing relationship between response time and omission time. The simulation study confirms the consistent recovery of the proposed method. In addition, the comparison of the proposed method and alternative methods showed that only the method accounting for both missing patterns could recover model parameters well. An empirical study based on the Program for International Student Assessment 2015 Science Test was further conducted.

Determining sample size is crucial in research study design. The hierarchical structure of the data in cluster-randomized trials (CRTs) complicates this process, thereby necessitating the determination of the sample size at each level. Most methods for these trials are based on null hypothesis significance testing, which has numerous pitfalls. Using the Bayes factor may avoid these drawbacks, but existing methods are limited to trials without a multilevel structure. This study presents a method to determine the sample size for a one-period two-treatment parallel CRT using the Bayes factor. We introduce the implementation of this method in an R package. Simulation results show that the required sample size increases with decreasing effect sizes and with increasing intraclass correlation and Bayes factors.

Approaches to multilevel measurement modeling are often distinguished as aggregated or disaggregated. While past research has established the benefits and limitations of these modeling frameworks, none have critically evaluated the reliability of factor score predictions representing level-1 and level-2 processes. We utilize simulation methodology to explore factor-scoring techniques within both modeling frameworks. Results suggest that factor scores from the aggregated approach are at least as reliable as those from the disaggregated approach after simple numeric transformations, contingent on accurate knowledge of the level-2 factor structure in the presence of cross-level configural non-invariance. Further, given differences in the range of reliability across approaches when fewer clusters are available for analysis, the aggregated approach may be preferable within any given sample.

This study introduces a correction to the approximation of effective df as proposed by Satterthwaite, specifically addressing scenarios where component df are small. The correction is grounded in analytical results concerning the moments of standard normal random variables. This modification is applicable to complex variance estimates that involve both small and large df , offering an enhanced approximation of the higher moments required by Satterthwaite’s framework. Additionally, this correction extends and partially validates the empirically derived adjustment by Johnson and Rust, as it is based on theoretical foundations rather than simulations used to derive empirical transformation constants. Finally, the proposed adjustment also provides a correction to the estimate of the total variance in cases missing data have been replaced by multiple imputations such as in the case of plausible values in national and international large scale assessments.

Zhu et al. proposed a person-fit method based on the neural network called machine learning person-fit method (MLP-F) and found promising improvements over some traditional methods. MLP-F relies on constructing an appropriate neural network and uses mean square error as a loss function of the neural network. The primary focus of this study is to explore the potential improvement in classifications by replacing mean squared error with cross-entropy. Additionally, the application of MLP-F requires the establishment of a large number of output nodes when dealing with numerous attributes in an exam. However, an excess of nodes in the output layer may diminish classification accuracy and escalate the demand for training data. This article introduces a novel neural network architecture designed to be more versatile and robust. The findings from the research indicate that utilizing a cross-entropy loss function and the new neural network architecture enhances the performance of MLP-F. Simulation studies, considering various aberrant behaviors, demonstrate that MLP-F is effective in identifying aberrant behaviors and particularly excels in shorter tests, showcasing its potential significance in classroom testing.

Observed-score test equating is a vital part of every testing program, aiming to make test scores across test administrations comparable. Central to this process is the equating function, typically estimated by composing distribution functions of the scores to be equated. An integral part of this estimation is presmoothing, where statistical models are fit to observed score frequencies to mitigate sampling variability. This study evaluates the impact of commonly used model fit indices on bivariate presmoothing model-selection accuracy in both item response theory (IRT) and non-IRT settings. It also introduces a new model-selection criterion that directly targets the equating function in contrast to existing methods. The study focuses on the framework of non-equivalent groups with anchor test design, estimating bivariate score distributions based on real and simulated data. Results show that the choice of presmoothing model and model fit criterion influences the equated scores. In non-IRT contexts, a combination of the proposed model-selection criterion and the Bayesian information criterion exhibited superior performance, balancing bias, and variance of the equated scores. For IRT models, high selection accuracy and minimal equating error were achieved across all scenarios.