Global Embedded Technologies (United States)

In large-scale educational assessment data consistent with a simple-structure multidimensional item response theory (MIRT) model, where every item measures only one latent variable, separate unidimensional item response theory (UIRT) models for each latent variable are often calibrated for practical reasons. While this approach can be valid for data from a linear test, unacceptable item parameter estimates are obtained when data arise from a multistage test (MST). We explore this situation from a missing data perspective and show mathematically that MST data will be problematic for calibrating multiple UIRT models but not MIRT models. This occurs because some items that were used in the routing decision are excluded from the separate UIRT models, due to measuring a different latent variable. Both simulated and real data from the National Assessment of Educational Progress are used to further confirm and explore the unacceptable item parameter estimates. The theoretical and empirical results confirm that only MIRT models are valid for item calibration of multidimensional MST data.

In certain educational assessments, there are many administrations of test forms of the same assessment over a specific period. The issue of equating these test forms from long series of test administrations is complicated, because the statistical properties of the items and the student populations can be volatile. This study demonstrates the possibilities of time series modeling for monitoring item and population characteristics in the context of test equating. More specifically, seasonal effects, trends, sudden breaks (or jumps), and outliers in population means and item parameters are studied in a series of simulations by making use of the frameworks of item response theory (IRT) and state space modeling. Three different state space models are used: the local level model, the linear trend model, and a seasonal model with linear trend. The goal is to capture peculiarities in the data in real time, so that, if necessary, immediate action can be taken. Preliminary results of the simulations indicate that many of the effects, as well as combinations, are well captured with the developed approach.