Kernel equating uses kernel smoothing techniques to continuize the discrete score distributions when equating test scores from an assessment test. The degree of smoothness of the continuous approximations is determined by the bandwidth. Four bandwidth selection methods are currently available for kernel equating, but no thorough comparison has been made between these methods. The overall aim is to compare these four methods together with two additional methods based on cross-validation in a simulation study. Both equivalent and non-equivalent group designs are used and the number of test takers, test length, and score distributions are all varied. The results show that sample size and test length are important factors for equating accuracy and precision. However, all bandwidth selection methods perform similarly with regards to the mean squared error and the differences in terms of equated scores are small, suggesting that the choice of bandwidth is not critical. The different bandwidth selection methods are also illustrated using real testing data from a college admissions test. Practical implications of the results from the simulation study and the empirical study are discussed.

Pseudo-guessing parameters are present in item response theory applications for many educational assessments. When sample size is not sufficiently large, the guessing parameters may be ignored from the analysis. This study examines the impact of ignoring pseudo-guessing parameters on measurement invariance analysis, specifically, on item difficulty, item discrimination, and mean and variance of ability distribution. Results show that when non-zero guessing parameters are ignored from the measurement invariance analysis, item discrimination estimates tend to decrease particularly for more difficult items, and item difficulty estimates decrease unless the items are highly discriminating and difficult. As the guessing parameter increases, the size of the decrease in item discrimination and difficulty tends to increase, and the estimated mean and variance of ability distribution tend to be inaccurate. When two groups have heterogeneous ability distributions, ignoring the guessing parameter affects the reference group and the focal group differently. Implications of result findings are discussed.

The calibration of the one-parameter logistic ability-based guessing (1PL-AG) model in item response theory (IRT) with a modest sample size remains a challenge for its implausible estimates and difficulty in obtaining standard errors of estimates. This article proposes an alternative Bayesian modal estimation (BME) method, the Bayesian Expectation-Maximization-Maximization (BEMM) method, which is developed by combining an augmented variable formulation of the 1PL-AG model and a mixture model conceptualization of the three-parameter logistic model (3PLM). By comparing with marginal maximum likelihood estimation (MMLE) and Markov Chain Monte Carlo (MCMC) in JAGS, the simulation shows that BEMM can produce stable and accurate estimates in the modest sample size. A real data example and the MATLAB codes of BEMM are also provided.

Most computerized adaptive testing (CAT) programs do not allow item review due to a decrease in estimation precision and aberrant manipulation strategies. In this article, a block item pocket (BIP) method that combines the item pocket method with the successive block method to realize reviewable CAT was proposed. A worst-case but still reasonable answering strategy and the Wainer-like manipulation strategy were simulated to evaluate the estimation precision of reviewable unidimensional computerized adaptive testing (UCAT) and multidimensional computerized adaptive testing (MCAT) under a series of BIP settings. For both UCAT and MCAT, it was found that the estimation precision of the BIP method improved as the number of blocks increased or the item pocket size decreased under the reasonable strategy. The BIP method was more effective in handling the Wainer-like strategy. With the help of block design, the BIP method can still maintain acceptable estimation precision under slightly large total IP size conditions. These results suggested that the BIP method was a reliable solution for both reviewable UCAT and MCAT.

Jin, Chen, and Wang (2018) proposed using the log of the odds ratio (OR) as a DIF index for item responses that follow the Rasch model. The OR is relatively simple to calculate and does not require grouping examinees by ability. This index was more robust to unbalanced DIF than the Mantel-Haenszel (MH) or logistic regression (LR) procedure. The authors cautioned against using this index with data that does not follow the Rasch model. This brief commentary explains one problem with using the OR method with 2PL or 3PL item responses: If the response functions are identical for both groups, when the group mean difference is large the log of the OR will vary with the value of the item parameters.

Fixed-precision between-item multidimensional computerized adaptive tests (MCATs) are becoming increasingly popular. The current generation of item-selection rules used in these types of MCATs typically optimize a single-valued objective criterion for multivariate precision (e.g., Fisher information volume). In contrast, when all dimensions are of interest, the stopping rule is typically defined in terms of a required fixed marginal precision per dimension. This asymmetry between multivariate precision for selection and marginal precision for stopping, which is not present in unidimensional computerized adaptive tests, has received little attention thus far. In this article, we will discuss this selection-stopping asymmetry and its consequences, and introduce and evaluate three alternative item-selection approaches. These alternatives are computationally inexpensive, easy to communicate and implement, and result in effective fixed-marginal-precision MCATs that are shorter in test length than with the current generation of item-selection approaches.

There are many item response theory software packages designed for users. Here, the authors introduce an environment tailored to method development and simulation. Implementations of a selection of...

The R package irtplay provides practical tools for unidimensional item response theory (IRT) models that conveniently enable users to conduct many analyses related to IRT. For example, the irtplay includes functions for calibrating online items, scoring test-takers' proficiencies, evaluating IRT model-data fit, and importing item and/or proficiency parameter estimates from the output of popular IRT software. In addition, the irtplay package supports mixed-item formats consisting of dichotomous and polytomous items.

The higher-order structure and attribute hierarchical structure are two popular approaches to defining the latent attribute space in cognitive diagnosis models. However, to our knowledge, it is still impossible to integrate them to accommodate the higher-order latent trait and hierarchical attributes simultaneously. To address this issue, this article proposed a sequential higher-order latent structural model (LSM) by incorporating various hierarchical structures into a higher-order latent structure. The feasibility of the proposed higher-order LSM was examined using simulated data. Results indicated that, in conjunction with the deterministic-inputs, noisy "and" gate model, the sequential higher-order LSM produced considerable improvement in person classification accuracy compared with the conventional higher-order LSM, when a certain attribute hierarchy existed. An empirical example was presented as well to illustrate the application of the proposed LSM.

Many educational testing programs require different test forms with minimal or no item overlap. At the same time, the test forms should be parallel in terms of their statistical and content-related properties. A well-established method to assemble parallel test forms is to apply combinatorial optimization using mixed-integer linear programming (MILP). Using this approach, in the unidimensional case, Fisher information (FI) is commonly used as the statistical target to obtain parallelism. In the multidimensional case, however, FI is a multidimensional matrix, which complicates its use as a statistical target. Previous research addressing this problem focused on item selection criteria for multidimensional computerized adaptive testing (MCAT). Yet these selection criteria are not directly transferable to the assembly of linear parallel test forms. To bridge this gap the authors derive different statistical targets, based on either FI or the Kullback-Leibler (KL) divergence, that can be applied in MILP models to assemble multidimensional parallel test forms. Using simulated item pools and an item pool based on empirical items, the proposed statistical targets are compared and evaluated. Promising results with respect to the KL-based statistical targets are presented and discussed.