Students' knowledge about chemical reactions – development and analysis of standard-based test items

This article examines the relationship between test administration tools and item types. It firstly examines the difference in the test administration tools such as paper-based test, computer-based test, and web-based test, in particular, dealing with speaking test. The difference is relevant to item or task types. Based on the considerations of Bachman and Palmer (1996) and Chapelle and Douglas (2006) on test development, and Weir's (2005) definition of context validity, the application of computer technologies to test administration needs to be considered an important construct affecting test takers' performance. To investigate the item type difference among current speaking tests and its relationship with computer technology in administrating the tests, twelve test takers, five proctors, and two computer lab administrator participated in the phone interview and e-mail survey. Six popular speaking tests (TSE, PELT, IELTS, MATE, iBT TOEFL, ESPT) were selected to be analyzed. The data were analyzed in terms of item types, computer technology use, and test-takers perceptions. The implication is suggested for students, teachers, and language testers.

The study explored the practices of assessing graduate students’ learning outcomes by instructors of higher education institutions (HEIs) in Ethiopia. Survey method was employed in collecting data from 131 instructors, which were selected with stratified sampling. Questionnaire and interview were used as instruments for collecting data on the instructors’ practices of types and frequency of using assessment strategies, types of test items in practice and use of assessment for improving learning. The data from these three variables were analyzed in terms of demographic factors: bands or fi elds of study, instructors’ academic rank and experience in teaching in HEIs. The results showed that there were no signifi cant differences among instructors from different bands or field of study in the type and frequent use of assessment strategies, in their practice of different types of tests items and use of assessment results for improving learning. Also no significant mean differences were observed in teaching experiences in HEIs. However, significant differences were observed among the mean scores of the above three variables by academic rank with better use of assessment strategies, different types of test items by lecturers than professors and associate professors. It was concluded that professors and associate professors should pay attention to the assessment of students learning and further study need to be conducted for identifying why professors and associate professors were paying less attention to student assessment than lecturers.

ABSTRACTThe controversy over the effectiveness of coaching for standardized admission tests has been fueled by studies that have sought dichotomous answers to questions that are more appropriately posed as ones of degree. Instead of asking categorically “Does coaching work?“, researchers would seem better advised to ask “How much time and effort, devoted to what kinds of coaching experiences, produce how much improvement in test performance?”To explore some relational answers, information on test preparation activities was collected from a large representative sample of candidates who took the GRE Aptitude Test in June 1980. About 3 percent of these candidates indicated that they had attended formal coaching programs for one or more sections of the test and provided information on the length, cost, and offerer of the courses.After adjusting for differences in the background characteristics of coached and uncoached students, effects on test scores were related to the length, the cost, and the type of programs offered. The effects on GRE verbal ability scores were not significantly related to the amount of coaching that examinees received. Quantitative coaching effects appeared to increase only slightly with time, but the relationship was not statistically significant. Effects on analytical ability scores, on the other hand, were related significantly to the length of coaching programs, through improved performance on two analytical item types, analysis of explanations and logical diagrams. These item types were shown previously to be susceptible to improvement through formal instructional intervention and have since been deleted from the test.Test performance was related to the kinds of coaching programs examinees attended only for the quantitative section of the test. With respect to the possibility that some kinds of test takers might profit more than others from coaching, exploratory analyses suggested that examinees intending to pursue higher‐level degrees may have benefited more than lower‐level degree seekers. This finding is consistent with the conjecture that highly motivated test takers may achieve greater effects from coaching than less motivated examinees.Overall, the data suggest that, when compared with the two highly susceptible item types that have been removed from the GRE Aptitude Test, the test item types in the current version of the test (now called the GRE General Test) appear to show relatively little susceptibility to formal coaching experiences of the kinds considered here.

Despite the widespread use of the previewing strategy (i.e., previewing multiple choice options prior to reading a main passage) in a test context like CSAT English Reading, there are few studies that empirically identify its effects. To fill this gap, the current study examined the effect of previewing in relation with four interaction variables regarding test-items (item types; item difficulty) and learner’s characteristics (learning styles; English proficiency). The research was implemented based on the experimental design. Specifically, Korean secondary school students were assigned to either previewing or non-previewing condition. Also, four moderating variables were manipulated or measured through test-item choices and the survey. The collected data was analyzed through four series of Analysis of Covariance (ANCOVA). Main findings are as follows: First, the previewing strategy had significant influence on the reading performance. Especially, this study identified specific conditions for previewing to backfire. Second, the result clarified what type of test-item (e.g., inference type items) is appropriate for previewing. Third, it also told who should avoid using the previewing strategy (e.g., holistic style; intermediate level). The findings provided pedagogical implications on how to teach a specific reading strategy in the frame of Strategy-Based Instruction (SBI). Limitations and future research areas are suggested in the final chapter.

Examinations are used widely in nursing education to evaluate knowledge attainment. New item types were initiated in April 2023 by the National Council of State Boards of Nursing (NCSBN) for use on the Next Generation National Council Licensure Examination for Registered Nurses (NGN NCLEX-RN). Little evidence exists for how much time is needed for exams that use the new item types. Item analysis was conducted on 772 questions that were answered by 14,728 nursing students. Seven item types were analyzed, including calculation, cloze, matrix, ordered response, hot spot, multiple choice, and multiple response. Calculation questions required additional time. Multiple choice items required 1 minute per question. Questions that were more difficult required more time. Exams administered early in the nursing curriculum need more time allocated. Nursing educators should focus on time allotment for examinations based on the item types and difficulty.

Test items created for dentistry examinations are often individually written by content experts. This approach to item development is expensive because it requires the time and effort of many content experts but yields relatively few items. The aim of this study was to describe and illustrate how items can be generated using a systematic approach. Automatic item generation (AIG) is an alternative method that allows a small number of content experts to produce large numbers of items by integrating their domain expertise with computer technology. This article describes and illustrates how three modeling approaches to item content-item cloning, cognitive modeling, and image-anchored modeling-can be used to generate large numbers of multiple-choice test items for examinations in dentistry. Test items can be generated by combining the expertise of two content specialists with technology supported by AIG. A total of 5,467 new items were created during this study. From substitution of item content, to modeling appropriate responses based upon a cognitive model of correct responses, to generating items linked to specific graphical findings, AIG has the potential for meeting increasing demands for test items. Further, the methods described in this study can be generalized and applied to many other item types. Future research applications for AIG in dental education are discussed.

Since its first pilot study was launched in 2003, China Accreditation Test for Translators and Interpreters (CATTI) has developed into the most authoritative translation and interpretation proficiency qualification accreditation test in China and played an important role in assessing and cultivating translaters and interpreters. Based on the analysis of its form and content, the current study appraises CATTI in terms of reliability, validity, practicality, fairness and washback. The results suggest that CATTI has a satisfying validity and reliability, but the amount and types of test items, test content, evaluation rubrics and transparency are expected to be improved.

To analyze and evaluate textbooks, researchers have either proposed scales and checklists to be filled by teachers and learners or conducted qualitative investigations of the match between SLA theories and textbook activities. This study, however, employs the microstructural approach of schema theory to scrutinize the reading passages of “Mosaic 1 Reading”. To this end, 17 passages of the textbook were randomly chosen and their constituting words were explored as semantic, syntactic, and parasyntactic schemata. The passages were also analyzed in terms of their readability indices. The results showed that they consist of 3722 schema types, 2979 (80%) of which are semantic in nature. Although the textbook aims at “academic success” at English language “proficiency levels”, it provides no objective definition of what they stand for. In terms of readability, however, the passages vary in difficulty from grade three in primary school to college level. Further, the textbook is discussed in terms of its constituting schemata and suitability to the Iranian context and suggestions are made for future research. The findings of this study have important implications for language teaching, testing and materials development. They show that language proficiency must be defined in terms of schema types and the bulk of class time must be spent on teaching semantic schemata rather than syntactic and parasyntactic ones. Similarly, for testing the reading comprehension of these passages, the number and type of test items must be based on the percentage of semantic and syntactic schema types and subjective criteria such as teachers’ intuition or experience must be avoided both in teaching and testing the comprehension of passages.

Although many instructors prefer multiple-choice (MC) items due to their convenience and objectivity, many others eschew their use due to concerns that they are less fair than constructed response (CR) items at evaluating student mastery of course content. To address three common unfairness concerns, I analyzed performance on MC and CR items from tests within nine sections of five different biology courses I taught over a five-year period. In all nine sections, students’ scores on MC items were highly correlated with their scores on CR items (overall r = 0.90), suggesting that MC and CR items quantified mastery of content in an essentially equivalent manner—at least to the extent that students’ relative rankings depended very little on the type of test item. In addition, there was no evidence that any students were unfairly disadvantaged on MC items (relative to their performance on CR items) due to poor guessing abilities. Finally, there was no evidence that females were unfairly assessed by MC items, as they scored 4% higher on average than males on both MC and CR items. Overall, there was no evidence that MC items were any less fair than CR items testing within the same content domain.

In conclusion, this presentation has reviewed the place of objective testing within the evaluation continuum. It is apparent that objective tests are appropriate for evaluating basic levels of Bloom's cognitive and affective domains. The item types of an objective test measure these different levels within each domain. It is the responsibility of the nursing educator to determine which type of test item will most efficiently measure the objectives for a particular learning experience. No matter what the purpose of the test, if the nursing educator wishes to increase validity, reliability and practicality of the test, rules of overall test and item construction should carefully be reviewed and utilized in test implementation.

Some prior studies of item recognition, source recognition, and judgments of learning have produced evidence of framing effects in episodic memory. A theoretical consequence of these phenomena is that emotional arousal is not a necessary condition for framing effects because, unlike the classic framing effects of social psychology and behavioral economics, different memory frames are not designed to stimulate different emotional reactions. However, a review of available evidence on framing effects in item recognition revealed that the data were inconsistent for old items and nonexistent for similar distractors. We attempted to secure definitive evidence of whether these types of items display framing effects by analyzing a corpus of 478 sets of data, in which old items and similar distractors are factorially crossed with old? and similar? recognition frames. Both types of items exhibited large framing effects: Recognition was far more accurate for similar items than for old items when probes asked if test items were old, whereas recognition was more accurate for old items than for similar items when probes asked if test items were similar. There was a frame-independence effect, too, such that accuracy in one memory frame was dissociated from accuracy in the other frame. The conjoint-recognition model predicted both the core memory framing effect and its direction (superior recognition of similar distractors in the old? frame but superior recognition of old items in the similar? frame). The model also explained frame independence as a by-product of between-frame differences in how three retrieval processes (true recollection, false recollection, and semantic familiarity) affect accuracy in the different frames. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Praise for the 3rd Edition: Without question, this book should be on every nurse educator's bookshelf. --Nursing Education Perspectives An invaluable resource for theoretical and practical application of evaluation and testing of clinical nursing students. Graduate students and veteran nurses preparing for their roles as nurse educators will want to add this book to their library. Score: 93, 4 stars --Doody's has again given us philosophical, theoretical and social/ethical frameworks for understanding assessment and measurement, as well as fundamental knowledge to develop evaluation tools for individual students and academic programs. -Nancy F. Langston, PhD, RN, FAAN Dean and Professor,Virginia Commonwealth University School of Nursing The fourth edition of the for evaluation and testing in nursing education helps educators measure and evaluate the level of learning that their students have achieved and presents fundamental concepts of what constitutes reliable tools for measuring students' classroom and clinical performance. The text presents the concepts of assessment, measurement, evaluation, and testing and explains how to prepare all types of tests including the alternate-format NCLEX test items. It describes how to assemble, administer, and analyze tests and test performance for both classroom and clinical evaluation. The book also provides strategies for evaluating higher cognitive levels of learning and for evaluating written assignments with sample scoring rubrics. New to the 4th edition: Offers expanded coverage of essential concepts of assessment, evaluation, and testing in nursing education Presents a new chapter on online testing, with and without an online teaching platform such as BlackBoard, standardized online testing packages, establishing valid 'online based testing, and related issues. Updates each chapter with current research and references and new examples Presents stronger focus on evaluating higher-level cognitive functions Updates test preparation for NCLEX and other certification tests Explains how to develop test items that prepare students for licensure and certification exams, including discussion of innovative items being piloted for possible use on the NCLEX Provides expanded coverage on establishing rubrics for assessing written assignments Includes new information on using simulation for High Stakes Testing Offers online teaching guide that includes PowerPoint slides, sample core syllabus, and learner activities This text is essential for all educators and students who are studying to become nurse educators as well as for nurse faculty and mentors who wish to improve their ability to test and increase the validity of their test instruments. Key Features: Presents the gold standard for evaluation and testing in nursing education Firmly grounded in concepts of assessment, measurement, evaluation, and testing across all levels of nursing education Provides guidelines for writing all types of test items (including NCLEX alternate format test items), scoring, and analyzing tests Provides valid evaluation methods for both classroom and clinical evaluation Meets NLN Certification Competency #3: Use Assessment & Evaluation Strategies (15% of CNE Test BluePrint)

This study aims to determine how many variables of physical test items are dominant and effectively used in the sport of muaythai regional training center of east java 100. This type of research is quantitative with a development research approach. This research was conducted in 3 stages, namely stage I tabulating the test items that have been determined by the team of East Java experts, stage II processing data and reducing test items that do not meet statistical prerequisites, stage III factor analysis of all test items that meet statistical prerequisites. . The data is processed using the application (SPSS, in stage I all the results of the physical tests totaling 16 test items are entered into SPSS and processed by factor analysis, there are two variables reduced and not included in the next factor rotation, stage II thetest is carried out. Kaiser Meyer Olkin Measure of Sampling (KMO) the result is 0.640> 0.5 with the results of the barletts test 201.084 with a significance of 0.000, there is 1 variable with MSA <0.5, namely Sit and reach (S&R) of 0.493, on the communality table. Based on the results of the communality test there are two variables <0.5 variable (X5) Chin Up 0.476 and (X16) Sit and Reach 0.368 these variables do not meet the requirements of communality and must be removed from the test Stage III Bartlett Test of Sphericiry is 183.067 at a significance of 0.000 with a KMO of 0.669. The conclusionfrom the results of the factor analysis of the item physical test items for muaythai in this study, there are 5 factors that are formed from 12 variables (+ 2 sub-variables formed from variable X14) which are correlated, from the initial 16 items on physical tests (with 5 sub-variables formed from 2 variables). The dominant factor for the formation of physical test items for muaythai is items related to leg and leg muscles (square jump, wattbike, kneel-up, triple jump and hamstring), the second dominant factor is the abdominal muscles which are represented by two items (sit ups). and a russian twist).

The questions of quality of test and test information by using Items Response Theory (IRT) are discussed in this work. Using the IRT gives many chances to make an effective test:
 
 to measure not only difficult of everyone test's item, but what number of information it brings in test;
 to get an even distribution of items by its difficulty which is harmonize with abilities of testing person;
 if it's known how many item gets information in test, it's possible to make a test which will evaluate the abilities of testing person group best.
 
 Analyse of test's results gives more reversible information for adjustment of education process. Also it lets to diagnose the vacancy of education process:
 
 to detect what themes are mastered worst;
 to find the more difficult items.
 
 The reversible information of test lets to recognize the testing persons:
 
 to measure the total testing person's abilities;
 to detect the vacancies of everyone testing person;
 to know how everyone of testing person can to work the different type of items.
 
 The employing of IRT technologies is correct in different stages of testing process. It helps in objectively evaluating the test quality and increasing efficiency of education process.
 The estimation of characteristics of tested persons and parameters of test items in the same scale enables to compare the ability of person with the difficult of item. It enables to create the modern potted systems of paperless learning control.

Gathered with the merits of expedition, convenience, and sharing, testing through the on-line item database has become commonly applied. The lack of interactive answering system within the item database system, however, makes it hard for the general teachers to incorporate interactive items into testing. To help testers to edit interactive items, this paper has developed interactive multimedia item templates with the technology of software component and JavaBean. We have defined an interactive test template software interface specification, have developed item template component bean (ITCBean), and have designed a relative visualized interactive test item system (ITIS). Each ITCBean represents a test item template, which not only carries interactive features of drag and drop, link, collide, and overlap, but also can add into the multimedia elements. The item editor can add test item template by dynamic loading new ITCBean (new test item type). Further, new items can be visually edited by simple drag&drop, link or parameter setting. Testing is conducted simply through the connection with the testing server, and the test report will be transmitted to the database after auto grading when the testing is completed.