Development and Validation of two Bronchoscopy Knowledge Assessments

Background: Item analysis ensures valid and reliable test construction in educational measurement. In postgraduate physical education particularly in sports biomechanics it is important to maintain the right balance between how challenging an assessment is and how well it distinguishes between different levels of student understanding. Objective: This study assessed item difficulty and discrimination indices of objective test items in sports biomechanics for M.P.Ed. students, validating their scientific soundness and educational relevance. Methods: Thirty-three (N=33) M.P.Ed. students (Batch 2024–26) enrolled in the “Fundamentals of Sports Biomechanics” course at IGIPESS, University of Delhi, participated in the study. A 17‑item knowledge test, comprising multiple choice, fill in the blank, and matching type questions, was administered under standardized conditions. Item difficulty (p‑value) and discrimination index (D) were calculated, and reliability was assessed using the odd–even method with Pearson correlation. Results: Analysis revealed that 47% of items were easy (p ≥ 0.70), 41% moderate (0.30 ≤ p < 0.70), and 12% difficult (p < 0.30). Discrimination indices showed that 29% of items were poor (<0.15), 29% acceptable (0.15–0.24), and 41% excellent (>0.34). Reliability testing demonstrated significant internal consistency (r = 0.599, p < 0.01), confirming the stability of the instrument. Conclusion: The assessment demonstrated balanced item difficulty, strong discriminatory power, and acceptable reliability, establishing its validity as a tool for evaluating biomechanics knowledge at the postgraduate level. Systematic item analysis strengthens test construction and ensures alignment with curricular objectives, supporting evidence‑based teaching in physical education. Keywords: Item Difficulty, Discrimination Index, Reliability, Sports Biomechanics, Knowledge Evaluation, Test Construction.

Item analysis is an important procedure to determine the quality of the items. The purpose of this study is to assess two important indices in item analysis procedure, namely (1) item difficulty (p) and (2) item discrimination (D) as well as a correlation between them. The study involves ten 40-item multiple-choice mathematics tests. A total of 1243 Form 2 students from public schools in Penang, Kedah, Perak, and Pahang are employed as sample for this study. Both indices are calculated based on classical test theory framework because of the advantage provided for teachers. On average, only 67 % (min = 50 %, max = 87.5 %) of the items are considered of good quality that can be kept for future testing. There is incomprehensible result regarding the correlation between the two indices. Implication in terms of teachers’ competency in test development is also discussed.

Developing high-quality multiple-choice questions (MCQs) for medical education is a challenging and time-consuming task. This study aimed to assess the applicability of Chat Generative Pre-trained Transformer (ChatGPT) in generating MCQs for ophthalmology residents. ChatGPT 4 was used to generate 100 MCQs, while an additional 100 MCQs were authored by university faculty. Item-writing flaws in both sets of questions were evaluated by a single reviewer. A quality assessment panel, consisting of board-certified ophthalmology subspecialists, compared the quality of two sets. Ophthalmology residents then answered all MCQs in a randomised order. The item difficulty and discrimination indices were calculated and compared between the two sets of questions. Item-writing flaws were more frequent in ChatGPT-generated MCQs (56%) compared with human-authored MCQs (27%, p<0.001). While ChatGPT-generated questions were comparable to human-written ones in most quality parameters, the distractor quality was significantly higher in human-generated MCQs (p=0.006). The mean resident scores were 46.5±9.5 for the ChatGPT-generated MCQs and 49.0±10.9 for the human-written MCQs (p=0.051). The difficulty index was 0.47±0.21 and 0.51±0.19, respectively (p=0.12). The discrimination index was significantly lower in the ChatGPT questions (0.20±0.19 vs 0.28±0.16, p<0.001). While ChatGPT can efficiently generate MCQs for ophthalmology residents, it has notable limitations, including higher rates of item-writing flaws and lower-quality distractors. Additionally, ChatGPT-generated MCQs are less effective at distinguishing high-performing from low-performing examinees. Integrating ChatGPT with human expertise is essential to enhance the quality and reliability of artificial intelligence-generated MCQs.

The study aimed to explore how improving conceptual understanding and problem-solving skills in secondary school students can lead to better academic success in the field of physics. The research utilised a multistage sampling technique, selecting 64 schools as clusters and randomly selecting 2350 male and female students for the study. The focus is on increasing enrolments and enhancing the overall quality of education by improving students' abilities in critical thinking and problem-solving. The ultimate goal is to boost academic achievement and ensure a higher standard of education in the sector. Two self-designed achievement tests were utilised in this research based on the updated Bloom's taxonomy to assess the conceptual understanding and problem-solving abilities of students. Conceptual Knowledge Test (CKT) was of 20 MCQs items whereas the Problems-Solving Skills Test (PSST) consisted on five problem-based supply type questions. The tests were validated by the experts. The pilot study revealed 0.87 of reliability for CKT and 0.67 for PSST. This reliability was verified with Ferguson delta 0.99 for both tests. The item difficulty and discrimination index remained moderate. The significance of the variables was determined using an independent samples t-test, while the correlation between the variables was assessed using the Pearson product moment correlation. The analysis revealed moderately significant relationships between the variables. The study revealed statistically significant gender differences in academic performance, with females outperforming males across all variables examined. Therefore, educators are advised to implement concept learning strategies and problem-based learning approaches in order to improve students' conceptual understanding and problem-solving abilities

School Climate scale is the tool used in measuring the Physical Environment, Academic Environment, Interpersonal Relationships, Institutional support, Safety and security and Professional Development and Teacher Empowerment. The school climate for teachers is a multidimensional construct that encompasses teacher-teacher relationships, teacher-administrator relationship, physical and psychological work environment. The present study was conducted to construct and standardize School Climate Scale for Post Graduate Teachers. Thirty-Eight Post Graduate school teachers were chosen at random from seven higher secondary schools in the Coimbatore District to make up the sample. 51 statements, addressing six dimensions of School Climate, namely: physical environment, academic environment, interpersonal relationships, institutional support, safety and security, and professional development and teacher empowerment for higher secondary school teachers were initially used to construct this tool. This scale was standardized using the item difficulty and discrimination index, and finally, 51 statements were retained for final analysis. With the help of subject experts, face and content validity were established. Validity and reliability of this scale were also measured through a specific procedure. The present study elaborates on the development of the scale to measure the level of School Climate among higher secondary school teachers.

The item difficulty and discrimination index values for the Tactual Performance Test Location scores are presented. The difference between this and the previous item analysis of the Location score is that in this analysis the subjects' Location scores were used only if the subject correctly remembered the block.

The test was developed during in 2022 in the Bareilly district of Uttar Pradesh state to measure farmers’ knowledge on management of parasitic infestation in dairy animals. A total number of 86 items were subjected to experts for relevancy testing and finally selected 69 items for the item analysis. These 69 items were pretested on 36 respondents from other than the study area. Based on the item analysis score, difficulty and discrimination index were calculated. The items with difficulty index ranging from 30 to 80, discrimination index above 0.3 were selected. Ultimately, 30 items were selected for the final knowledge test for dairy farmers on management of parasitic infestation. The reliability of the developed test was measured by using Cronbach’s alpha method and found to be 0.8. The overall test content validity index (CVI) was found to be 0.93 which considered fit for test. Thus, the reliability and validity of the current test indicate the consistency and precision of the results. The developed test will help in assessing the knowledge level of farmers regarding parasitic infestation management and accordingly, the awareness and training programs can be planned to enhance the knowledge level of farmers.

CME examinations: one year of experience.

The knowledge test was developed to assess the understanding of farming system for nutrition among members of Farmer Interest Group (FIG) which is crucial for enhancing agricultural practices and promoting better health outcomes. In total, 50 items were primarily constructed with the objective of endorsing rational understanding as opposed to root memorization in order to differentiate the well-informed members of the FIG regarding Farming System for Nutrition from those exhibiting inadequate knowledge. The data retrieved from the sample respondents underwent rigorous item analysis, which encompassed evaluations of both the item difficulty index and the item discrimination index. The final set-up of the knowledge test in the current study includes items with difficulty indexes ranging from 20 to 96.66 and discrimination indexes ranging from 0 to 0.65. Split-Half method was employed to check the reliability of knowledge test being developed and it was found to be 0.629. 3 FIG on farming system for nutrition from 3 villages were purposefully selected for the study. Results indicated that 61.70% of FIG members had low-level knowledge, while 38.30% exhibited high-level knowledge, highlighting the need for targeted educational interventions within farming communities.

Few objective measures of evidence-based practice (EBP) knowledge/skill exist. The Fresno Test, one objective method, was validated first with medicine, followed by versions for physical therapists, occupational therapists, speech therapists, social workers, dietitians, pediatric nurses, and health care students. The Fresno Test was adapted and tested with cohorts of acute care nurses. Six of the 14 items required revision to achieve acceptable psychometrics. The aim of this second validation study was to revise and validate a new version of the test, The Modified Fresno Test-Acute Care Nursing (MFT-ACN), to determine if it could distinguish EBP knowledge/skills among acute care nurses. A panel of eight EBP experts were engaged in multiple Delphi rounds to revise items. Individual-content validity indices (CVI) ranged from 0.83 to 1.0, with scale-CVI 0.92. Using a cohort design, a cross-sectional sample of 90 novice, master, and expert nurses were recruited via national listservs or snowball sampling to complete the revised test. Two doctorally prepared EBP experts independently scored tests using the standardized rubric. The MFT-ACN yielded strong psychometric properties (intra-class correlation coefficients > 0.80; item discrimination indices > 0.20; item-total correlations > 0.30). One poorly performing item was dropped per a priori cut-off values. The final test included 13 items, with a Cronbach's alpha=0.77. Item difficulty was moderate to high. Most items discriminated well between cohorts. Mean total scores were positively correlated with age, years since graduation, years of acute care experience, and formal EBP or research workshops or conferences or courses or immersions. Perceived level of EBP expertise was not associated with mean scores. Further research is recommended with a larger sample to assess the tests' responsiveness to change in EBP knowledge/skills over time and enhance its acceptability and feasibility. Additional research should further evaluate construct and concurrent validity against other objective assessments of EBP knowledge/skills. The validated 13-item MFT-ACN can be used to evaluate EBP competencies of acute care nurses in academic and practice settings. It can also be used by nurse scientists to build a stronger evidence base on types of educational programs that excel in advancing the critical competency of EBP among our nursing workforce.

Background: Rasch analysis has the advantage of placing both the items and the person along a single ratio scale and calibrates person ability and item difficulty onto an interval scale by logits. Therefore, Rasch analysis has been recommended as a better method for evaluating functional outcome questionnaires than traditional analyses. Objects: The aim of current study was to investigate item fit, item difficulty, rating scale, and separation index of the Korean version of the Fullerton Advanced Balance (KFAB) scale using Rasch analysis. Methods: In total, 93 patients with stroke (male=58, female=35) participated in this study. To investigate the item fit, difficulty, rating scale, and separation index of the KFAB scale, Rasch analysis was completed by the Winsteps software program. Results: In this study, all items of the KFAB scale were included in the Rasch model. The most difficult item was ‘standing with feet together and eyes closed’, and the easiest item was ‘two-footed jump’. The rating scale was a 4-point scale instead of the original 5-point scale. Person and item separation indices showed high values that can identify a person with a wide range of balance ability. Conclusion: The KFAB scale appears to be a reliable and valid tool to assess balance function in patients with stroke. Furthermore, the scale was found to discriminate among stroke patients of varying balance abilities.

This study described the responses of 180 elderly care home residents on the Tonal and Rhythm subtests of the Primary Measures of Music Audiation test (Gordon, 1979). Separate analyses on each subtest yielded a mean score, a standard deviation, two reliability coefficients and their corresponding standard errors, a frequency distribution, and item difficulty and item discrimination indexes with individual item response distributions. Tonal item errors seemed due to subjects' inabilities to discriminate small interval changes in melodies, while rhythm item errors seemed due to subjects' inabilities to discriminate changes in duration or changes in complex rhythm patterns. Any generalizations or implications are primarily confined to female, Caucasian, elderly care home residents because of the restricted population sample used in the study. Even so, implications are that institutionalized elderly persons may have difficulty learning or performing music which requires discrimination of small interval changes, duration changes, or complex rhythm patterns. Music with marked changes in pitch or duration and with simple rhythm patterns should facilitate successful musical experiences.

Item analysis is performed by developers as an integral part of the scale development process. Thus, items are excluded from the scale depending on the item analysis prior to the factor analysis. Existing item discrimination indices are calculated based on correlation, yet items with different response patterns are likely to have a similar item discrimination index. This study proposed a new item discrimination index that can be used in Likert type of scales and examined its effect on factor analysis results. For this purpose, simulative datasets were generated, and items were excluded from the analysis according to the .20, .30 and .35 item discrimination index criteria, and exploratory factor analysis was performed for a single factor. Accordingly, it was found that more variance could be explained by a single factor with fewer items compared to other discrimination indices when the .20 criterion of the slope coefficient was used as suggested in this study. Similar findings were obtained using the .35 criterion with other discrimination indices. In this context, it is recommended to use the slope coefficient as an additional discrimination index calculation method in the scale development process.

In fashion design education, students are taught how to prepare 2D patterns to create 3D garments through flat pattern making and draping courses. However, there is a lack of focus on garment fitting, and students are expected to have a natural intuition regarding fit corrections. As a result, most students graduate without gaining the crucial skill of reading misfit signs in 3D and acting on correcting 2D patterns accordingly. For this reason, in the current study, we developed a novel skill test instrument based on the Extended Skill Cycle Model using 3D simulations to quantify students' fit correction skills. We evaluated the instrument’s reliability and validity for its suitability in teaching and research applications. Fit correction questions with virtual garment simulations were designed, and participants were provided with three potential pattern modification options, out of which one option corrected garment fit. The reliability of the questionnaire was determined using Cronbach’s alpha, and validity was analyzed by comparing pre- and post-training scores, comparing scores of participants with different levels of experience, and calculating the item difficulty and discrimination index. There was a positive correlation between the pre- and posttraining test scores, confirming that the instrument could measure the increase in skill level. In addition, the pre-training scores of participants at the very minimum patternmaking experience level were lower than those of participants with higher experience levels. Therefore, our instrument could be used in patternmaking courses in fashion design education to measure student skill development.

Laboratory practical exams are a component of knowledge-based assessment for first-year medical students in our Patient-Centered Learning (PCL) curriculum and have historically been constructed in a selected response (multiple choice) format. This study was conducted in order to address influences that question stimulus and/or response formats may have on assessment of student learning in a teaching laboratory setting. A neuroscience laboratory practical exam, previously included as an actual part of PCL assessment, was offered as a pre-exam, self-assessment exercise to two classes of students who were completing the Year 1 neuroscience curriculum. Randomized groups of students received the exercise in either selected-response or constructed-response (fill-in-the-blank) formats and student performance results were compared. Students performed significantly better on the exercise constructed in selected-response format. Comparisons of individual item difficulty and discrimination indices indicated that while asking a student to reflect on multiple options and come to an informed decision about the answer they choose is valid and reliable, “cueing” based on recognition and recall rather than depth of knowledge and understanding can explain student performance differences, raising legitimate issues for continuous improvement in curriculum and assessment. Supported by UNDSMHS.