Expert Validation of a Python Test, Reliability, Difficulty and Discrimination Indices

Objective: The objective of the study was to assess the quality of multiple-choice questions (MCQs) of three different assessments in the subject of Community Medicine by computing the difficulty index, discrimination index and reliability and to estimate the relationship between difficulty and discrimination indices. Study Design: Retrospective observational study. Place and Duration of Study: Department of Community Medicine at Wah Medical College from August to December 2018. Materials and Methods: Three sets of MCQs were included in the study. Mean and standard deviation of difficulty and discrimination indices were calculated and one-way analysis of variance and Kruskal Wallis test were applied on difficulty and discrimination indices. The association was determined by Pearson correlation and considered significant at p value of < 0.05. Results: The mean difficulty index of first term, second term and send-up examination were 41.5, 48.8 and 51.9 respectively. Mean discrimination indices were 0.28, 0.27 and 0.26 and reliability were 0.83, 0.81 and 0.79. In the study, 72% MCQs of the first term, 61.5 % of the second term and 63% of the send-up examinations were in the range 30-70% of difficulty. There was a significant difference in the difficulty indices of the three question papers. The correlation between discrimination and difficulty indices was curvilinear and positively correlated. Conclusion: It is concluded that all three question papers have acceptable reliability, more than 65% MCQs have acceptable difficulty index and about 69% have good discriminatory power.

New generations are using and playing with mobile and computer applications extensively. These applications arethe outcomes of programming work that involves skills, such as computational and algorithmic thinking. Learningprogramming is not easy for students children. In recent years, academic institutions like theMassachusetts Institute of Technology (MIT) and hi-tech companies, such as Google and Khan Academy, haveintroduced online environments to facilitate the teaching and learning of programming. Most of these programmingenvironments are web-based, and interactive and are supported with visual multimedia features. Therefore, they havebecome easy to use, very attractive and helpful for teaching children how to program and to develop theiralgorithmic and computational thinking skills. The proposed presentation will describe research that examined theteaching of a course to primary school children based on three on-line interactive environments: "Plastelina" for logicgames, "Code with Anna and Elsa" via the Hour of Code project block-oriented programming environment, forblock programming and "Turtle Academy" for textual programming in the Logo language. The current researchincluded the development, implementation and evaluation of the course at an elementary school. In addition, it wasaimed at investigating the pupils' attitudes toward the learning of computer programming, both before and afterparticipation in the course. The results revealed that the pupils' attitudes towards programming remained positivealso also after the participation in the course. It was also found that programming improved children's problemsolving skills.

Computational thinking has already been acknowledged as an important learning objective for all students in compulsory education. Our research interests focus on the assessment of fundamental computational thinking competencies, such as collection, organisation and analysis of data, algorithmic thinking and abstraction. For the needs of our research, we implemented the digital platform PhysGramming, which constitutes the backbone of a relevant assessment tool we have built. PhysGramming is developmentally appropriate for first and second grade primary school students and provides the opportunity to create their own digital games, transforming them from passive consumers to active creators of digital technology. The assessment tool has been applied to 450 students in the first and second grade of primary schools on Crete, Greece from February to June 2019. The research was conducted attuned to the ethical guidelines of educational research, within the framework of the physical and natural science courses and evolved around three axes: (a) Checking the validity and the reliability of the results that come out when applying the proposed assessment tool. (b) Designating the levels of computational thinking skills of students at first stages of schooling. (c) Checking the correlation between the levels of computational thinking skills and the comprehension of the lesson's content. In other words, we check whether high learning performance is a necessary and sufficient condition for diagnosing high levels of basic computational thinking skills. In this paper, we shortly describe the assessment tool we propose.

Multiple choice questions (MCQs) have considerable role in the preclinical medical assessment, both formative as well as summative. This cross sectional descriptive study was conducted to observe the quality of MC items (completion type) of anatomy, biochemistry and physiology used in preclinical undergraduate medical examinations of 2012 and 2013 of a public university of Bangladesh. Each MC item had a stem and 5 options, and 1200 options were analyzed for difficulty and discrimination indices. Total 556 options were false statements (distracters) and were analyzed to observe their effectiveness as distracter. The study revealed that 18.67% of options were with appropriate difficulty (0.660.80). Highest frequency (43.5%) of difficulty indices was in easy class interval (0.911). Over all frequencies of items of three subjects in the ascending order were difficult, appropriate, marginal and easy as per their difficulty indices. Satisfactory or better discrimination indices (=0.20) were observed in 29.33% options. The mean difficulty and discrimination indices observed were respectively 0.82±0.18 (95% confidence interval [CI] 0.81 to 0.83) and 0.13±0.14 (95% CI 0.122 to 0.138). Out of the options, 6.75% had negative discrimination indices. Items with difficulty index around 0.60 had maximum discriminatory power (up to 0.68) and more difficult as well as easy items had less discriminatory ability. Out of the distracters 83.45% were observed effective and the mean effectiveness was 22.3±18.7% (95% CI 20.75% to 23.85%). The study recommended using the method and findings to improve the quality of the items leading to development of a standard Question Bank.Bangladesh Journal of Medical Education Vol.7(1) 2016: 2-7

Assessment plays a crucial role in medical education providing both measures of student performance and feedback for curriculum improvement. This study evaluates the quality of A-type (single-best-answer) multiple-choice questions (MCQs) used in the Fixed Prosthodontics exams over three academic years (2020-2022) at a Sudanese dental school, using psychometric item analysis. This cross-sectional study (February-April 2023) retrospectively analyzed 155 MCQs administered between 2020 and 2022. Each item was assessed for item-writing flaws (IWFs), difficulty index (DIF: percentage of correct responses), discrimination index (DIS: ability to differentiate between highest and lowest 27% of examinees), and reliability (Cronbach's alpha). The normality of DIF and DIS data was tested; Spearman's or Pearson's correlation was used as appropriate. The percentage of items with IWFs was 48% (2020), 42% (2021), and 33% (2022). Mean ± SD of the DIF index values were (75.51 ± 8.47), (72.25 ± 7.93), and (60 ± 11.63). While Mean ± SD of the DIS index values were (0.18 ± 0.17), (0.13 ± 0.16), and (0.21 ± 0.13) respectively. Cronbach's alpha, coefficients (α) increased from 0.69 (2020) and 0.64 (2021) to 0.76 (2022). There were no significant correlations between DIF and DIS across the three years (p > 0.05). Item-writing quality improved from 2020 to 2022, with the proportion of flawed items decreasing and reliability increasing. Most items were easy in 2020 and 2021, with more excellent difficulty in 2022. Discrimination improved but remained suboptimal. There was no significant correlation between difficulty and discrimination indices. Routine psychometric analysis and enhanced faculty training are recommended to maintain assessment standards in dental education.

<p>The recent advancement in computational thinking (CT) research has reported numerous learning benefits to school-age children. The long-standing perceived difficulty of computer programming has challenged the acquisition of CT skills from programming education. Several block-based programming environments (BBPEs) have been developed to reduce this difficulty and enhance active engagement in computational-related activities. Although numerous studies have examined students’ level of interactions during block-based programming modality (BPM) activities, a major gap in the literature is the paucity of research evidence reporting the association between these interactions and CT. This study, therefore, investigates the association between interaction patterns during BPM activities and CT skills. The present study employed a longitudinal approach where the same participants were observed over eight weeks. Thirty-five, second-year-level computer science and computer education students (mean age: 19.8; male = 23, female = 12) from a research university in Nigeria were recruited. Their computational activities over the study periods were video-recorded. The participants’ CT skills were collected using the computational thinking test and the computational thinking scale. Findings indicate four interaction patterns: learner–learner, learner–content, learner–teacher, and learner–distractor. Learner–learner and learner–content were prevalent. The interaction patterns significantly predict CT skills although significant differences exist across gender, cognitive load, spatial ability, and programming proficiency. The research has provided opportunities for educators to integrate BBPEs in learning programming and CT concepts. Although such integration is likely to occur with the help of strong educational policies, teachers are encouraged to cultivate the spirit of collaboration in students during programming activities.</p>

Enhancing elementary school students' computational thinking and programming learning with graphic organizers

Programming attitudes predict computational thinking: Analysis of differences in gender and programming experience

Learning to program is the new literacy of the 21st century. Computational thinking, closely related to programming, requires thinking and solving problems with different levels of abstraction and is independent of hardware devices. The early childhood education stage provides teachers with the opportunity to lay the foundations for a comprehensive quality education using innovative tools and technologies. Educational robotics in early childhood education becomes a tool that facilitates the acquisition of knowledge to children, playfully, based on the principles of interactivity, social interrelationships, collaborative work, creativity, constructivist and constructionist learning, and a student-centered didactic approach, allowing in turn that student can acquire digital competencies and develop logical and computational thinking in an underlying way. This project explores the current state of teaching and learning computational thinking and programming in early childhood education in an inclusive manner. Moreover, the lack of diversity and inequality is particularly latent in science, Technology, Engineering, and Mathematics (STEM) fields. Therefore, this work considers this problem and presents an inclusive coeducation approach to this new literacy, eliminating gender stereotypes and extending them to people with Down syndrome and hospitalized minors.

This study investigated minority students’ learning in programming. The variables, including self-efficacy, computational thinking, and learning performance were the focus of this study. This study explored the relationships of creative self-efficacy, learning self-efficacy, computational thinking, and learning performance among minority undergraduate students. The influence of creative self-efficacy, learning self-efficacy, and computational thinking on learning performance was explored. The participants were minority students from a HBCU institution in the southeastern United States. Quantitative approaches were performed to analyze the collected data. The results indicated that self-efficacy, learning self-efficacy, and computational thinking were positively correlated with learning performance. Learning self-efficacy and computational thinking were significant predictors of learning performance among minority students.

Programming has received extensive attention for the embodiment of computational thinking (CT) skills. However, the lack of in‐depth research on the characteristics of college students' CT skills in text‐based programming learning has resulted in the unknown distribution characteristics and evolution patterns of their CT skills, which cannot effectively constitute an effective tracking analysis of their CT skills, thus affecting the cultivation of creative thinking and problem‐solving skills for them. Therefore, the purpose of the paper is to explore the characteristics of college students' CT skills through text‐based programming work and discover the overall characteristics, individual differences and evolutionary patterns of their CT skills on the relation between text‐based programming and CT skills through data analysis. It was found that there are roughly five categories of CT difference and four major categories with nine subcategories of CT evolution among college students. The results showed that the Control and Data skills were the students' dominant skills. In contrast, Algorithm skills were relatively weak. Only about 1/3 of the students had the phenomenon of continuous skill improvement, and programming learning did not have a significant impact on the improvement of CT skills for some students. Finally, we propose several suggestions for CT skills development strategies in response to the students' characteristics in CT skills. The study can not only fill the gaps of the current study but also identify the evolutionary characteristics of students' CT skills, which can effectively intervene in college students' programming learning and help improve their CT skills.

The study examined the effect of teaching text-based programming with a physical programming tool on secondary school students’ computational thinking skills and self-efficacy perceptions. The study was conducted according to a sequential explanatory design as a mixed method research. The study group consisted of 85 secondary school students. Within the scope of the study, a physical programming tool called Micro:bit was used to teach Python programming for a period of 6 weeks. Data were collected using the Self-Efficacy Perception Scale for Computational Thinking Skill, Bebras: International Challenge on Informatics and Computational Thinking Tasks, tests focused on programming tool, concepts, and processes, and through semi-structured interview questioning. According to the findings obtained from pretests and posttests, a significant and positive difference was found in the students’ computational thinking skills and self-efficacy perceptions towards computational thinking skill. As a result of having received instruction in programming, the students satisfactorily learnt the required programming concepts and processes. Through learning Python programming with a physical programming tool, the students not only gained the skills required to write appropriate syntax, and to test and debug code, but they also learnt programming concepts such as variables, conditional expressions, loops, and functions.

Developing high-quality multiple-choice questions (MCQs) for medical education is a challenging and time-consuming task. This study aimed to assess the applicability of Chat Generative Pre-trained Transformer (ChatGPT) in generating MCQs for ophthalmology residents. ChatGPT 4 was used to generate 100 MCQs, while an additional 100 MCQs were authored by university faculty. Item-writing flaws in both sets of questions were evaluated by a single reviewer. A quality assessment panel, consisting of board-certified ophthalmology subspecialists, compared the quality of two sets. Ophthalmology residents then answered all MCQs in a randomised order. The item difficulty and discrimination indices were calculated and compared between the two sets of questions. Item-writing flaws were more frequent in ChatGPT-generated MCQs (56%) compared with human-authored MCQs (27%, p<0.001). While ChatGPT-generated questions were comparable to human-written ones in most quality parameters, the distractor quality was significantly higher in human-generated MCQs (p=0.006). The mean resident scores were 46.5±9.5 for the ChatGPT-generated MCQs and 49.0±10.9 for the human-written MCQs (p=0.051). The difficulty index was 0.47±0.21 and 0.51±0.19, respectively (p=0.12). The discrimination index was significantly lower in the ChatGPT questions (0.20±0.19 vs 0.28±0.16, p<0.001). While ChatGPT can efficiently generate MCQs for ophthalmology residents, it has notable limitations, including higher rates of item-writing flaws and lower-quality distractors. Additionally, ChatGPT-generated MCQs are less effective at distinguishing high-performing from low-performing examinees. Integrating ChatGPT with human expertise is essential to enhance the quality and reliability of artificial intelligence-generated MCQs.

This paper proposes a scenario-based learning (SBL) methodology for teaching Computational Thinking (CT). The presented scenario includes educational material that teaches the basic concepts of a Python course for beginners. The scenario allows the educator to utilize a combination of tools and services and follow a mind map. Moreover, it presents a story through steps which the Generation Z students can follow to learn, understand, and test their acquired knowledge on the Python programming language while engaging in non-coding-related topics such as the history of coding and coding ethics. The proposed SBL methodology allows teachers of CT to familiarize themselves with a series of tools (e.g., Twine, Kahoot!, Repl.it, Discord, and Gather.town) so as to facilitate Gen Z students to learn in synchronous and asynchronous educational environments. The SBL methodology was assessed by using a variety of inspection methods depending on the scope of the individual using the method (e.g., the effectiveness of the tools for the educator, the ease of use, and the interest generated by a Gen X student). The assessment results were quite encouraging.

Programming learning has become a 21st century skill, several countries have integrated it into their school curricula. Numerous studies have addressed the teaching of computational thinking and programming in primary schools and even nursery schools. These studies have shown that learning programming presents many difficulties for learners, especially beginners. This paper proposes a new classification of the difficulties encountered in learning programming for beginners, in particular for secondary school students. We also aim to study the impact of gender and level of learners in less developed regions on programming learning. We carried out two questionnaires with 79 students in a final year science class at a secondary school in Tunisia. The study showed that the majority of learners encountered difficulties of several types and with several concepts. The most difficult concepts were variables, operator priority rules and loops. The study also showed that gender had no effect on programming.