Experiences of assessment using multiple choice questions on a first year engineering module in electrical engineering

Multiple Choice Questions (MCQs) are a very well known, traditional and accepted method of assessment. The use of MCQs for testing students has produced numerous debates amongst academics concerning their effectiveness as they are viewed as practical and efficient but also perceived as possibly „too easy‟ and potentially unable to appropriately test the higher order cognitive skills that essay questions can assess.The use of MCQs in a forensic science context is currently being investigated, not only for use within forensic science education, but also for the testing of competency of qualified forensic practitioners. This paper describes a Higher Education Academy funded project that is investigating the design and the implementation of MCQs for testing forensic practitioners and the lessons that have been learnt so far, that will assist academics in the development of robust MCQ assessments within forensic science degrees to promote and assess deep learning.

The use of multiple-choice questions (MCQs) in law schools has not gained widespread acceptance, unlike in medical schools where they enjoy global usage. Law Schools traditionally use essay-type/problem-solving questions to assess students. The efficacy of this form of assessment is increasingly being attacked due to increasing numbers of students and advancements in technology that enable students to generate answers. Bloom’s Taxonomy provides for a hierarchy of learning processes, which include ‘remember’, ‘understand’, ‘apply’, ‘analysis’, ‘evaluate’, and ‘create’. Studies on whether MCQs can test higher-order learning processes required in law courses have been inconclusive. We did a retrospective study to investigate whether MCQs are an efficient and effective way of assessing law courses. Underlying this study was the desire to find an alternative mode of assessment to overcome the threats facing the essay type. Results from selected law courses were analyzed in which students’ performance on MCQ tests, oral tests, and final examinations were compared. MCQs were analyzed on a two-dimensional Bloom’s table to establish the extent to which they tested higher-order learning processes. We compared the results of scores on the MCQs with those of oral tests and final essay-type examinations using correlational analysis and one-way analysis of variance (ANOVA). This was for the courses Constitutional Law (n = 22, MCQs = 75), Intellectual Property Rights (n = 113, MCQs = 100), Broadcast & TV Law (n = 11, MCQs = 76), Administrative Law (n = 65, MCQs = 91), and Cyberspace Law (n = 28, MCQs = 101). In general, students performed best on the MCQ test compared to orals and final exams. A two-tailed correlation analysis comparing all five courses showed a strong correlation between MCQs and Orals (r = 0.699, p = 0.189) and a weak correlation between MCQ test and final exam (r = 0.196, p = 0.752). Sixty-eight percent of 75 MCQs in Constitutional Law tested higher-order processes. The mean scores for MCQ (+ = 39.45 + 4.83), orals (+ = 7.64+18.72), and final exam (+ 28.5 + 8.88) showed a best performance for MCGs. ANOVA comparing test scores for MCQs, Orals and Final examination showed a very significant difference (F (2, 63, 0.05) = 38.11, p < 0.0001). Fifty-four out of 100 MCQs for intellectual property Law tested higher-order learning processes. The mean scores for MCQ (+ = 39.78 + 5.22), orals (+ = 18.88+14.66), and final exam (+ 29.0442 + 7.71) showed a best performance for MCQs. ANOVA results were highly significant (F (2,342,0.05) = 125.565, p < 0.0001). In Broadcast & TV Law, 44.7% of 76 MCQs tested higher-order learning processes. The mean scores for MCQ (+ = 43.27+6.89), orals (+ = 31.09+12.53), and final exam (+ 32.09 + 6.69) were different. ANOVA results were very significant (F (2,30,0.05) = 6.057, p < 0.01). There were 91 MCQs in Administrative Law of which 74% tested higher-order processes. The mean scores for the three tests were MCQ (+ = 33.05 + 4.76), orals (+ = 22.89 + 16.08), and final exam (+ 32.58 + 5.58). ANOVA results were very significant with F (2,191,0.05) = 20.388, p < 0.0001. Cyberspace Law had 101 MCQs of which 38.6% tested higher-order learning processes. The mean scores for MCQ (+ = 43.50 + 4.51), orals (+ = 36.43 + 2.27), and final exam (+ 35.04 + 7.71) were different. ANOVA results were very significant (F (2,81,0.05) = 20.375, p < 0.0001). We concluded that MCQs are efficacious and efficient in testing higher-order learning processes. MCQs can be used to assess learning of law courses. We recommend that law schools should embrace MCQs for assessing law courses.

The use of multiple-choice questions in medical assessments is ubiquitous and extends in breadth from undergraduate formative and summative assessments to postgraduate certifications and reaccreditations. Despite the clear importance of multiple-choice questions in medicine, in the last thirty years there has been no authoritative peer reviewed publication that reviews the overall generation and use of multiple-choice questions in medicine. Producing effective and useful medical multiple-choice questions is as much of an art form as it is a science - unfortunately, every new writer of such questions tends to rely on their own experiences of such questions and then travel along their individual and tortuous pathway to produce “acceptable” questions. Our review article provides practical guidance and information that will assist writers of medical multiple-choice questions. As well as extensively discussing common pitfalls, we provide a checklist that represents optimum practice in question generation. We directly compare examples of good and poor multiple-choice questions. We describe the metrics for measuring effective questions; specifically, the discrimination and difficulty indices. We discuss the optimum approach to training faculty to produce flawless multiple-choice questions. We describe the potential role and benefits of students in writing multiple-choice questions. We address the current and future effectiveness of the major artificial intelligence software brands in developing objectively functional multiple-choice questions. We continue to support the pragmatic use of MCQs in undergraduate and postgraduate medical training, with the caveat that ongoing training is made available to the creators in order to facilitate the production of the highest quality questions.

<p>This study aimed to determine the effect of the number of alternative answers <br />and scoring techniques to reliability test. Forms of this test includes multiple choice test three answer choice and multiple choice test four answer choices with penalty scoring techniques (Punishment Score) and scoring techniques compensation (Reward Score). The method uses the experiment, while the designused istreatment bylevel 2 x 2. Research hypotheses were tested using two way analysis of variance (ANOVA). The result concluded that: (1) reliability tests with three multiple choice answer options is higher than the multiple choice test four answer choices, (2) reliability tests with scoring techniques penalty higher than the reliability of the technique of scoring compensation, <br />and (3) No interaction between the number of alternative answers to the scoring <br />technique. The results of this study can be considered for the teacher to give test using a form of multiple choice test three answer choice and multiple choice test four answer choices test scoring with a penalty techniques. <br /> <br />Keywords : reliability, scoring techniques, alternative answers.</p>

Comparison of examination methods based on multiple-choice questions and constructed-response questions using personal computers

The use of multiple choice questions in exams remains a common choice in education for various reasons, such as ease of assessment, perceived objectivity, and the ability to provide rapid feedback especially in large classes. Research shows that the use of multiple choice questions can strengthen retention of information, especially involving alternative wrong answers, as well as measure students' basic understanding in various subjects. However, to create quality multiple choice questions, an in-depth evaluation of the question elements is required, including item analysis to ensure the validity, reliability and fairness of the assessment. The results of this research using quantitative descriptive methods show that most of the questions can be improved, while a small number need to be rejected. The research conclusions suggest that rejected items should not be reused, while items that need to be corrected should be improved to improve the overall quality of the exam. Thus, analyzing the quality of multiple choice questions is crucial for increasing the effectiveness of assessment, especially in higher education contexts such as nursing and medical education.

This study attempt to investigate and explore the use of multiple-choice questions (MCQs) at Jordanian public and private universities, from the point view of faculty member who are teaching accounting specialization and use MCQs in examinations of accountancy subjects. The study seeks to identify the extent MCQs can achieve regarding level of knowledge among students, assess the deep understanding and In-depth learning to measure and evaluate the range of learning in subjects of accounting specialization. To achieve the objectives of the study, a questionnaire designed according to the requirements of Ministry of Education in Jordan and the International Accounting Education Standard Board (IAESB).Based on analyzing responses and examining hypotheses, results clarify that MCQs are not enough tools to achieve the educational objectives concerning accounting subjects, in both theoretically and practically aspects. It was clear that there are no significant differences between the opinions of faculty members, neither public nor private universities. The study recommends that there is a need to re-consider the use of MCQs as a tool to evaluate the students’ abilities and skills in subjects of accounting specialization.

Background & Objective: In dental education, understanding optimal assessment methods and factors like stress and confidence is essential.This research assessed second-year dental students' performance in fixed prosthodontics using multiple-choice questions (MCQs) and preclinical practical exams, examining impacts of gender, stress, and confidence.Materials & Methods: Using a quasi-experimental design, 495 students from a single faculty underwent assessment.Selected via convenience sampling, they were exposed to MCQs and practical exams in fixed prosthodontics.An expert-reviewed questionnaire gauged their stress and confidence.Data was analyzed using descriptive statistics, t-tests, and Pearson's correlation.Additionally, a balanced sub-set of 176 students (88 males and 88 females) was chosen for gender-based analysis.Results: Findings indicate statistical parity between MCQ and practical exam performances (p>0.001).Females slightly outperformed in MCQs, while males excelled in practicals, without reaching statistical significance (p>0.05).Stress correlated with practical exam outcomes (r=0.34,p=0.001), and confidence with MCQ scores (r=0.41,p<0.0001). Conclusion:The research underscores near-equivalence of MCQs and practical exams for student assessments in fixed prosthodontics.Recognizing the roles of stress and confidence in assessments offers insights for balanced evaluations.Dental faculties should integrate these findings, and future work should pivot towards tool validations for enriched learning.

Tests of educational achievement typically present items in the multiple-choice format. Some achievement test items may be so saturated with aptitude (Willingham, 1980) as to be insensitive to skills acquired through education. Multiple-choice tests are ill-suited for assessing productive thinking and problem-solving skills, skills that often constitute important objectives of education. Viewed as incentives for learning, multiple-choice tests may impede student progress toward these objectives. There is need for accelerated research to develop alternatives to multiple-choice achievement tests, with content selected to match the specified educational objectives.

The advances of generative AI have posed a challenge for using open-ended questions to assess conceptual learning outcomes, as it is increasingly common for students to use tools like ChatGPT to generate long textual answers. However, teachers still have to spend substantial time reading the answers and inferring students' learning outcomes. We present TreeQuestion, a human-in-the-loop system designed to help teachers create a set of multiple-choice questions to assess students' conceptual learning outcomes. When a teacher seeks to assess students' comprehension of specific concepts, TreeQuestion taps into the wealth of knowledge embedded within large language models and generates a set of multiple-choice questions organized in a tree-like structure. We evaluated TreeQuestion with 96 students and 10 teachers. Results indicated that students achieved similar performance in multiple-choice questions generated by TreeQuestion and open-ended questions graded by teachers. Meanwhile, TreeQuestion could reduce teachers' efforts in creating and grading the multiple-choice questions in contrast to manually generated open-ended questions. We estimate that in a hypothetical class with 20 students, using multiple-choice questions from TreeQuestion may require only 4.6% of the time compared to open-ended questions for assessing learning outcomes.

Assessment plays a vital role in evaluating students’ academic performance, particularly in higher education contexts where language proficiency is essential for professional development. This study investigates the written performance of English for Medical Purposes (EMP) students at the University of Tlemcen, with a specific focus on assessment practices that contribute to enhancing students’ writing skills. The study aims to achieve two primary objectives: first, to explore the effectiveness of different assessment techniques in measuring writing proficiency; and second, to assess how writing-focused evaluations contribute to the development of academic language skills among medical students. A total of 108 students from three academic departments—Medicine, Pharmacy, and Dental Medicine—participated in the research. Data collection was carried out using a combination of student questionnaires and both formative and summative assessments. The evaluation included three types of assessment tools: multiple choice questions (MCQs), short open responses (QROC), and extended writing exams. The findings revealed strong student performance in MCQs, where 83.33% of participants provided correct answers. However, the writing exams exposed notable difficulties in grammar, spelling, and idea generation. In particular, 83.33% of the total written errors were related to spelling, and 55.56% of students experienced difficulties in organizing and expressing ideas effectively. These results underscore the need for more targeted writing assessments that can diagnose specific learner challenges and inform effective teaching strategies. The study concludes that the integration of varied assessment methods is essential for identifying linguistic weaknesses, supporting student learning, and ultimately improving writing proficiency in medical English contexts.

A uniform proof is given that the entries of the hvector of the cluster complex ∆(Φ), associated by S. Fomin and A. Zelevinsky to a finite root system Φ, count elements of the lattice L of noncrossing partitions of corresponding type by rank. Similar interpretations for the h-vector of the positive part of ∆(Φ) are provided. The proof utilizes the appearance of the complex ∆(Φ) in the context of the lattice L in recent work of two of the authors, as well as an explicit shelling of ∆(Φ). This article has appeared in International Mathematical Research Notices, (2007), article ID 69705. Department of Mathematics (Division of Algebra-Geometry), University of Athens, Panepistimioupolis, 15784 Athens, Greece E-mail address: caath@math.uoa.gr School of Mathematical Sciences, Dublin City University, Glasnevin, Dublin 9, Ireland E-mail address: tom.brady@dcu.ie Mathematics Department, University of California, Santa Barbara, Santa Barbara, CA 93106, U.S.A E-mail address: jon.mccammond@math.ucsb.edu School of Mathematical Sciences, Dublin Institute of Technology, Dublin 8, Ireland E-mail address: colum.watt@dit.ie Date: June 20, 2006, Final Version 26/6/06. 2000 Mathematics Subject Classification. Primary 20F55; Secondary 05E99. 1

A fascinating feature on Linked Data (LD) for libraries, archives and other cultural heritage institutions. Lucy McKenna is in the final year of her PhD in the ADAPT Centre, Trinity College Dublin. The Centre, funded by Science Foundation Ireland, ‘focuses on developing next generation digital technologies that transform how people communicate by helping to analyse, personalise and deliver digital data more effectively for businesses and individuals. ADAPT researchers are based in four leading universities: Trinity College Dublin, Dublin City University, University College Dublin and Dublin Institute of Technology. ADAPT's transformative tools allow you explore video, text, speech and image data in a natural way across languages and devices, helping companies unlock opportunities that exist within digital content to re-imagine how to connect people, process and data to realise new economic value.’ Lucy invites library and information professionals to evaluate an LD interlinking framework that she has developed as part of her research. She is also looking for information professionals who would like to trial the framework locally to create interlinks from their organisation’s LD datasets.

Using Testing as a Learning Tool

BackgroundWritten assessments fall into two classes: constructed-response or open-ended questions, such as the essay and a number of variants of the short-answer question, and selected-response or closed-ended questions; typically in the form of multiple-choice. It is widely believed that constructed response written questions test higher order cognitive processes in a manner that multiple-choice questions cannot, and consequently have higher validity.DiscussionAn extensive review of the literature suggests that in summative assessment neither premise is evidence-based. Well-structured open-ended and multiple-choice questions appear equivalent in their ability to assess higher cognitive functions, and performance in multiple-choice assessments may correlate more highly than the open-ended format with competence demonstrated in clinical practice following graduation. Studies of construct validity suggest that both formats measure essentially the same dimension, at least in mathematics, the physical sciences, biology and medicine. The persistence of the open-ended format in summative assessment may be due to the intuitive appeal of the belief that synthesising an answer to an open-ended question must be both more cognitively taxing and similar to actual experience than is selecting a correct response. I suggest that cognitive-constructivist learning theory would predict that a well-constructed context-rich multiple-choice item represents a complex problem-solving exercise which activates a sequence of cognitive processes which closely parallel those required in clinical practice, hence explaining the high validity of the multiple-choice format.SummaryThe evidence does not support the proposition that the open-ended assessment format is superior to the multiple-choice format, at least in exit-level summative assessment, in terms of either its ability to test higher-order cognitive functioning or its validity. This is explicable using a theory of mental models, which might predict that the multiple-choice format will have higher validity, a statement for which some empiric support exists. Given the superior reliability and cost-effectiveness of the multiple-choice format consideration should be given to phasing out open-ended format questions in summative assessment. Whether the same applies to non-exit-level assessment and formative assessment is a question which remains to be answered; particularly in terms of the educational effect of testing, an area which deserves intensive study.Electronic supplementary materialThe online version of this article (doi:10.1186/s12909-014-0249-2) contains supplementary material, which is available to authorized users.