Assessing professional competence: from methods to programmes

The purpose of this chapter is to provide a conceptual and practical perspective on instructional design problems that librarians face, from the perspective of design research. Librarians see increasing need to provide instruction to their users in new ways and on new topics as the world becomes digital and global in nature. The expanding instructional role of librarians requires instructional design skills. However, many librarians have little formal training on instructional design, leading to concerns and confusion about instructional design skills, problems and processes. This chapter discusses design, instructional design issues, concerns and problems (typical and specific to librarians), and explores a conceptual framework to support flexible instructional design based upon the concept of design as a discipline. Examples, recommendations and supporting resources are included.

Expert consensus on gender criteria for assessment in medical communication education

This study focuses on weaving drafts as the core research subject, exploring their application value in the pattern arrangement principles of plane composition and instructional design. The research samples were selected from the Tie-ups in Straight Twill, Crepe, and Dice Weaves in The Weaver’s Book of 8-Shaft Patterns. A total of 55 samples were collected and analyzed. Threading draft and Treadling were used as structural references to examine the correspondence between the visual composition of weaving drafts and the arrangement in graphic design. The research methodology consists of two aspects. From the perspective of plane composition, seventeen compositional principles were applied to analyze pattern characteristics, including symmetry, rotation, displacement, translation, and black–white inversion. From the perspective of instructional design, this study investigates how weaving drafts can be transformed into teaching content for plane composition, constructing a framework for pattern analysis and teaching that can be applied in basic design courses. The results indicate that weaving drafts not only possess clear structural logic in weaving construction but also exhibit a high degree of visual order and formal aesthetics. Among the compositional principles, black–white inversion, translation, reflection, and rotation are the most frequently observed, demonstrating regularity, symmetry, and variation in pattern generation. Furthermore, weaving drafts can serve as concrete and systematic visual teaching materials in plane composition. Through the correspondence among Tie-ups, Threadling draft, and Treadling, students are able to better understand the relationship between abstract compositional principles and actual patterns, thereby enhancing their compositional ability and formal interpretation in graphic design.

With most academic instructional design and technology (IDT) degree programs located within education units in higher education, teacher education is a focal point for research on the classroom teacher as instructional designer and implementer of technology in K-12. Further, teacher education serves as a locus for modelling and testing theory-based teaching practice arising from the discipline. This review examines the historical foundations and recent scholarship in teacher education from an instructional design and technology perspective in US and international contexts, providing a lens to the issues of theory versus practice and evolving research paradigms. Research areas reviewed include teacher thinking and planning, novice versus expert teacher differences, the use of systematic instructional design in classroom practices, and the teacher as designer of instructional materials. Changing research approaches and constructivist philosophies have widened the understanding of teacher instructional planning and action from earlier process–product causation to a more complex, situated view of practice. From an examination of the uneasy relationship between the two disciplines, prospects for future cooperation and research are explored in terms of theory building, impacts on training, debates on the nature of design practice, and potential for shaping educational reform efforts.

Coming from a long tradition, research on assessment in medical education has been mainly focussed on single assessment instruments. This focus fits the typical assessment approach of dividing competence into separate, individually measurable elements (e.g. knowledge, skills, attitude, problem solving). The aim of most of these studies was to achieve the best possible instrument to measure these separate elements. This measurement approach to assessment builds on research and theory in psychological testing and, consequently, approaches to quality of the assessment are consistent with the psychometric framework. Quantifiable measures of assessment quality, such as reliability, are key in the evaluation of assessment instruments. This wealth of research in past decades has provided valuable insights into the strengths and weaknesses of single assessment instruments used in medical education (Van der Vleuten et al., 2010). It has further resulted in an extensive toolbox with assessment instruments that can be used to assess many different competency areas or skills (e.g. the Objective Structured Clinical Examination for measuring technical/procedural skills (Van der Vleuten and With this assessment approach, decisions about student achievement are typically based on the collection and combination of the separate outcomes of each of the examinations without taking into account if and how these building blocks represent a complete and integrated picture of professional competence. Results on a written multiple choice test measuring knowledge are combined with results on an OSCE for skills and professional behaviour assessment for attitude. Simply adding up or lumping together individual and independent exams does not capture competence comprehensively. This division of one test per element leads to loss of integration, as the total is more than the sum of its parts. Competence in regarded as a whole task, which cannot be broken down into separate parts. Competence does not consist of one--dimensional traits, but is a complex integrated construct (Schuwirth and Van der Vleuten, 2006). Assessing only easily measured elements of competence in isolation might provide a skewed view on the qualities of an individual. It is only logical to conclude that no single instrument, however psychometrically sound, will ever be able to provide all the information for a comprehensive evaluation of competence in a domain as broad as medicine. Furthermore, while acknowledging the importance of psychometrics, it is clear that exclusively focussing on psychometrics is an insufficient basis (Schuwirth and Van der Vleuten, 2006) for selecting assessment instruments. It is a well--accepted fact that decisions about selecting assessment instruments require trade-offs between various quality aspects. Not only should reliability and validity be taken into account, but educational impact, acceptation, and costs need to be considered too (Newble et al.,

Purposeful assessment

The purpose of this qualitative case study is to examine the use of TPACK and Twitter in an graduate level online course. The overarching research question is, “How is social media, in particular, Twitter, being used in an online graduate course?” This study uses the technological pedagogical content knowledge (TPACK) framework to provide a common language for describing the use of Twitter from an instructional design and research perspective. Data collection included (a) conversational and interactive dialogue between the researchers that included the instructional designer/course instructor, (b) the archived online course, and (c) the class Tweets (from Twitter) identified with class hashtags. Through collaborative autoethnography, the researchers (a) analyzed the instructional designer’s (also course instructor) practices, contextualizing TPACK principles for the design of the online course learning objectives and Twitter activities, and (b) engaged in an in-depth analysis of students’ tweets. Major themes identified are (a) TPACK framework explicitly supported online course design using Twitter, (b) facilitated fluency for technology integration, (c) promoted consciousness and reflection, (d) enhanced transfer of knowledge, and (e) heightened presences (cognitive, social, and teaching). One primary implication is that using TPACK for instructional design can create a common vocabulary and understanding for educators, instructional designers, and researchers.

This chapter looks at symbolizing and mathematical learning from a social constructivist perspective that is motivated by an interest in instructional design. The central theme is that of a concem for the way students actually use tools and symbols. Point of departure are analyses treat people’s activity with symbols as an integral aspect of their mathematical reasoning rather than as external aids to it. As a consequence, the process of learning to use symbols in general, and conventional mathematical symbols in particular, is cast in terms of participation. Symbol use is then seen not so much as something to be mastered, but as a constituent part of the mathematical practices in which students come to participate. This view corresponds with the author’s instructional design perspective, according to which it is essential to account for the mathematical learning not merely of individual students but of the classroom community taken as a unit of analysis in its own right. To account for this collective learning, the theoretical construct of a classroom mathematical practice is introduced, which involves taken-as-shared ways of symbolizing negotiated by the teacher and students.

This resource is a lecture on the assessment of professionalism in medical education. It draws on the current literature of both assessment in medical education and assessment of professio...

Reliability and validity of exam questions are crucial in the assessment of medical education. Validity refers to the meaning if the exam questions effectively measure as its purpose to measure, while reliability refers the consistency of a measurement or test. Various methods could be applied to measure the validity and reliability of examination questions. In this review, content validity, face validity, concurrent validity, construct validity, predictive validity are briefly mentioned for further application. Moreover, test-retest reliability, inter-rater reliability, alternate form reliability, split-half reliability are presented. Regular assessment on validity, reliability, and refinement of assessments based on feedback could reduce bias and improve quality of assessments in medical education.

The rapid development of AI (Artificial Intelligence) is enhancing the potential to accelerate a paradigm shift in foreign language teaching and learning. Instructional design must shift beyond knowledge learning and skills training to include more significant opportunities for practical application in authentic contexts. To do so, it is necessary to investigate the pedagogical possibilities of generative AI through specific educational practices. Thus this study intends to explore the potential of using generative AI to improve the efficiency and effectiveness of PBL (Problem/Project Based Learning) which integrates foreign language, information media, and professional education. This study suggests a PBL course design integrating Chinese language, tourism, and multimedia creation learning outcomes at a Japanese university, as a preliminary phase to compare and examine the teaching and learning processes between traditional methods and proactive AI-enabled methods. Through qualitative exploration of integrating generative AI into traditional instructional design, this paper considers how to organize the elements of teaching and learning for a renewed approach with greater efficiency and quality of learning outcomes.

A systematic review on how educators teach AI in K-12 education

Effective instructional design is a major prerequisite for desired learning outcomes. Even though the field of instructional design (ID) is relatively young, its multidisciplinary and rapidly evolving nature requires meticulous work to implement the most current evidence-based heuristics in instructional settings. The sixth edition of Designing Effective Instruction by G. R. Morrison, S. M. Ross, H. K. Kalman, and J. E. Kemp (Wiley, 2010, 491 pages, ISBN 13: 978-0-470-522820) serves as an invaluable source, which provides instructional designers with the essential guidance to implement these evidence-based heuristics to a variety of instructional problems appropriately. The authors are recognized scholars who have authored numerous contributive studies on instructional design and technology. Morrison has been the senior author of the book since the third edition. In the sixth edition, Kalman has joined the team. The skeleton of the book evolves from Jerrold Kemp’s early textbooks. The model presented in the book considers ID a continuous cycle of nine elements, which are supported by ongoing processes related to planning, implementation and evaluation. An emphasis on designing in business settings is observed which can be distractive for some readers; however, the approach is applicable to professionals in other settings including higher education, p-12 classrooms, and military as well as medical settings. Numerous examples based on authors’ professional experiences are provided for different settings which supports the book’s focus on the application. Moreover, various pedagogical features were included to stimulate thinking towards the goal of using the ID model for the development of effective instruction. In this regard, an appropriate balance between theory and application is provided since complex issues are integrated into the text with an easyto-follow presentation. For example, at the start of each chapter is a real-world scenario which can be used to trigger classroom discussions. Provided questions guide the readers throughout the chapters. Experiences of leading practitioners and scholars familiarize readers with the real world projects in the “Expert Edge” sections of each chapter. ‘The ID Process’ pieces show the readers how the heuristics are actually put into practice. At the end of chapters, readers are further provided with exercises and realistic problems. Last but not the least, key issues and questions regarding quality management of the ID projects are included as a separate section in many chapters. Similar to the ID model provided in the book, the organization of the chapters is somewhat flexible. Almost every chapter looks like an independent reference source in its entirety. That is, a linear reading from cover to cover is not necessary for those who are familiar with the field. On the other hand, the source can be used as a textbook for an introductory course as transitions are very effective and interrelationships between the elements of the model are highlighted very often. The authors also provide readers with suggestions on how to use the book in both

The Outcome Project requires high-quality assessment approaches to provide reliable and valid judgments of the attainment of competencies deemed important for physician practice. The Accreditation Council for Graduate Medical Education (ACGME) convened the Advisory Committee on Educational Outcome Assessment in 2007-2008 to identify high-quality assessment methods. The assessments selected by this body would form a core set that could be used by all programs in a specialty to assess resident performance and enable initial steps toward establishing national specialty databases of program performance. The committee identified a small set of methods for provisional use and further evaluation. It also developed frameworks and processes to support the ongoing evaluation of methods and the longer-term enhancement of assessment in graduate medical education. The committee constructed a set of standards, a methodology for applying the standards, and grading rules for their review of assessment method quality. It developed a simple report card for displaying grades on each standard and an overall grade for each method reviewed. It also described an assessment system of factors that influence assessment quality. The committee proposed a coordinated, national-level infrastructure to support enhancements to assessment, including method development and assessor training. It recommended the establishment of a new assessment review group to continue its work of evaluating assessment methods. The committee delivered a report summarizing its activities and 5 related recommendations for implementation to the ACGME Board in September 2008.

This paper attempts to explore effectiveness of designing multimedia interactive English learning resources in online learning environment. This paper selects ET website as a case study. In order to achieve purpose of study, method of interview is used to collect research data. Data analysis shows that users of ET website affirm positive role of effective teaching design of Multimedia Interactive English learning resources in pure online environment to promote English On whole, ET website resources meet design standards. However, due to limitation of human resources and funds, final implementation and evaluation stage has not been included in ET website system. This paper focuses on effectiveness of Multimedia Interactive English learning resources from perspective of addle teaching design. Combined with common interview methods, research finds that: from perspective of pure online learning and instructional design, practicability of resources, attraction of interface, interaction between users and resources, especially timely feedback loop, constitute a part of the change of teaching essence brought about by network learning. It can also be used as part of principle of designing meaningful and participatory content in a web-based learning environment.