Script concordance testing in genetic counseling training: Apilot study.

Prosthodontics involves replacing lost dentofacial structures using artificial substitutes. Due to availability of many materials and techniques, clinician's clinical decision-making regarding appropriate selection of prosthesis requires critical thinking abilities and is demanding. Especially during graduate training years, learners do not receive the exposure to a variety of cases, thus their clinical reasoning skills are not developed optimally. Therefore, using the trend of incorporating technology in education, we developed a mobile learning app for this purpose. The aim of this study was to evaluate learners' perceptions of this app's utility and impact on their clinical decision-making skills. After taking informed consent, interns of the Department of Prosthodontics of VSPM Dental College, Nagpur, India, during the academic year May 2015-May 2016 were sent the link for the app to be installed in their Android smartphones. Their perceptions were recorded on a feedback questionnaire using 5-point Likert scale. The script concordance test (SCT) was used to check for changes in clinical reasoning abilities. Out of 120 students who were sent the link, 102 downloaded the link and 92 completed the feedback questionnaire and appeared for the SCT (response rate: 76%). The overall response to the app was positive for more than two-thirds of interns, who reported a greater confidence in their clinical decision-making around prostheses through this app and 94% of the students felt that this app should be regularly used along with conventional teaching techniques. Mean SCT scores were pretest 41.5 (±1.7) and posttest 63 (±2.4) (P < 0.005). Clinical decision-making in prosthodontics, a mobile learning app, is an effective way to improve clinical reasoning skills for planning prosthodontic rehabilitation. It is well received by students.

BackgroundThe integration of artificial intelligence (AI) in medical education is evolving, offering new tools to enhance teaching and assessment. Among these, script concordance tests (SCTs) are well-suited to evaluate clinical reasoning in contexts of uncertainty. Traditionally, SCTs require expert panels for scoring and feedback, which can be resource-intensive. Recent advances in generative AI, particularly large language models (LLMs), suggest the possibility of replacing human experts with simulated ones, though this potential remains underexplored.ObjectiveThis study aimed to evaluate whether LLMs can effectively simulate expert judgment in SCTs by using generative AI to author, score, and provide feedback for SCTs in cardiology and pneumology. A secondary objective was to assess students’ perceptions of the test’s difficulty and the pedagogical value of AI-generated feedback.MethodsA cross-sectional, mixed methods study was conducted with 25 second-year medical students who completed a 32-item SCT authored by ChatGPT-4o (OpenAI). Six LLMs (3 trained on the course material and 3 untrained) served as simulated experts to generate scoring keys and feedback. Students answered SCT questions, rated perceived difficulty, and selected the most helpful feedback explanation for each item. Quantitative analysis included scoring, difficulty ratings, and correlations between student and AI responses. Qualitative comments were thematically analyzed.ResultsThe average student score was 22.8 out of 32 (SD 1.6), with scores ranging from 19.75 to 26.75. Trained AI systems showed significantly higher concordance with student responses (ρ=0.64) than untrained models (ρ=0.41). AI-generated feedback was rated as most helpful in 62.5% of cases, especially when provided by trained models. The SCT demonstrated good internal consistency (Cronbach α=0.76), and students reported moderate perceived difficulty (mean 3.7, SD 1.1). Qualitative feedback highlighted appreciation for SCTs as reflective tools, while recommending clearer guidance on Likert-scale use and more contextual detail in vignettes.ConclusionsThis is among the first studies to demonstrate that trained generative AI models can reliably simulate expert clinical reasoning within a script-concordance framework. The findings suggest that AI can both streamline SCT design and offer educationally valuable feedback without compromising authenticity. Future studies should explore longitudinal effects on learning and assess how hybrid models (human and AI) can optimize reasoning instruction in medical education.

Clinical reasoning is a topic that has received significant attention over the last decade, within both the academic and clinical communities. The numerous and varied publications in the Journal of Physical Therapy Education (JOPTE) reflect the profession's interest in studying clinical reasoning in physical therapy education and practice.In this issue, 4 papers address the topic of clinical reasoning. Drs Huhn and Parrott present a research report exploring the use of the Health Sciences Reasoning Test (HSRT) and academic measures (GPA and GRE scores) at the time of admission to predict physical therapist (PT) student success on the NPTE.1 This paper builds on Dr Huhn's prior work examining the utility of the HSRT in demonstrating development of critical thinking skills across PT education programs.2 Drs Venskus and Craig have developed and validated a tool to measure the degree of self-efficacy with respect to the development of clinical reasoning skills reported by PT students. This paper adds to the growing body of literature about the development of clinical reasoning, specifically addressing the student's capability to reason.3The other 2 papers in this issue are research reports that explore the development of clinical reasoning abilities. Dr. Gilliland has applied qualitative research methods to explore the development of diagnostic reasoning in PT students. She employed think aloud processes using a musculoskeletal paper case at 2 points in the students entry-level PT education to identify evolution of diagnostic reasoning.4 Drs Trommelen, Karpinski, and Chauvin studied the effectiveness of a novel instructional activity to promote reflection in a case-based delivery model for neuromuscular physical therapy.5 Like Drs Huhn and Parrott, these authors used standardized measures (the Diagnostic Thinking inventory and the Self-Assessment of Clinical Reasoning Tool) to demonstrate improvement in the student's clinical reasoning and reflective abilities. One of the challenges in conducting and applying this research is the varied methods and assessment tools that are used in studying clinical reasoning.Study of clinical reasoning in PT students has demonstrated that clinical reasoning abilities develop over time within PT curricula.6,7 Research on expertise has demonstrated the continued development of these clinical reasoning abilities beyond formal entry-level education. For example, a study of promising novices over a 2-year period as they entered practice indicated ongoing development and subsequent expansion and refinement of formal and informal learning, clinical skills, and collaboration.8 Findings such as these indicate the importance of creating opportunities to develop clinical reasoning skills as a student and novice practitioner.Another area of study in clinical reasoning has been the development of rubrics to assess clinical reasoning during practical assessment of student's clinical knowledge and skill performance. Two such assessment tools have been reported in JOPTE. The Think Aloud Standardized Patient Examination (TASPE)9 was developed to assess clinical reasoning competency in novice physical therapy students during standardized musculoskeletal patient encounters. Using the TASPE, students are assessed on their ability to synthesize data to inform think-aloud sessions across the hypothetico-deductive reasoning process: hypothesis generation, hypothesis evaluation, and intervention. A second tool, the Clinical Reasoning Grading Rubric, 10 has 2 intended purposes: (1) assess the clinical reasoning skills of PT students, and (2) evaluate readiness to progress to full-time clinical education. This tool incorporates assessment across cognitive, psychomotor, and affective domains with criteria from low to high performance of knowledge, skills, and attitudes.Another category of study is the development of skills that are components of clinical reasoning. Several studies exploring the development of critical thinking have been published in JOPTE over that past few years. …

Le test de concordance de script : un outil pédagogique multimodal

It has been argued that the best method of acquiring clinical reasoning is through seeing new out-patients. The purpose of this interventional study was to establish a clinical clerkship course for Japanese medical students in an out-patient care setting, with multiple opportunities for reflective practice and improving the clinical reasoning abilities of the students. The effectiveness of the course was also examined. Students performed examinations of new patients and made diagnostic decisions in 20minutes. They presented their case using the SNAPPS (Summarize the case, Narrow the differential, Analyze the differential, Probe the preceptor, Plan management, and Select an issue for self directed learning) method, and this was followed by feedback from faculty members using the 1-minute preceptor method and a mini clinical evaluation exercise (mini-CEX). Students' clinical reasoning abilities were assessed by the objective structured clinical examination (OSCE) and the script concordance test (SCT). Students' written comments and responses to an interview about the course were also analysed. Cross-sectional data were examined by comparing individual OSCE and SCT scores, and the multiple-choice question examination (MCQ) completed by students who did and did not participate in this project. Students in the programme had higher scores on the mini-CEX in all areas. The SCT and OSCE scores were also significantly higher than the scores for the control group. Students' comments about the course, which provided an opportunity for daily reflection, were positive. Students rapidly acquired clinical reasoning skills through reflective practice. Students also demonstrated motivation to learn through the examination of new patients. The clinical clerkship programme with multiple opportunities for reflective practice in an out-patient care setting substantially improved the clinical reasoning abilities of medical students.

BackgroundWhile educational techniques to improve clinical reasoning (CR) skills among medical students have been developed, the lack of faculty expertise in teaching CR remains a significant barrier.1 Teaching the teachers...

BackgroundClinical reasoning is an essential medical competence that should be taught and assessed from the beginning of medical studies. These skills can be evaluated using the Script Concordance Test (SCT), which presents daily clinical scenarios characterised by uncertainty. Due to the lack of validated research on this method in Germany, particularly in the field of neurology, we developed and implemented an SCT at Ruhr University Bochum. We compared different teaching methods (clinical seminar vs. digital video course) and their outcomes on the examination.MethodsA group of 6th-year medical students who had received the same education completed an SCT after participating in either a clinical seminar or a digital video course. The SCT was developed using blueprints on stroke and epilepsy. The test consisted of 40 case vignettes with a total of 120 items. Initially, experts completed the test to establish the reference panel. The final high-stakes examination was created using the aggregate scoring method and an item analysis.ResultsThe SCT was completed by 15 experts and 59 students. The final SCT consisted of 112 items and achieved a Cronbach’s alpha of 0.85. A significant difference (p < 0.05) was observed between the experts, who achieved a mean score of 81.75, and the students on the first assessment day, who achieved a mean score of 68.92. No significant differences were found between the groups (interactive video course and seminar) or assessment time points. The questionnaire revealed a sense of insecurity in clinical decision-making before the SCT and highlighted the need to incorporate clinical reasoning practices from the beginning of medical studies to mitigate fear in uncertain situations. The SCT helped students structure decision-making processes and and improved their confidence in making decisions.ConclusionThe SCT is a reliable and valid tool for assessing medical students throughout their university education. Regular exposure to the SCT format would facilitate familiarity with its structure. We propose utilising the SCT as a learning tool rather than solely for assessment purposes. For instance, it could be integrated into teaching methodologies as a think-aloud exercise or incorporated into progress tests.

Background and objectivesSkilled clinical reasoning is a critical tool for physicians. Educators agree that this skill should be formally taught and assessed. Objectives related to the mastery of clinical reasoning skills appear in the documentation of most medical schools and licensing bodies. We conducted this study to assess the differences in clinical reasoning skills in medical students following paper- and computer-based simulated instructions.Materials and methods A total of 52 sixth semester medical students of the Dow University of Health Sciences were included in this study. A tutorial was delivered to all students on clinical reasoning and its importance in clinical practice. Students were divided randomly into two groups: group A received paper-based instructions while group B received computer-based instructions (as Flash-based scenarios developed with Articulate Storyline software [https://articulate.com/p/storyline-3]) focused on clinical reasoning skills in history-taking of acute and chronic upper abdominal pain. After one week, both groups were tested at two objective structured clinical examination (OSCE) stations to assess acute and chronic pain history-taking skills in relation to clinical reasoning.ResultsThere were 27 students in group A and 25 students in group B. The mean OSCE score for group A (paper-based) was 28.6 ± 9.4 and that for group B (computer-based) was 38.5 ± 6.0. Group B’s mean score was statistically significantly greater (p < 0.001) than group A’s mean score for clinical reasoning skills. ConclusionA computer simulation program can enhance clinical reasoning skills. This technology could be used to acquaint students with real-life experiences and identify potential areas for more training before facing real patients.

Residents in tutored practice exchange groups have better medical reasoning as measured by script concordance test: a controlled, nonrandomized study

Programmes of assessment should measure the various components of clinical competence. Clinical reasoning has been traditionally assessed using written tests and performance-based tests. The script concordance test (SCT) was developed to assess clinical data interpretation skills. A recent review of the literature examined the validity argument concerning the SCT. Our aim was to provide potential users with evidence-based recommendations on how to construct and implement an SCT. A systematic review of relevant databases (MEDLINE, ERIC [Education Resources Information Centre], PsycINFO, the Research and Development Resource Base [RDRB, University of Toronto]) and Google Scholar, medical education journals and conference proceedings was conducted for references in English or French. It was supplemented by ancestry searching and by additional references provided by experts. The search yielded 848 references, of which 80 were analysed. Studies suggest that tests with around 100 items (25-30 cases), of which 25% are discarded after item analysis, should provide reliable scores. Panels with 10-20 members are needed to reach adequate precision in terms of estimated reliability. Panellists' responses can be analysed by checking for moderate variability among responses. Studies of alternative scoring methods are inconclusive, but the traditional scoring method is satisfactory. There is little evidence on how best to determine a pass/fail threshold for high-stakes examinations. Our literature search was broad and included references from medical education journals not indexed in the usual databases, conference abstracts and dissertations. There is good evidence on how to construct and implement an SCT for formative purposes or medium-stakes course evaluations. Further avenues for research include examining the impact of various aspects of SCT construction and implementation on issues such as educational impact, correlations with other assessments, and validity of pass/fail decisions, particularly for high-stakes examinations.

Recent reviews have claimed that the script concordance test (SCT) methodology generally produces reliable and valid assessments of clinical reasoning and that the SCT may soon be suitable for high-stakes testing. This study is intended to describe three major threats to the validity of the SCT not yet considered in prior research and to illustrate the severity of these threats. We conducted a review of SCT reports available through the Web of Science database. Additionally, we reanalysed scores from a previously published SCT administration to explore issues related to standard SCT scoring practice. Firstly, the predominant method for aggregate and partial credit scoring of SCTs introduces logical inconsistencies in the scoring key. Secondly, our literature review shows that SCT reliability studies have generally ignored inter-panel, inter-panellist and test-retest measurement error. Instead, studies have focused on observed levels of coefficient alpha, which is neither an informative index of internal structure nor a comprehensive index of reliability for SCT scores. As such, claims that SCT scores show acceptable reliability are premature. Finally, SCT criteria for item inclusion, in concert with a statistical artefact of the SCT format, cause anchors at the extremes of the scale to have less expected credit than anchors near or at the midpoint. Consequently, SCT scores are likely to reflect construct-irrelevant differences in examinees' response styles. This makes the test susceptible to bias against candidates who endorse extreme scale anchors more readily; it also makes two construct-irrelevant test taking strategies extremely effective. In our reanalysis, we found that examinees could drastically increase their scores by never endorsing extreme scale points. Furthermore, examinees who simply endorsed the scale midpoint for every item would still have outperformed most examinees who used the scale as it is intended. Given the severity of these threats, we conclude that aggregate scoring of SCTs cannot be recommended. Recommendations for revisions of SCT methodology are discussed.

The objective of this review was to determine whether the use of Personal Digital Assistants (PDAs) would provide greater support in developing undergraduate nursing students' clinical reasoning, in comparison to more traditional resources such as textbooks. The search strategy sought to identify published data from five electronic databases: CINAHL, Medline, Cochrane Library, Meditext and Scopus. Unpublished literature was also sought in digital dissertations, conference proceedings, relevant scholarly websites and reference lists. All undergraduate nursing students were considered eligible for inclusion. Types of interventions considered for this review were inclusive of all forms of PDAs and traditional resources. The research setting of this systematic review reflects the diversity of nursing practice, and includes the classroom, clinical or simulated environment. The process of clinical reasoning was defined by four outcome measures; alterations in theoretical nursing knowledge, clinical skills, problem solving and reflection. Studies of potential significance to the review were assessed for methodological quality independently by two reviewers using the Joanna Briggs Institute Meta Analysis of Statistics Assessment and Review Instrument. Authorship of the studies was not concealed from the two reviewers. From the nine studies assessed for quality, only data from four studies were included in the review. Four published studies were included in the systematic review of literature. The designs of the studies included a nonrandomised quasi-experimental design, case control study, comparative descriptive design and a pre test post test mixed method study. Four outcomes were identified by the four included studies. These outcomes addressed possible effects of PDA usage on undergraduate nursing students' practice of medication administration, self-efficacy, anticipation to exercise professional nursing judgment and clinical reasoning.This systematic review provides evidence that the use of PDAs is able to improve nursing students' self-efficacy and accuracy in clinical situations that require direct and context-free answers, such as medication administration, but is not as supportive as textbooks in assisting students to apply this knowledge critically in decision making and problem solving. The use of PDAs by undergraduate nursing students can improve students' confidence in the often stressful clinical environment. This bears significance in contemporary nursing education where undergraduates are failing to maximise the clinical experience because of insufficient support or guidance from busy clinicians or supervisory staff. From these findings, PDAs are also beneficial in improving the accuracy and efficiency of medication administration in nursing students. However, the application of the knowledge provided by PDAs is not supported to be used critically in its application to decision making and problem solving. Nursing students need to be educated in methods to develop critical analysis in order to use these resources effectively. This systematic review highlights the poor quantity of literature currently available in nursing and the subsequent need for primary quantitative studies examining the effect of PDAs in developing undergraduate nursing students' clinical reasoning skills.

The COVID‐19 pandemic created unprecedented challenges worldwide that required rapid adaptation and transformation across the entire healthcare system. Graduate medical training programs across all specialties have moved to rapidly adjust to the virtual landscape. This created a unique opportunity for genetic counselors who work in industry and within diagnostic laboratories to develop internship and rotation programs that can be offered virtually to meet the needs of genetic counseling training programs. Myriad Genetics, Inc., was contacted by numerous graduate programs in genetic counseling beginning in March 2020 requesting the opportunity for their students to participate in remote laboratory‐based rotations. As a result of these requests, a working group of genetic counselors across Myriad came together to adapt existing experiences to fully remote formats and develop new remote‐based opportunities for students. We describe our experience of expanding genetic counseling student rotations during the COVID‐19 pandemic with the goal of providing examples of remote learning experiences that may be applicable to other diagnostic laboratory industry‐based rotations for genetic counseling students. In 2020, a total of 59 second‐year genetic counseling students, from 21 different genetic counseling training programs, participated in one of five different virtual experiences. Furthermore, two new rotation experiences were created to increase capacity and highlight diversity of industry roles. Genetic counselors in industry are uniquely positioned to provide both remote training opportunities for genetic counseling students and exposure to the variety of roles that genetic counselors can occupy. Increasing the exposure to these roles is important as the genetic counseling workforce continues to expand and diversify, and it is imperative among all programs to enable access to these opportunities.

ABSTRACTProblem: Clinical reasoning is a necessary skill for medical students to acquire in the course of their education, and there is evidence that they can start this process at the undergraduate level. However, physician educators who are experts in their given fields may have difficulty conveying their complex thought processes to students. Providing faculty development that equips educators with tools to teach clinical reasoning may support skill development in early medical students. Intervention: We provided faculty development on a modified Bayesian method of teaching clinical reasoning to clinician educators who facilitated small-group, case-based workshops with 2nd-year medical students. We interviewed them before and after the module regarding their perceptions on teaching clinical reasoning. We solicited feedback from the students about the effectiveness of the method in developing their clinical reasoning skills. Context: We carried out this project during an institutional curriculum rebuild where clinical reasoning was a defined goal. At the time of the intervention, there was also increased involvement of the Teaching and Learning Center in elevating the status of teaching and learning. Outcome: There was high overall satisfaction with the faculty development program. Both the faculty and the students described the modified Bayesian approach as effective in fostering the development of clinical reasoning skills. Lessons Learned: Through this work, we learned how to form a beneficial partnership between a clinician educator and Teaching and Learning Center to promote faculty development on a clinical reasoning teaching method for early medical students. We uncovered challenges faced by both faculty and early learners in this study. We observed that our faculty chose to utilize the method of teaching clinical reasoning in a variety of manners in the classroom. Despite obstacles and differing approaches utilized, we believe that this model can be emulated at other institutions to foster the development of clinical reasoning skills in preclerkship students.

PurposeGraduating medical students need broad clinical diagnostic reasoning skills that integrate learning across clinical specialties to deal with undifferentiated patient problems. The opportunity to acquire these skills may be limited during clinical placements on increasingly specialized hospital wards. We developed an intervention of regular general practitioner (GP) facilitated teaching in hospital placements to enable students to develop broad clinical diagnostic reasoning. The intervention was piloted, refined and delivered to a whole cohort of medical students at the start of their third year. This paper examines whether students perceived opportunities to improve their broad diagnostic clinical reasoning through our intervention.MethodsGP-facilitated teaching sessions were delivered weekly in hospital placements to small groups of 6–8 students for 90 mins over 6 weeks. Students practiced clinical reasoning with real patient cases that they encountered on their placements. Evaluation of learning outcomes was conducted through a student questionnaire using Likert scales with free-text boxes for additional explanation. Focus groups were conducted to gain a more in-depth understanding of student perspectives.ResultsAs high as 87% of students agreed that their broad clinical diagnostic reasoning ability had improved. Thematic analysis of the qualitative data revealed four factors supporting this improvement: practicing the hypothetico-deductive method, using real patient cases, composing student groups from different speciality placements and the breadth of the facilitators’ knowledge. Students additionally reported enhanced person-centredness in terms of understanding the patient’s perspective and journey. Students perceived that the added value of general practitioner facilitators lay in their broad knowledge base and knowledge of patient needs in the community.ConclusionOur results suggest that medical students can develop broad clinical diagnostic reasoning skills in hospital settings through regular GP-facilitated teaching. Our approach has the advantage of working within the established curricular format of hospital placements and being deliverable at scale to whole student cohorts.