ObjectiveAssessment at medical schools plays a crucial role by providing feedback, monitoring student promotion, and informing resident selection. Limited research has been conducted to synthesize key features of assessments, and even less is known about how these studies sought validity and reliability evidence. A scoping review was performed to explore key features of assessments and their validity and reliability evidence.MethodsVarious databases were searched ranging from 2004 to April 2021 following PRISMA guidelines. In-depth reviews of the full text were performed on all selected empirical studies during the extraction phase.ResultsThe final analysis included 218 empirical studies. Various tools were identified, ranging from traditional multiple-choice questions to more contemporary tools incorporating technology and more contextualized workplace-based assessment. Patient care and medical knowledge were the most frequently assessed, primarily using the quantitative methodology. These studies often adopted traditional perspectives in collecting validity evidence based on relations to other variables and internal structure. Many of them used a narrow conceptualization of validity, with some failing to reference validity at all.ConclusionsThere was no single assessment that could easily differentiate medical students in a standardized, meaningful way. Despite the existence of contemporary validity theories for over two decades, there remains a need for greater education regarding the pivotal role of validity in conducting assessment research. While psychiatry and other residency programs will continue to face challenges in differentiating applicants, these changes present opportunities for medical educators and schools to validate assessments that are highly contextualized to their specific educational environments.

ObjectiveAdults with intellectual and developmental disabilities (IDD) experience high rates of poor mental health and challenges accessing timely and high-quality services. There is limited interprofessional training on mental health care for this population.MethodsA virtual, synchronous program based on the Project Extension for Community Healthcare Outcomes (ECHO) Ontario IDD Mental Health program was developed for health and social service trainees. Participants represented 10 disciplines across 12 Canadian university or college programs. The program was taught by a team of health and social service providers together with individuals with lived experience and included didactics and case-based discussions. Program evaluation utilized a pre-, post-, and 12-week follow-up survey design with feedback surveys following each session.ResultsFifty participants registered for the program; 34 (68%) completed baseline measures and attended at least two sessions. Overall, participants reported high session satisfaction (average rating of 4.47 of 5). Participants demonstrated significant improvement in self-efficacy regarding communication (p < 0.001), management of mental health needs (p < 0.001), and working across systems (p < 0.001). Participants self-reported feeling more knowledgeable about common comorbidities (p < 0.001), assessing behavioral challenges (p < 0.001), the role of interdisciplinary professionals (p < 0.001), and community resources (p < 0.001). Improvements were maintained at follow-up across outcomes.ConclusionThe pilot Project ECHO for health and social service trainees in adult IDD mental health demonstrated high participant satisfaction and positive impact on trainees’ self-efficacy and knowledge. Interprofessional educational interventions can be effectively implemented using virtual technology to teach about other mental health populations requiring multisector care.

Working with suicidal patients can elicit negative emotional responses that can impede clinicians' empathy and affect clinical outcomes. Virtual human interactions represent a promising tool to train clinicians. The present study investigated the impact of virtual human interaction training to enhance clinicians' emotional self-awareness and empathy when working with suicidal patients. Clinicians were randomly assigned into two groups. Both groups interviewed a virtual patient presenting with a suicidal crisis; clinicians in the intervention condition (n = 31) received immediate feedback about negative emotional responses and empathic communication, whereas those in the control condition (n = 33) did not receive any feedback. All clinicians interviewed a second virtual patient 1week later. Clinicians' emotional response to the two virtual patients and their empathic communication with each of them were assessed immediately after each interaction. Linear mixed models were used to assess change in clinicians' emotional response and verbal empathy between the two interactions across conditions. Clinicians' emotional responses toward the suicidal virtual patients were unchanged in both conditions. Clinicians in the intervention condition presenting low empathy level with the first virtual patient showed higher empathy level with the second virtual patient than with the first (B = 1.15, SE = 0.25, p < 0.001, 95% CI [0.42, 1.89]). This work demonstrates the feasibility of using virtual human interactions to improve empathic communication skills in clinicians with poor empathy skills. Further refinement of this methodology is needed to create effective training modules for a broader array of clinicians.

The purpose of this study is to explore the impact of board-certified psychiatric pharmacist (BCPP)-led psychopharmacology lectures to psychiatry residents and fellows. Surveys were administered to psychiatry residents and geriatric psychiatry fellows at two teaching institutions between Fall 2021 and Spring 2023, including two distinct residency programs and one fellowship program. The survey consisted of three quantitative questions and one qualitative question soliciting open-ended constructive feedback. Of 39 participants (response rate: 80%), 100% strongly agreed that learning from a BCPP enhanced their learning of psychopharmacology concepts. Additionally, 100% strongly agreed they would recommend psychopharmacology lectures from a BCPP to other psychiatry residents and that concepts taught by the BCPP were applicable to their clinical practice. Qualitative feedback indicated valuing pharmacist input and stated preference to learn from medication-experts on psychopharmacology topics. Integrating BCPPs into psychiatry resident/fellow didactic training is well received by psychiatry residents and may simultaneously enhance education of psychopharmacologic concepts in addition to enrichment of interprofessional experiences by increased routine exposure to working directly with a clinical pharmacist. Program directors are encouraged to meet with BCPPs at their respective institutions to discuss opportunities for collaboration.