Evaluating Curricular Modules in the Care of Children With Medical Complexity: A Mixed-Methods Randomized Controlled Trial

A 2011 study evaluating oral health training in pediatric medical residency programs highlighted opportunities to improve residents' oral health competence. The aim of this 2017 follow-up study was to assess progress since 2011 in promoting development of pediatric residents' oral health competence. A survey was sent to all 281 pediatric and med-peds medical residency program directors to assess the structure and determinants of oral health training in their programs. A total of 110 program directors responded to the 2017 survey (response rate 40%). Just over half (54%) of the respondents reported that their residents received one to two hours of oral health education during residency, while 38% received ≥3 hours (compared to 47% and 48%, respectively, in 2011). There was improvement in the proportion of respondents reporting that their residents integrated fluoride varnish application training in 2017 vs. 2011 (83% vs. 43%; p<0.001). Departmental support for integrating oral health and having an oral health elective were significantly associated with satisfaction with residents' oral health competence. These findings suggest that pediatric and med-peds medical residency programs have improved their residents' oral health awareness and training through integration of at least one hour of oral health education and fluoride varnish training. However, given deficiencies in program structure and uneven perceptions of residents' oral health competence found in this study, opportunities remain to improve pediatric residency programs' oral health curricula.

To evaluate the usefulness of an objective structured clinical examination (OSCE) for assessing and providing feedback to postgraduate residents in pediatrics. A 5-station OSCE given in 1996, based on the educational objectives of a general pediatric training program. Each station assessed the residents' interviewing and history-taking skills with a standardized patient. The results were correlated with those of the in-training evaluation reports. The Department of Paediatrics, University of Toronto Faculty of Medicine, Toronto, Ontario. Forty-three of 61 pediatric residents: 14 first-year, 12 second-year, 8 third-year, and 9 fourth-year residents. Scores for each of the 5 stations were broken down into 15 points for the check-list, 5 for the global assessment, and 10 for the written postencounter question, for a total score of 150. The in-training evaluation report ratings were converted to a 5-point numerical scale, ranging from 1 (unsatisfactory) to 5 (outstanding). The mean OSCE score for the 43 pediatric residents was 104.9. Although the residents in their senior year scored higher, there was no statistically significant difference among the 4 years for the total OSCE score or for any of the 5 stations. The fourth-year residents' scores on the postencounter questions were significantly (P < .05) higher than the first-year residents' scores. Two residents scored less than 60%. The internal consistency of the 5-station OSCE was limited (r = 0.69). Residents received verbal feedback at the conclusion of the OSCE, and they received their scores when they were calculated. The mean overall in-training evaluation report score for all 61 pediatric residents was 3.9. There was a moderate, but statistically significant, correlation between the overall mean OSCE results and the overall mean in-training evaluation report scores (r = 0.45). The OSCE can provide a useful formative evaluation of postgraduate residents, but the usefulness of the evaluation data and the feedback must be balanced with the logistic difficulties and expense.

Abstract: Introduction: A medical residency is the specialization par excellence in medical training and the program is responsible for ensuring that the recently graduated doctor reaches the established level of competence. A well-prepared assessment system with feedback is an effective tool to enhance the performance of future specialists and guarantee their qualification. Objective: To analyze the assessment system for pediatric residents at a university hospital, aiming to promote teaching training in assessment methods. Methodology: educational action research (research-teaching), conducted with teachers and preceptors in the pediatric residency of a university hospital. The stages consisted of: a) the application of a questionnaire on participant profile and assessment methods used to evaluate the medical residents; b) intervention (workshop) on the assessment of clinical skills and feedback; c) immediate assessment after the workshop, by applying another questionnaire based on level 1 of the Kirkpatrick model. Simple statistical analysis was used for the objective data and the content analysis, according to recommendations by Malheiros (2011) and Bardin (2013), for the qualitative part. Result: Ten (48%) of the 21 participants declared not being formally trained in assessment and that they applied the more traditional methods learned from personal experience. Regarding the assessment methods, 81% (17/21) of the participants used more than one, aiming to obtain a more encompassing and reliable assessment. However, none of the teachers/preceptors used a systematized assessment of the clinical skills or provided feedback to the medical residents. After the workshop, with a focus on assessing performance in a simulated ‘Objective Structured Clinical Examination’ (OSCE) environment, all the participants were favorable to apply the OSCE in their teaching-learning practice with residents and undergraduate students, reaching Kirkpatrick level 3. Conclusion: The action research helped identify limitations in the assessment and feedback system of the pediatric resident physician. The methodology used revealed an aggregating effect and contributed to the development of the collaborative and integrative sense in the group. However, it was not enough to positively interfere with the pediatric medical residency assessment in the short term.

Anesthesiology residency programs may need new simulation-based programs to prepare residents for the new Objective Structured Clinical Examination (OSCE) component of the American Board of Anesthesiology (ABA) Primary Certification process. The design of such programs may require significant resources, including faculty time, expertise, and funding, as are currently needed for structured oral examination (SOE) preparation. This survey analyzed the current state of US-based anesthesiology residency programs regarding simulation-based educational programming for SOE and OSCE preparation. An online survey was distributed to every anesthesiology residency program director in the United States. The survey included 15 to 46 questions, depending on each respondent's answers. The survey queried current practices and future plans regarding resident preparation specifically for the ABA APPLIED examination, with emphasis on the OSCE. Descriptive statistics were summarized. χ and Fisher exact tests were used to test the differences in proportions across groups. Spearman rank correlation was used to examine the association between ordinal variables. The responding 66 programs (49%) were a representative sample of all anesthesiology residencies (N = 136) in terms of geographical location (χ P = .58). There was a low response rate from small programs that have 12 or fewer clinical anesthesia residents. Ninety-one percent (95% confidence interval [CI], 84%-95%) of responders agreed that it is the responsibility of the program to specifically prepare residents for primary certification, and most agreed that it is important to practice SOEs (94%; 95% CI, 88%-97%) and OSCEs (89%; 95% CI, 83%-94%). While 100% of respondents reported providing mock SOEs, only 31% (95% CI, 24%-40%) of respondents provided mock OSCE experiences. Of those without an OSCE program, 75% (95% CI, 64%-83%) reported plans to start one. The most common reasons for not having an OSCE program already in place, and the perceived challenges for implementing an OSCE program, were the same: lack of time (faculty and residents), expertise in OSCE development and assessment, and funding. The results provide data from residency programs for benchmarking their simulation curriculum and ABA APPLIED Examination preparation offerings. Despite agreement that residency programs should prepare residents for the ABA APPLIED Examination, many programs have yet to implement an OSCE preparation program, in part due to lack of financial resources, faculty expertise, and time. Additionally, in contrast to the SOE, the OSCE is a new format for ABA primary certification. As a result, the lack of consensus concerning preparation needs could be related to the amount information that is available regarding the examination content and assessment process.

Pediatric and Adolescent Gynecology Education through Simulation (PAGES): Development and Evaluation of a Simulation Curriculum

Background: Examination methods change over time, and audits are useful for quality assurance and improvement. Objective: Comparison of traditional clinical test and objective structured clinical examination (OSCE) in a department of surgery. Methods: Examination records of results of the fifth year MBChB examinations for 2012–2013 (traditional) and 2014–2015 (OSCE) were analyzed. Using 50% as the pre-agreed pass mark, the pass rate for the clinical examinations in each year was calculated and these figures were subjected to t-test to determine any significant differences in each year and in type of clinical test. P value of &lt;0.05 determined significant statistical differences in the test score. Results: We analyzed 1178 results; most (55.6%) did OSCE. The average clinical scores examinations were 59.7% for traditional vs 60.1% for OSCE examination; basic surgical skills were positively skewed. Conclusion: OSCE in the same setting of teaching and examiners may give more marks than the traditional clinical examination, but it is better at detecting areas of inadequacies for emphasis in teaching. Keywords: Clinical examination, Traditional, OSCE, Comparison

American association for cancer education had emphasized the role of structured educational programs for medical students and residents in primary care specialties in order to improve palliative oncology education. Dissatisfaction with the conventional methods of clinical assessment on the part of teachers and students led assessors to search for appropriate alternatives and in 1975, Harden and his colleagues introduced the objective structured clinical examination (OSCE). OSCE was introduced as a standardized tool for objectively assessing clinical competencies−including history-taking, physical examination, communication skills, data interpretation, etc. It consists of a circuit of stations connected in series, with each station devoted to the assessment of a particular competency using pre-determined guidelines or checklists. The Structured Clinical Instruction Module (SCIM) modifies the OSCE for teaching purposes. The objective of this review is to provide a focused update on the status and applicability of SCIM and OSCE in cancer for educational use in palliative care. From the 12 studies which were on OSCE and 6 studies which were on SCIM, it appears that the two competency-based evaluation methodologies used in cancer education namely the OSCE and the SCIM are well validated and reliably used across settings and samples of students, practitioners, and patients. Future studies in Indian palliative care settings are warranted prior to extrapolation of existing evidence.

Introduction Professionalism training and assessment have received much attention. Multiple strategies have been devised to help educational programs deal with unprofessional behavior, which is not always detected by attending staff. Trainees are in a unique position to address unprofessional behavior in their colleagues because of their supervisory role and overlapping working hours. To underline the importance of addressing unprofessional behavior, we created two professionalism objective structured clinical exam (OSCE) stations that focused on confronting a junior level trainee's unprofessional behavior. These two activities are termed the Communications OSCE, for first-year pediatric residents, and the Culture OSCE, for second-year pediatric residents. Methods Prior to the encounter, participants read an instruction sheet and view a short video. In the “Communication with a Colleague” OSCE, participants must confront a junior colleague who treated an ambulance driver disrespectfully. In the “Pediasure Please!” OSCE they have to address a junior colleague who made racist comments about a patient to a medical student. OSCE participants have 10 minutes to complete the task and receive immediate feedback from an observing faculty member and the standardized resident. Post-OSCE group debriefing reinforces the teaching objectives. Following the completion of the OSCE, residents were surveyed anonymously and again 6 to 10 months later to evaluate the effects of the OSCE learning experience. Results The residents were surveyed immediately after the completion of the OSCE. The educational value of the stations was rated moderate to high by 100% and 96% of the first- and second-year students (N = 96), respectively. Six to 10 months after the completion of the exercise, 88% and 76% of the residents still felt that they were to some degree comfortable in dealing with a similar unprofessional encounter as depicted in the OSCE and 88% and 65% in a dissimilar unprofessional encounter. Discussion Our professionalism OSCE station only addresses residents confronting junior residents' unprofessional behaviour. It does not deal with unprofessional behaviour in senior residents, attending staff, or ancillary staff. Our hope is that as the residents graduate to become attending staff, they will take the lessons learned with them and apply them at a higher level of leadership.

Upholding Our PROMISE: Negotiating Belonging and Professional Identity Formation in Underrepresented in Medicine Pediatric Residents.

Introduction: Effective patient-centered communication in a physician-patient relationship is challenging, and these skills may need reinforcement as faculty progress in their career. Formal curricula to assess and enhance patient-centered care skills are lacking for practicing physicians. Methods: Eight GI faculty participated in an IRB-approved pilot program which included a four-station OSCE: 1) “Breaking Bad News” - a fatalistic patient undergoes a screening colonoscopy and is found to have a likely malignancy; 2) “Apologizing for a Complication” - a patient who mistrusts the medical system has a perforation during a screening colonoscopy; 3) “Geriatric IBD Case” - an elderly patient newly diagnosed with IBD requires complex outpatient care at the time of discharge; 4) “Giving Sign-Out” - a clinically depressed colleague with alcohol dependence is in crisis upon arrival for her shift. Trained standardized patients/physicians (SPs) assessed the faculty's performance using a previously validated behaviorally anchored checklist. Immediately following each case, the SP provided verbal feedback to the participant. Faculty completed a survey after each encounter about the relevance of the scenarios. Each station was videotaped, and faculty reviewed and assessed their videotaped performance using checklists. All participants completed pre- and post-OSCE surveys. Results: 25% of faculty participants have been in practice for 10+ years, 25% for 6-10 years, and 50% for 1-5 years. 5/8 participants strongly agreed that faculty development program initiatives are a worthwhile endeavor prior to the OSCE, compared with 8/8 post-OSCE. Pre-OSCE, 75% strongly agreed that faculty development programs serve a personal and professional benefit, compared with 100% postintervention. Pre-OSCE, only 2/8 agreed that OSCEs are a worthwhile method of learning, compared with 7/8 strongly agreeing with this attestation post-OSCE. Close to 90% would recommend this faculty development OSCE to their colleagues. Data on the SPs' assessment of faculty performance, and the faculty's self-assessment, are pending. Conclusion: This experiential faculty development pilot program using performance-based assessment and self-review was very well-received, and has potential to be expanded as a formal curriculum for all faculty in clinical academic practice.

Assessing residents' understanding and application of the 6 intrinsic CanMEDS roles (communicator, professional, manager, collaborator, health advocate, scholar) is challenging for postgraduate medical educators. We hypothesized that an objective structured clinical examination (OSCE) designed to assess multiple intrinsic CanMEDS roles would be sufficiently reliable and valid. The OSCE comprised 6 10-minute stations, each testing 2 intrinsic roles using case-based scenarios (with or without the use of standardized patients). Residents were evaluated using 5-point scales and an overall performance rating at each station. Concurrent validity was sought by correlation with in-training evaluation reports (ITERs) from the last 12 months and an ordinal ranking created by program directors (PDs). Twenty-five residents from postgraduate years (PGY) 0, 3 and 5 participated. The interstation reliability for total test scores (percent) was 0.87, while reliability for each of the communicator, collaborator, manager and professional roles was greater than 0.8. Total test scores, individual station scores and individual CanMEDS role scores all showed a significant effect by PGY level. Analysis of the PD rankings of intrinsic roles demonstrated a high correlation with the OSCE role scores. A correlation was seen between ITER and OSCE for the communicator role, while the ITER medical expert and total scores highly correlated with the communicator, manager and professional OSCE scores. An OSCE designed to assess the intrinsic CanMEDS roles was sufficiently valid and reliable for regular use in an orthopedic residency program.

As point-of-care ultrasound (POCUS) evolves into a standard tool for the care of children, pediatric residency programs need to develop POCUS training programs. Few POCUS training resources exist for pediatric residents, and little is known about POCUS training in pediatric residencies. We aim to describe pediatric residency leadership perspectives regarding the value of POCUS and to elucidate the current state of POCUS training in pediatric residency programs. A group of pediatric educators and POCUS experts developed a novel survey followed by cognitive interviews to establish response-process validity. The survey was administered electronically to pediatric residency associate program directors between December 2019 and April 2020. Program characteristics, including region, setting, and size, were used to perform poststratification for analyses. We performed comparative analyses using program and respondent characteristics. We achieved a 30% (58 of 196) survey response rate. Although only a minority of respondents (26%) used POCUS in clinical practice, a majority (56%) indicated that all pediatric residents should be trained in POCUS. A majority of respondents also considered 8 of 10 POCUS applications important for pediatric residents. Only 37% of programs reported any POCUS training for residents, primarily informal bedside education. Most respondents (94%) cited a lack of qualified instructors as a barrier to POCUS training. Most pediatric residency programs do not provide residents with POCUS training despite its perceived value and importance. Numerous POCUS applications are considered important for pediatric residents to learn. Future curricular and faculty development efforts should address the lack of qualified POCUS instructors.

Breastfeeding Education and Support Services Offered to Pediatric Residents in the US

Introduction: Children with medical complexity have high care needs, functional limitations, and require frequent hospital presentations. This can be challenging for both children and their families. Despite often being well known to specialty and allied health teams involved in their care, carers reported frustrations with the need to continually provide their child’s medical history to multiple clinicians. Carers also report difficulty recalling information due to their child’s very complex detailed medical histories and lack of continuity with high turnover of healthcare staff. Clinicians also reported relevant information being scattered and difficult to find in the child’s electronic medical record (eMR). Aim: To develop a care management plan easily accessible for the cohort of General Medicine Chronic and Complex patients in collaboration with carers. Method: A survey was sent to 52 carers in February 2019 whose child was enrolled in Chronic and Complex Care at the time. This identified that all carers felt that a care plan would be of benefit to improve communication and care coordination. A team of clinicians from General Medicine came together to pilot the Chronic and Complex Individualized Care Plan – CCICP, working with the eMR team to develop a template. The template was first trialled using paper and then embedded in the eMR as a management care plan. Throughout template development, two Chronic and Complex Care Clinical Nurse Consultants (CNC) worked with carers seeking feedback on the content and the usability of the CCICP. The CCICP included a comprehensive summary of the patient’s demographics, diagnoses, medications, clinicians involved in care and specific management plans. All CCICPs were reviewed by carers and their primary lead clinicians prior to completion. The CCICP would be reviewed regularly at clinic appointments, admissions and updated with relevant new information. Evaluation of the CCICP was conducted via survey and informal feedback during care encounters. Key Findings: To date, 25 CCICPs have been completed. Progress was impacted by redeployment of staff during the COVID-19 Pandemic response. Interviews were conducted with five families and whilst an evaluation survey was sent to clinicians, there was a low response rate. However, verbal and informal feedback regarding the CCICP has been positive. Carers and clinicians reported ease of accessibility with the care plan uploaded to the eMR and printed copies given to carers. Benefits included improved communication between teams and families, more accurate clinical documentation. The care plans empowered carers to advocate for their child’s care, and reduced stress and burden of information repetition. Despite overwhelming benefits for the CCICP, resource limitations exist with the time taken to develop, complete and update the care plans in collaboration with carers. Future digital integration would enable a more dynamic document in the patient record accessible across community and private sector healthcare.

The outbreak of COVID-19 necessitated a move to online teaching and assessment. The objective structured clinical examination (OSCE) has been an integral part of dental examinations for several decades. The COVID-19 pandemic stopped face to face examinations around the world. An Online Virtual OSCE (VOSCE) was developed and piloted for dental undergraduate assessment. This initial report outlines the steps required to run an OSCE online. Planning and preparation required in adaptation of an OSCE for the online environment is dreported. The necessity for familiarisation sessions is addressed, and VOSCE logistics described. With careful planning, the VOSCE is a useful assessment method in difficult times. Feedback from staff and students was favourable. Although significant organisation was required, the examination process worked well for both students and examiners. Despite limitations in relation to technical dental procedures, the VOSCE could be a viable alternative to face to face clinical examination.