Video assessment of basic surgical trainees' operative skills

We thank Ahmad and colleagues for their positive comments regarding our article and share in their concern about the quality and quantity of training to which basic surgical trainees are exposed. Although their suggestion that basic trainees should be exposed to more laparoscopic cases is laudable, we are concerned as to how this would work in practice. Increasing amounts of the SHOs' time are now given over to acute day- and night-time service provision, which includes cross-cover for a number of specialities. This environment offers little in the way of training opportunities and we are finding that SHOs are increasingly unavailable to involve themselves in simple out-of-hours operations like abscess drainage or appendicectomy. Moreover, this schedule of acute commitments, along with enforced leave to maintain compliant weekly hours, is compressing the day-time elective exposure of trainees. This includes access to CEPOD lists along with the opportunity to pick up operative experience (laparoscopic or otherwise). As a result, we are now finding that some newly appointed higher surgical trainees are lacking experience in basic surgical procedures. This perpetuates the problem as the higher trainees then utilise training opportunities of interest to the basic trainees, in order to acquire proficiency in these procedures. We believe that any solution must ultimately lie in freeing up training opportunities for basic trainees and providing them with their own dedicated training lists.

Accurate and appropriate assessment of surgical trainees requires clear determination of the skills needed for surgical competence. This study was designed to identify those skills, rank them in order of importance and translate them into behavioural terms. A Delphi technique, using anonymous postal questionnaires, was used. All consultant surgeons in South-East Scotland were asked to identify the skills they expected of surgical trainees. Skills identified were then returned to all consultants for weighting. Differences among specialties in the importance of each item were identified using analysis of variance. The qualities identified fell into five domains: technical skills, clinical skills, interaction with patients and relatives, teamwork, and application of knowledge. Consultants from all specialties gave high weightings to the generic domains of clinical skills, teamwork, and interaction with patients and relatives. This study has identified the skills considered necessary by consultant surgeons in Scotland for a successful surgical career. Contrary to expectation, consultant surgeons value many generic skills more highly than technical skills, indicating that they value well rounded doctors, not just those with technical ability. The characteristics identified are being used to develop an assessment tool for use on basic surgical trainees.

Enhancing neonatal thoracoscopic surgical training with rabbit model

Assessing the educational environment in the operating room—a measure of resident perception at one Canadian institution

Background: After the introduction of simulators in urology, urology education has evolved significantly. The development of Transurethral Resection of Prostate (TURP) skills has shifted to simulator-based learning from being directly on patients. We evaluated the outcome of simulation-based teaching of TURP procedural skills on virtual reality simulator and objectively assessed the operative skills on the Global Rating Scale (GRS). Methodology: In this prospective comparative study, high fidelity (VirtaMed | UroSim™) simulator was used. A total of 17 participants were enrolled in this study, 4 were urology faculty members, and 13 were post-graduate urology trainees. All participants performed five attempts of real-time TURP complete procedure and were allowed to resect the maximum possible prostate tissue (goal ≥ 85%). Skills were assessed in terms of resection, bleeding control and safety parameters. Objective assessment of simulative operative skills was done on GRS score sheets by faculty, based on the Global Rating Index for Technical Skills (GRITS) and OSATS. The data from the 1st and 5th attempts of all participants was analyzed on SPSS version 22.0. Results: There was statistically significant difference in the mean resection score at 1st attempt between faculty and trainees i.e. 112±8.6 and 86.69±9.95, respectively (p<0.01), while the scores were comparative by the 5th attempt. All the enrolled members resected >80% of prostate volume (40 score points). Similarly, for the parameters associated with bleeding control, there was a significant difference in the bleeding control, lost blood and remaining bleeders between trainees and faculty members (p<0.01) at 1st attempt while on 5th attempt, the outcomes were comparative in both groups. The objective assessment of operative skills of the trainee’s displayed improvement in all 5 parameters including tissue injury, movement and time, instrument application, surgical process and surgical details. Conclusion: Simulation-based TURP teaching significantly improves simulative operative skills in resection, bleeding control and safety parameters. GRS is an effective tool in assessing simulative operative skills and may evaluate endo-urological procedural training in urology trainees periodically.

To ensure satisfactory clinical outcomes, surgical skill assessment must be objective, time-efficient, and preferentially automated—none of which is currently achievable. Video-based assessment (VBA) is being deployed in intraoperative and simulation settings to evaluate technical skill execution. However, VBA is manual, time-intensive, and prone to subjective interpretation and poor inter-rater reliability. Herein, we propose a deep learning (DL) model that can automatically and objectively provide a high-stakes summative assessment of surgical skill execution based on video feeds and low-stakes formative assessment to guide surgical skill acquisition. Formative assessment is generated using heatmaps of visual features that correlate with surgical performance. Hence, the DL model paves the way for the quantitative and reproducible evaluation of surgical tasks from videos with the potential for broad dissemination in surgical training, certification, and credentialing.

Basic mathematical operations are fundamental for numeracy learning in elementary education, yet many fourth-grade students still struggle with addition, subtraction, and multiplication, especially in complex tasks. Previous methods often focus on final results and rarely apply systematic formative assessment at the beginning of instruction to support both numeracy skills and learning effort. This study examined Mini Quizzes as an early formative assessment strategy for Grade IV students at SD Muhammadiyah 4 Pucang Surabaya. Using a quantitative one-group pretest–posttest design with 62 students, results showed a 36.34% improvement in basic operation skills, with the largest gain in multiplication (+73.85%). Learning effort, measured through summative score growth, increased by 17.47%, and effect sizes ranged from moderate to high. The study demonstrates the novelty of integrating Mini Quizzes routinely at the start of lessons to reinforce conceptual understanding and sustain learning effort. Mini Quizzes are therefore recommended as a practical tool to strengthen numeracy achievement and continuous mathematics learning outcomes in elementary schools. Highlights: Students demonstrated substantial growth in arithmetic skills after systematic early-session assessments. The strongest improvement occurred in higher-complexity multiplication tasks. Summative score progression reflected consistent increases in student learning effort. Keywords: Mini Quiz, Basic Mathematical Operations, Formative Assessment, Numeracy Elementary School

To investigate the validity and reliability of the Biodex system 4 in the assessment of abdominal strength in patients with rectus diastasis, we studied 10 healthy volunteers and 10 patients with rectus diastasis of more than 3 cm. We assessed test-retest reliability at 30o and 60o of extension/flexion spinal angles, respectively, and isometric muscle strength with and without the abdominal girdle. Reliability was calculated using the interclass correlation coefficient (ICC). Validity was evaluated by correlation with the International Physical Activity Questionnaire (IPAQ) and a visual analogue scale (VAS) for self-assessment of abdominal muscular strength. The Kendall-Tau and Spearman tests were used. Reliability was excellent with ICC values ranging between 0.77 and 0.97. There was no significant difference in muscular strength for tests with or without a girdle except with 60o extension. The internal validity assessed as the correlation between VAS and isometric strength (p = 0.008) was excellent, whereas there was no correlation with IPAQ. The Biodex system 4 is a valuable and reliable instrument to measure abdominal wall strength in patients with rectus diastasis. The internal validity is excellent. The incongruence between abdominal muscle strength, as measured using the Biodex system 4, and IPAQ indicates that there is no relation between general physical activity (IPAQ) and muscular strength in patients with rectus diastasis.

BackgroundNew technology for clinical data collection is rapidly evolving and may be useful for both researchers and clinicians; however, this new technology has not been tested for accuracy, reliability, or validity.ObjectiveThis study aims to test the accuracy of visual analog scale (VAS) for pain on a newly designed application on the iPad (iPadVAS) and measure the reliability and validity of iPadVAS compared to a paper copy (paperVAS).MethodsAccuracy was determined by physically measuring an iPad scale on screen and comparing it to the results from the program, with a researcher collecting 101 data points. A total of 22 healthy community dwelling older adults were then recruited to test reliability and validity. Each participant completed 8 VAS (4 using each tool) in a randomized order. Reliability was measured using interclass correlation coefficient (ICC) and validity measured using Bland-Altman graphs and correlations.ResultsOf the measurements for accuracy, 64 results were identical, 2 results were manually measured as being 1 mm higher than the program, and 35 as 1 mm lower. Reliability for the iPadVAS was excellent with individual ICC 0.90 (95% CI 0.82-0.95) and averaged ICC 0.97 (95% CI 0.95-1.0) observed. Linear regression demonstrated a strong relationship with a small negative bias towards the iPad (−2.6, SD 5.0) with limits of agreement from −12.4 to 7.1.ConclusionsThe iPadVAS provides a convenient, user-friendly, and efficient way of collecting data from participants in measuring their current pain levels. It has potential use in documentation management and may encourage participatory healthcare.Trial RegistrationAustralia New Zealand Clinical Trials Registry (ANZCTR): 367297; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=367297&isReview=true (Archived by Webcite at http://www.webcitation.org/6d9xYoUbD).

The Accreditation Council for Graduate Medical Education (ACGME) and American Board of Orthopaedic Surgery (ABOS) recently mandated major changes to curricular requirements for the PGY-1 (intern) year for categorical orthopaedic surgery residents. Since July 1, 2013, PGY-1 residents are required to have 6 months of orthopaedic surgery rotations, 6 months of nonorthopaedic surgery rotations, and complete a mandatory surgical skills curriculum [1, 2]. The ACGME and ABOS executed these changes in an effort to improve the quality and breadth of education for interns, preparing them for more advanced orthopaedic education. These changes were prompted by shifts in general surgery procedures (from open to laparoscopic), which have reduced the operating room experience of suturing and knot tying for interns, as well as by duty hour requirements that further reduce the operating room experience for interns. Together, these shifts in first-year resident experience have resulted in less surgical training for the orthopaedic surgery intern. The new ABOS requirements, which commence on July 1, 2014, seek to offset those losses [1, 2]. This column describes the changes mandated by the ABOS and ACGME for the orthopaedic intern year, discusses some limitations associated with these changes, and offers program strategies for effective implementation.

An accurate assessment of surgical operation skills is essential for improving the vascular intervention surgical outcome and the performance of endovascular surgery robots. In existing studies, subjective and objective assessments of surgical operation skills use a variety of indicators, such as the operation speed and operation smoothness. However, the vascular conditions of particular patients have not been considered in the assessment, leading to deviations in the evaluation. Therefore, in this paper, an operation skills assessment method including the vascular difficulty level index for catheter insertion at the aortic arch in endovascular surgery is proposed. First, the model describing the difficulty of the vascular anatomical structure is established with characteristics of different aortic arch branches based on machine learning. Afterwards, the vascular difficulty level is set as an objective index combined with operating characteristics extracted from the operations performed by surgeons to evaluate the surgical operation skills at the aortic arch using machine learning. The accuracy of the assessment improves from 86.67 to 96.67% after inclusion of the vascular difficulty as an evaluation indicator to more objectively and accurately evaluate skills. The method described in this paper can be adopted to train novice surgeons in endovascular surgery, and for studies of vascular interventional surgery robots. Graphical abstract Operation skill assessment with vascular difficulty for vascular interventional surgery.

The objective of this work was to evaluate the safeguards implemented by surgical trainees to protect the confidentiality of electronic patient data through a structured questionnaire sent to Northern Ireland surgical trainees. A group of 32 basic and higher surgical trainees attending a meeting of the Northern Ireland Association of Surgeons-in-Training were invited to complete a questionnaire regarding their computer use, UK Data Protection Act, 1988 registration and electronic data confidentiality practices. Of these 32 trainees, 29 returned completed questionnaires of whom 26 trainees regularly stored sensitive patient data for audit or research purposes on a computer. Only one person was registered under the Data Protection Act, 1988. Of the computers used to store and analyse sensitive data, only 3 of 14 desktops, 8 of 19 laptops and 3 of 14 hand-held computers forced a password logon. Of the 29 trainees, 16 used the same password for all machines, and 25 of 27 passwords were less than 8 characters long. Two respondents declined to reveal details of their secure passwords. Half of all trainees had never adjusted their internet security settings, despite all 14 desktops, 16 of 19 laptops and 5 of 14 hand-helds being routinely connected to the internet. Of the 29 trainees, 28 never encrypted their sensitive data files. Ten trainees had sent unencrypted sensitive patient data over the internet, using a non-secure server. Electronic data confidentiality practices amongst Northern Ireland surgical trainees are unsafe. Simple practical measures to safeguard confidentiality are recommended.

Background: Subspecialization in general surgery is being encouraged by various surgical societies. The aim of this study was to view attitudes of Royal Australasian College of Surgeons (RACS) trainees to subspecialization in surgery, in particular, breast ­surgery. Methods: A postal questionnaire survey of registered RACS basic and advanced surgical trainees was conducted in February 2002. Trainees were asked to nominate their preferred specialty and to indicate the level of support and interest for subspecialty training in breast surgery. Trainees indicating breast surgery as their preferred career choice were then asked to nominate their reasons for choosing breast surgery, preferred options for cross-specialty training and for vocation specifications such as a continuing ‘on-call’ responsibility. Results: Trainees returned 291 of 1049 (28%) completed questionnaires. One hundred and sixty-nine trainees felt that the concept of breast subspecialization in general surgery was reasonable (58%). For all respondents, the most popular specialty choices were plastic surgery (15.8%), orthopaedics (15.5%) and general surgery (15.4%). Breast surgery was chosen by 14 of 291 (4.8%) respondents as their first specialty preference and a further 25 respondents as their second specialty preference. Of 189 trainees who did not choose breast surgery as their preferred specialty, 45% stated repetitive stress, escalating litigation or demanding patients as deterrent factors. Only 36% of trainees interested in breast surgery were interested in undertaking after hours ‘on-call’ work as a consultant, although 36 of 39 (92%) were interested in other forms of general or subspecialty elective surgical operating (i.e., endocrine surgery, surgical oncology) after completion of their training. According to trainees with an interest in breast surgery, the two most important aspects requiring inclusion in the proposed provisional training program were breast reconstruction (38%) and breast screen assessment (34%). Conclusion: Breast surgery is an unpopular subspecialty for RACS trainees. Breast surgery is likely to experience increasing ­problems with recruitment unless the skill base is reviewed and revised in line with the aspirations and needs of today's trainees.

Use of a Novel Measure of Nontechnical Skills in Surgical Trainees: Is There an Association With Technical Skills Performance?

PURPOSE Documentation of resident operative performance remains challenging. Although numerous assessment tools exist, they tend to be lengthy and ill-suited for use after every case, leading to inconsistent or insufficient data informing mandatory semiannual resident reviews. For these reasons, we created a simple, online assessment of resident operative skill to facilitate documentation of training activities in compliance with the American Council of Graduate Medical Education’s (ACGME) Next Accreditation System (NAS). METHODS Working with the Johns Hopkins Technology Innovation Center, we developed an online assessment tool: the Operative Entrustability Assessment (OEA). The tool includes a 5-point scale with descriptive anchors, can be applied to any current procedural terminology code, can be mapped to the NAS Milestones, and mirrors the ACGME case log structure. Department-wide use of the OEA started in January 2014, and residents and faculty were instructed to complete OEAs for all plastic surgery residents performing operative procedures with Department of Plastic Surgery attending physicians. Residents initiate assessments, complete a self-assessment, and hand (or e-mail) the OEA to the attending for completion. Attendings assign cases to the most relevant NAS Milestone and grade the resident on his/her ability to perform the case autonomously. In-person evaluations provide opportunities for immediate feedback; scores are available for viewing in real time via the Data Dashboard. Data can be viewed by Milestone category, CPT code, or case. Data can be further sorted by postgraduate year level, specific dates, Milestone, and CPT code. RESULTS From January 21, 2014, to October 31, 2014, 1298 OEAs were completed by 21 residents and available for use by the Clinical Competency Committee. Six residents did not complete any OEAs due to being on off-service or elective rotations (n = 1); on required, nonclinical (eg, research) rotations (n = 3); or PGY-1 residents assigned to general surgery rotations (n = 2) during the time assessed. Clinical Competency Committee resident assessment and documentation took 275 minutes (4.58 hours), averaging 11.5 minutes (range, 2–28 minutes) per resident across all 36 Milestones, including discussion and recording of action items/assignment of learning activities. OEA data were most often used to inform Patient Care Milestones. Comparisons with the operating room case counts per attending physician estimate the OEA completion rate at approximately 60% of all resident-assisted cases; 38% of residents reported increased immediate feedback on operative performance. CONCLUSIONS Implementation of the OEA has greatly benefitted our semiannual and ongoing assessment of >20 residents, reducing evaluation time >5 times from pilot program evaluation times (1 hour per resident). The OEA also informs program mentor and director the meetings and assignment of training activities. Data transparency provides residents with their own progress compared to the pooled mean of their peers, empowering residents to proactively address areas for potential improvement. Completing OEAs immediately following cases provides valuable opportunities for immediate feedback on operative performance. We continue using the OEA and Data Dashboard for resident assessment and plan to make this tool, which has changed the ACGME case log from a passive to an active measure of operative skill, available to other Plastic Surgery Programs in the near future to validate these findings.