Practical application of hierarchical task analysis and cognitive work analysis in a surgical training system design

The Endoscopic Surgical Skill Qualification System was established in Japan to evaluate safe endoscopic surgical techniques and teaching skills. Trainee surgeons obtaining this certification in rural hospitals are disadvantaged by the limited number of surgical opportunities. To address this problem, we established a surgical training system to educate trainee surgeons. Eighteen certified expert surgeons affiliated with our department were classified into an experienced training system group (E group, n = 9) and a non-experienced group (NE group, n = 9). Results of the training system were then compared between the groups. The number of years required to become board certified was shorter in the E group (14years) than that in the NE group (18years). Likewise, the number of surgical procedures performed before certification was lower in the E group (n = 30) than that in the NE group (n = 50). An expert surgeon was involved in the creation of the certification video of all the E group participants. A questionnaire to board-certified surgeons revealed that guidance by a board-certified surgeon and trainee education (surgical training system) was useful for obtaining certification. Continuous surgical training, starting with trainee surgeons, appears useful for expediting their acquisition of technical certification in rural areas.

The emerging changes in healthcare significantly impact structure of surgical training. To achieve parity in surgical training globally significant work needs to be done in developing nations. The aims of this study are to identify and evaluate the current surgical training system in India and also examine the surgical training system of residents from India and compare the results against the US training system, to suggest changes for parity. On-site observations and cross-sectional survey of residents and staff from 22 institutions in India were done. Results were compared with the current US training system. Difference in pattern of surgical training and lack of coordination in curriculum was noted. Contradictory responses were noted for didactic teaching. No guidelines were mentioned for minimum number of surgical case required or certification courses. No work hour restrictions were followed, and supervision was ill defined. Residents were highly interested for fellowships and experience from the US medical system. Results revealed discordance in surgical training within the evaluated country. However, the two training systems studied have their own pros and cons. We strongly feel that incorporation of certain aspects of the American residency can aid in progression of the Indian training system.

Training needs analysis is critical to the orderly development of training systems and the procurement of simulators for training. The standard methods of Instructional System Development have been neglected in part because they fail to take full account of cognitive skills and because the extensive detail generated by these methods has made it onerous to manipulate the knowledge representation. A contrasting concern is that many of the newer methods of cognitive task analysis de-emphasise non-cognitive skills. Training needs analysis for complex weapons platforms must take account of all aspects of skill. A new project to develop a training system for an advanced fighter aircraft will identify discrete operational tasks at a behavioural level but will extend that effort into a complete cognitive work analysis as outlined by Rasmussen, Petjersen, and Goodstein (1994). In addition, the scope of the cognitive work analysis will be expanded to include issues unique to training and instruction.

During surgical training, it is important for the surgeon develops good motor skills throughout his training. For this reason, various surgical training systems have been developed to enhance these skills. However, one of the great challenges in these training systems is being able to objectively measure the ability and performance of the main surgical tasks, where currently only a global measurement is obtained once the task is completed. In this work, a temporal evaluation scheme is proposed, that is, an evaluation of local surgical performance at different time intervals during the training of typical tasks (knot-tying, needle-passing and suturing). The goal is to automatically classify expert (experience >100 hrs) and non-expert (experience <10 hrs) surgeons according to their performance during training, based on three classifiers: K-Nearest Neighborhood, Random Forest, and Support Vector Machine Unlike other previously reported methods, this work proposes a new evaluation scheme based on segments or time intervals, which can be an indicator of the surgeon's local performance during a robotic surgical task, without the need for direct labeling of the data at the segment level. The classification performance from obtained results was in accuracy 83% to 100%, 88% to 100% of AUC-ROC, and 88% to 100% of F1-Score in the final test between experts and non-experts surgeons, where the Support Vector Machine classifier presented the best performance. These results suggest that this proposed method by time intervals could be used in various surgical trainers to evaluate the local performance of a surgeon during trainingand thus be able to provide a tool for the quantitative visualization of opportunities to improve surgical skills.Clinical relevance- We consider that the proposed method to carry out a local performance evaluation during surgical training can provide useful information in the learning and improvement of surgical skills.

Prostate cancer progression is characterized by tumor dedifferentiation. Cancer cells of less differentiated tumors have a gene expression/transcriptome more similar to that of stem cells. In dedifferentiation, cancer cells may follow a specific program of gene expression changes to a stem-like state. In order to treat cancer effectively, the stem-like cancer cells and cancer differentiation pathway need to be identified and studied. Due to the very low abundance of stem-like cancer cells, their isolation from fresh human tumors is technically challenging. Induced pluripotent stem cell technology can reprogram differentiated cells into stem-like, and this may be a tool to generate sufficient stem-like cancer cells.

Simulation-based surgical training is beginning to be recognized by medical society with its apparent benefits. As an important function of virtual surgical training system, providing systemic and competitive surgical quality assessment to trainees is still a brand new field. In this paper, we present several metrics and their corresponding measurement methods used in our surgical training system designed for UK’s Royal Bournemouth Hospital to assess trainees’ task accomplishing quality. Classification algorithm and evaluation result of the training record is concisely discussed at the end of the paper.

Background Surgical training is a critical component of healthcare, especially in high-income countries such as the United Kingdom (UK) and the United States (US), which have established distinct, well-organised training frameworks. Comparing these systems provides valuable insights that may enhance global surgical education, particularly in low- and middle-income countries, where training and retaining proficient surgeons are considerable challenges. Methodology This comparative study examines the surgical training systems in the UK and the US, focusing on key aspects, including training structure, competency-based assessments, and work-hour regulations. Data were sourced from regulatory organisations such as the General Medical Council, the Royal College of Surgeons, the Accreditation Council for Graduate Medical Education, and the American Board of Surgery. The analysis explores how elements of these models might be adapted to support sustainable surgical education frameworks in resource-limited environments. Ethical approval was not required due to the use of publicly accessible data and no patient involvement. Results The UK and US surgical training systems differ substantially in their structure, training duration, and specialisation timing. The UK employs a tiered approach, offering generalist experience before specialisation, while the US favours early specialisation directly after medical school. Both systems implement competency-based evaluations, though the US system places a greater emphasis on case volume and procedural exposure. Work-hour regulations also vary, with the UK capping weekly hours at 48 under the European Working Time Directive, compared to an 80-hour maximum in the US, which results in differing levels of trainee satisfaction and burnout rates. Conclusions The competency-based assessments in both the UK and the US offer adaptable frameworks for resource-limited settings. The phased training approach in the UK is well-suited for environments requiring versatile surgeons capable of handling a wide range of cases. By implementing these adaptable elements, along with cost-effective training innovations such as simulation tools, e-learning platforms, and international partnerships, resource-constrained regions can foster a sustainable, skilled surgical workforce. These insights offer pathways to improve healthcare outcomes and equity globally by enhancing surgical capacity in regions with limited resources.

To improve training facilities for surgeons, a surgical training system based on virtual reality techniques has been developed. The goal of the developed system is to improve education of surgeons by making the knowledge of expert surgeons directly available to trainees. The system realizes two different approaches: the library and the driving school paradigm. In its current form, the system consists of two modules. The main module combines the virtual reality kernel KISMET, a visual and haptic display, and a database of different operations and/or techniques. The master station is a copy of the input and output facilities of the main module. Both modules communicate by a TCP/IP-based connection. Initial tests demonstrated the feasibility of the chosen framework. Further developments include the gathering of data not only from virtual reality but also from real operations. Robotic-assisted surgery provides an attractive way of accomplishing this.

The article “Situation Awareness Misconceptions and Misunderstandings” (Endsley, 2015) discusses common fallacies noted in the literature in the interpretation of Endsley’s 1995 model of situation awareness (SA; Endsley, 1995). The clarifications presented in the article provide a more complete and comprehensible explanation of the model. As in the complex domains that we study, our attention suffers from limitations as we simplify each other’s work in our eager and well-intentioned quest to attack new and exciting problems. Dr. Endsley is to be thanked for giving us pause to rebuild our own SA of SA in her thoughtful article. SA has, for many years, been a powerful and influential construct. In our own work in cognitive work analysis (CWA), we have viewed SA as a complementary framework that challenges and drives CWA. Without doubt, the output of a CWA-based design process should be the design of a system that promotes better SA and performance (Burns et al., 2008). CWA and goaldirected task analysis may organize the world in slightly different dimensions, but the overall intent is the same: to create systems that support human decision making as well as we can. Indeed, it is this common intent that unites us in our field. To advance cognitive engineering, there are times when we must challenge each other, critique each other’s models, hunt for flaws, and identify promising new directions. Assuredly, this helps us progress, strengthen our methods, and deepen our understanding. This article clearly identifies such activity and responds to it. We are all better for this exercise, as it challenges both SA and all our approaches to grow and deepen. Acknowledging this, we would like to change our perspective to a larger one and discuss challenges facing cognitive engineering as a whole. In these challenges, our existing methods, SA, CWA, and other approaches must adapt and grow. Advancements in intelligent systems and automation have increased in the amount of data produced by information systems and yet placed the human in new roles. In many cases, these roles are partially in the loop and partially out of the loop and may involve supervisory control or may have the human working in systems with very little supervisory control at all because the automation is largely nontransparent. We outline three core areas of challenges: self-awareness and self-regulation, memory failures or performance with incorrect SA, and design for unstructured environments.

Surgical Process Modeling is a growing field of biomedical data science, aiming to create and support context aware surgical systems. As a part of it, novel research intends to provide standardized, formal description of surgical processes. Surgical workflow recordings based on ontologies can provide objective measurements of surgical skill, thus standardizing surgical performance. Comparing the operational phase to the calculated optimal process could allow for new, context aware surgical training, evaluation and assistant systems. In this paper, we present a new software tool, named OntoFlow, developed to record ontology-based surgical workflow during the clinical practice, with post-event editing and reviewing capabilities. OntoFlow directly accesses the background ontology, therefore it can speed up the process of ontology development. As a surgical workflow reviewing software it can also be used as a training tool for surgical residents.

Highly mechanized forestry operations are essential for efficient timber harvesting. Therefore, the skills of harvester operators appear to be key to productive and sustainable use of the machines. Recent research has revealed a knowledge deficit regarding the work practices of forest machine operators. This urges systematic research into forestry machine handling and a corresponding refinement of analytical methods. Current analyses of operator tasks in forestry are less formalized and focus predominantly on machine efficiency and overall performance, but not so much on the human-related conditions of work performance and workload. Therefore, the objective of this paper is to introduce hierarchical task analysis (HTA) into forestry science. HTA is a versatile, formalized human-factors method that can be used to describe the work objectives of forest machine operators. HTA is suitable, for example, for describing (in)efficient work practices and thus as a basis for designing machine operator training and for systematically evaluating assistive technologies. The task analyses in this paper draw on a recently published empirical approach to analyzing work practices, workflows, and machine operator behavior for optimal human–machine collaboration in forestry application research. Specifically, the main work methods of clearcutting and thinning stand in European forestry were considered, with examples from Scandinavian and German method application. The process of HTA is described and a prototypical approach to HTA for both working methods provided. As a result, this work could show that a single work practice affects operator goals within different work elements and sets out how inefficient work practices can be described in terms of operator goals. With the introduction and exemplary application of HTA, a structured task definition in human-centered approaches is encouraged to analyze work practices, workflows, and machine operator behavior for optimal human–machine collaboration in forestry application research.

The US Department of Energy Office of Environmental Management (DOE-EM) oversees one of the largest and most technically challenging cleanup programs in the world. The mission of DOE-EM is to complete the safe cleanup of the environmental legacy from five decades of nuclearweapons development and government-sponsored nuclear energy research. Since 1995, Florida International University’s Applied Research Center (FIU-ARC) has supported the DOE-EM mission and provided unique research capabilities to address some of these highly technical and difficult challenges. This partnership has allowed FIU-ARC to create a unique infrastructure that is critical for the training and mentoring of science, technology, engineering, and math (STEM) students and has exposed many STEM students to “hands-on” DOE-EM applied research, supervised by the scientists and engineers at ARC. As a result of this successful partnership between DOE and FIU, DOE requested FIU-ARC to create the DOE-FIU Science and Technology Workforce Development Initiative in 2007. This innovative program was established to create a “pipeline” of minority STEM students trained and mentored to enter DOE’s environmental cleanup workforce. The program was designed to help address DOE’s future workforce needs by partnering with academic, government and private companies (DOE contractors) to mentor future minority scientists and engineers in the research, development, and deployment of new technologies and processes addressing DOE’s environmental cleanup challenges. Since its inception in 2007, the program has trained and mentored 78 FIU STEM minority students. Although, the program has been in existence for only six years, a total of 75 internships have been conducted at DOE National Laboratories, DOE sites, DOE Headquarters and field offices, and DOE contractors. Over 100 DOE Fellows have participated in the Waste Management (WM) Symposia since 2008 with a total of 84 student posters and 7 oral presentations given at WM. The DOE Fellows participation at WM has resulted in three Best Student Poster Awards (WM09, WM10, and WM11) and one Best Professional Poster Award (WM09). DOE Fellows have also presented their research at ANS DD&amp;R and ANS Robotics Topical meetings and this year two Fellows will present at the International Conference on Environmental Remediation and Radioactive Waste Management (ICEM13) in Brussels, Belgium. Moreover, several of our DOE Fellows have already obtained employment with DOE-EM, other federal agencies, DOE contractors, commercial nuclear power companies, and other STEM industry (GE, Boeing, Lockheed Martin, Johnson &amp; Johnson, Beckman-Coulter, and other top companies). This paper will discuss how DOE Fellows program is training and mentoring FIU STEM students in Department of Energy’s Office of Environmental Management technical challenges and research. This training and mentoring has resulted in the development of well-trained and polished young scientists and engineers that will become the future workforce in charge of carrying on DOE-EM’s environmental cleanup mission. The paper will showcase FIU’s DOE Fellows model and highlight some of the applied research the DOE Fellows have conducted at FIU’s Applied Research Center and across the DOE Complex by participating in summer internship assignments.

Performance ImprovementVolume 42, Issue 2 p. 16-18 Article Visibility into the work TQM work process analysis with HPT and ISD Charles A. Beagles, Charles A. Beagles Charles A. Beagles is the Director, Technical Training and Evaluation Office, Veterans Benefits Administration (VBA). Prior to joining VBA, Charles served as Deputy Program Director, Marine Corps Programs, and as Division Head, Training Plans and Analysis Division, Naval Air Warfare Systems Training Systems Division. Prior to that, he worked at the Center for Needs Assessment and Planning, Florida State University. In addition to 22 years experience in training analysis and design, Charles is a former Marine Corps Captain and served as a Rifle Company Commander in Vietnam. He holds a PhD in educational psychology, from Florida State University. Charles may be reached at vrccbeag@vba.va.gov.Search for more papers by this authorSteven L. Griffin, Steven L. Griffin Steven L. Griffin, an Instructional Systems Specialist, is Program Manager of Training and Performance Support Systems for the Technical Training and Evaluation Office, Veterans Benefits Administration (VBA). He holds a MEd from the University of Louisville. Steven has 32 years of government and industry experience in training system and courseware design and development, with VBA, the Department of Defense, Science Applications International Corporation, and General Electric. He may be reached at vsdsgrif@vba.va.gov.Search for more papers by this author Charles A. Beagles, Charles A. Beagles Charles A. Beagles is the Director, Technical Training and Evaluation Office, Veterans Benefits Administration (VBA). Prior to joining VBA, Charles served as Deputy Program Director, Marine Corps Programs, and as Division Head, Training Plans and Analysis Division, Naval Air Warfare Systems Training Systems Division. Prior to that, he worked at the Center for Needs Assessment and Planning, Florida State University. In addition to 22 years experience in training analysis and design, Charles is a former Marine Corps Captain and served as a Rifle Company Commander in Vietnam. He holds a PhD in educational psychology, from Florida State University. Charles may be reached at vrccbeag@vba.va.gov.Search for more papers by this authorSteven L. Griffin, Steven L. Griffin Steven L. Griffin, an Instructional Systems Specialist, is Program Manager of Training and Performance Support Systems for the Technical Training and Evaluation Office, Veterans Benefits Administration (VBA). He holds a MEd from the University of Louisville. Steven has 32 years of government and industry experience in training system and courseware design and development, with VBA, the Department of Defense, Science Applications International Corporation, and General Electric. He may be reached at vsdsgrif@vba.va.gov.Search for more papers by this author First published: 08 February 2007 https://doi.org/10.1002/pfi.4930420208 AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat No abstract is available for this article. Volume42, Issue2February 2003Pages 16-18 RelatedInformation

Cognitive work analysis and cognitive task analysis approach the analysis of decisions in ways that are at least superficially dissimilar. In this paper, I review the two approaches, work (or control) task analysis and naturalistic decision making, to identify similarities and differences, and I assess the implications of looking at decisions from these two different perspectives. I conclude that although these two approaches are superficially dissimilar, they are theoretically compatible. However, the standard form of decision ladder, which is the representational product of work (or control) task analysis, does not take full account of contemporary insights from the three-stage model of situation awareness, the distinction between implicit and explicit cognitive processing, or the principles of naturalistic decision making. I propose modifications of the decision ladder to incorporate these ideas.

This paper is concerned with a novel problem: learning temporal models using only relative information. Such a problem arises naturally in many applications involving motion or video data. Our focus in this paper is on videobased surgical training, in which a key task is to rate the performance of a trainee based on a video capturing his motion. Compared with the conventional method of relying on ratings from senior surgeons, an automatic approach to this problem is desirable for its potential lower cost, better objectiveness, and real-time availability. To this end, we propose a novel formulation termed Relative Hidden Markov Model and develop an algorithm for obtaining a solution under this model. The proposed method utilizes only a relative ranking (based on an attribute of interest) between pairs of the inputs, which is easier to obtain and often more consistent, especially for the chosen application domain. The proposed algorithm effectively learns a model from the training data so that the attribute under consideration is linked to the likelihood of the inputs under the learned model. Hence the model can be used to compare new sequences. Synthetic data is first used to systematically evaluate the model and the algorithm, and then we experiment with real data from a surgical training system. The experimental results suggest that the proposed approach provides a promising solution to the real-world problem of motion skill evaluation from video.