A Collaborative Virtual Environment for the Simulation of Temporal Bone Surgery

Preoperative planning for skull base meningioma surgery requires a detailed understanding of complex anatomical relationships and careful strategy development. Although three-dimensional (3D) simulation has been increasingly used for surgical planning, conventional simulations mainly visualize anatomical structures and rarely reproduce the sequential process of tumor removal. We developed a method to perform a near-complete digital 3D simulation of skull base meningioma surgery using a surgical simulation application. Preoperative imaging data in Digital Imaging and Communications in Medicine (DICOM) format were imported into a simulation platform that automatically generated 3D models of the brain, vasculature, skull, cranial nerves, and tumor through artificial intelligence-based segmentation. Additional reconstruction techniques were applied to create a virtual dura mater. By combining built-in functions for tissue retraction and drill-based removal and labeling the tumor as a removable structure, the simulation reproduced key surgical steps, including craniotomy, tumor detachment from the dural attachment, internal debulking, and dissection from surrounding neurovascular structures. This method was applied to a 59-year-old woman with a symptomatic anterior clinoidal meningioma compressing the optic nerve and encasing the terminal portion of the internal carotid artery. Preoperative simulation reproduced the planned surgical workflow, including frontotemporal craniotomy, anterior clinoidectomy, tumor detachment, compartmental debulking, and dissection from adjacent vessels and the optic nerve. Intraoperative findings closely corresponded to the simulated procedure, and gross total resection was achieved without neurological complications. Near-complete digital simulation of skull base meningioma surgery may facilitate visualization of operative strategies and procedural steps in complex cases. This approach has the potential to serve as a useful tool for surgical planning and neurosurgical training.

3D imaging has become a standard tool in medical diagnostics and, while software is available to visualize volumetric data sets, we do not yet have software that can efficiently transform 3D scan data to solid models that are useful for engineering design and analysis. Why not? Currently, deriving solid models from 3D scans involves 3 steps: (1) segmentation: identification of voxels associated with the structure; (2) polygonization: computing a set of polygons that approximate the surface of the structure; and (3) repair: removing stray voxels and polygons, specifying connectivity, and establishing consistent orientation. Significant progress has been made on accurate, automated segmentation (recent work by Hu et al. (Image Segmentation and Registration for the Analysis of Joint Motion From 3D MRI,” Proc SPIE 6141, pp. 133–142, Medical Imaging: Visualization, Image-Guided Procedures, & Display, 2006), combining graph cuts with level sets is of particular interest) but effective polygonization cannot be guaranteed. In the worst case, manual repairs are needed to patch holes and remove stray elements. Even if a valid boundary representation (b-rep) model is obtained, accurate models contain so many polygons that modeling operations become unfeasible. Moreover, regardless of accuracy, the surface of a polyhedral model will never be truly smooth. In previous work (Storti, D., et al., Artifact vs. Anatomy: Dealing With Conflict of Geometric Modeling Descriptions,” SAE 2007 Transactions Journal of Passenger Cars: Electronic and Electrical Systems, Paper No. 2007-01-2450, Vol. 116, pp. 813–823, 2007), we proposed overcoming the barriers to creating solid models from 3D scans by employing a new solid modeling description, wavelet SDF-reps, that lies much closer to the native 3D scan format and eliminates polygonization. Here, we focus on the ability to produce models with smooth surfaces that are important for various biomedical simulations. For example, careful studies of joint function involve detailed modeling of ligament wrapping; i.e., connective tissue moving across bone surface as the joint configuration changes. Realistic behavior cannot be obtained if the ligament is snagging on or snapping across convex vertices of a polyhedral model. Similarly, haptic simulation of a catheter navigating through the circulatory system cannot be realistic if the catheter gets stuck in concave vertices of the anatomical model. How can the new modeling format address such issues? Wavelet SDF-reps take advantage of a by-product of the segmentation algorithm (Hue et al.) which converts the raw voxel intensity values to a grid of signed distance values. Applying an appropriate interpolant such as Daubechies wavelets (Daubechies, I., Wavelets, CBMS-NS Series in Applied Mathematics, SIAM Publications, Philadelphia, 1992) then produces an implicit or function-based (f-rep) solid model of the segmented structure. Wavelet SDF-reps are inherently multi-resolution and support significant data compression and medial axis computation. We illustrate the capability of wavelet SDF-reps to support smooth models and enable analysis of curvature features.

In surgery simulation, precise contact modeling is essential to obtain both realistic behavior and convincing haptic feedback. When instruments create deformations on soft tissues, they modify the boundary conditions of the models and will mainly modify their behavior. Yet, most recent work has focused on the more precise modeling of soft tissues while improving efficiency; but this effort is ruined if boundary conditions are ill-defined. In this paper, we propose a novel and very efficient approach for precise computation of the interaction between organs and instruments. The method includes an estimation of the contact compliance of the concerned zones of the organ and of the instrument. This compliance is put in a buffer and is the heart of the multithreaded local model used for haptics. Contact computation is then performed in both simulation and haptic loops. It follows unilateral formulation and allows realistic interactions on non-linear models simulated with stable implicit scheme of time integration. An iterative solver, initialized with the solution found in the simulation, allows for fast computation in the haptic loop. We obtain realistic and physical results for the simulation and stable haptic rendering.

The degrees to which transtrochanteric rotational osteotomy moves the region of osteonecrotic femoral head out of the weight-bearing area as evaluated by computer simulation

In this article we present the visualization of the human body basing upon the X-ray CT and MR images. Topics and examaples of images rendered by the computer are all selected from the studies in the authors' laboratory during about twenty years. Topics are classified into four groups. First we introduce a few historical pictures of cross section images, and the surface of bones and the skull. Second, the selected pictures in the surgical simulation system NUCSS (=Nagoya University Craniofacial Surgical Simulation system) we developed are shown. This system was for the simulation of the craniofacial surgery of the skull and the orthopedic surgery of the hip joint. This was the pioneering work which showed the potential of computer aid in surgery based upon 3D CT images about ten years in advance of the present rapid extention of the computer aided surgery. One of the earliest work to visualize the performance of the soft tissue - rendering of the cut skin is also included. Third, the virtualized endoscope system is presented in detail. The images of both the inside and the outside view of the bronchus, the vessel and the stomach are shown here with the introduction to the new concept of the navigation diagnosis. Finally the navigation through an Egyptian mummy is added.

Endoscopic sinus surgery is of essential clinical significance. Nevertheless, the surgery demands very high skills, as the doctor has to hold the endoscope with one hand and perform dexterous operation in the constrained space inside the nose with the other hand. A robotic system can be of help. In this paper, we designed a robotic system to assist endoscopic sinus surgery, and established a virtual surgery platform for clinical simulation. Firstly, the clinical requirements for the surgical robot, such as degrees of freedom, workspace, etc., were analyzed. Then, the mechanism configuration was determined after comparison and study of possible types. Further, the mechanism parameters were specified according to the clinical requirements. Afterwards, a virtual prototype of the surgical robot was developed. Finally, a virtual surgery platform was built up, in which the the user can use a haptic interface to teleoperate the virtual robot and perform virtual endoscopic sinus surgery.

Strain-Initialized Robust Bone Surface Detection in 3-D Ultrasound

To provide more realistic feedback force for training during surgery simulation, a novel model of interaction between a virtual surgical tool and surface of soft tissue was created based on experimental data. Using a self-designed device, normal contact force and tangential friction were tested. A generalized Kelvin-element was introduced to model the nonlinear and relaxation characteristics of the contact force. The parameters of the model were obtained by the least square curve fitting of experimental data. A surgical tool and deformable skin model were constructed in a virtual environment. Phantom Desktop interface allows the operator perceive the feedback force during interacting the virtual skin model.KeywordsVirtual surgeryFeedback forceMass-spring model

With the development of computer technology, Virtual Surgery Simulation (VSS) becomes into reality. Virtual surgery is multi-disciplined and challenged. There have been some open source resources for VSS but they are all in progress. A new open source based VSS system is presented. An object oriented system analysis and design are described in detail including use case diagram, sequence diagrams, class diagram and a multi-level architecture that is multi platform. The system is a general framework that can be configured or customized by experts or developers resulting in a concrete XML scene file to prepare for another user - trainee training his/her surgical skills. The framework is based on open source libraries and existed VSS framework or codes. Some good ideas and good structures are integrated in this framework. It is extensible for developers or experts studying new algorithms and testing new methods. To test the framework lens capsule and liver simulator are configured. It is obvious that the presented framework needs further developing and perfecting.

Modeling mobility is a key aspect when simulating different types of networks. To cater to this requirement, a large number of models has emerged in the last years. They are typically (a) trace-based, where GPS recordings are re-run in simulation, (b) synthetic models, which describe mobility with formal methods, or (c) hybrid models, which are synthetic models based on statistically evaluated traces. All these families of models have advantages and disadvantages. For example, trace-based models are very inflexible in terms of simulation scenarios, but have realistic behaviour, while synthetic models are very flexible, but lack realism. In this paper, we propose a new mobility model, called TRAILS (TRAce-based ProbabILiStic Mobility Model), which bridges the gap between these families and combines their advantages into a single model. The main idea is to extract a mobility graph from real traces and to use it in simulation to create scalable, flexible simulation scenarios. We show that TRAILS is more realistic than synthetic models, while achieving full scalability and flexibility. We have implemented and evaluated TRAILS in the OMNeT++ simulation framework.

Background It is still far away from most of us in that it requires complex 3D modeling. Aims/objectives To investigate a more precision, simple, convenient and economical three-dimensional (3D) printed temporal bone model printed by a commercial desktop 3D printer, which can be widely promoted and applied in the training of beginners in otology. Material and Methods The CT data of the temporal bone were imported into Mimics to construct a 3D digital model of the temporal bone. After loaded into a high-precision 3D printer, a high-precision temporal bone model was printed at a scale of 1:1. Then, the model was evaluated by 5 senior attending physicians, including its morphological accuracy, simulation about surgery, advantages and educational value, using the 7-point Likert scale. Results A life-like temporal bone model was successfully printed out. Five senior attending physicians all thought that the printed model was similar to the natural temporal bone in physical properties and the haptic sensation of bone drilling, and was accurate, simple, convenient and effective. In addition, the model was considered to be of high application value in the teaching of temporal bone anatomy and surgery simulation, which had a material cost of only 3 dollars. Conclusions The high-precision 3D printed temporal bone model is highly similar to the natural temporal bone, and can be conveniently and effectively used in the training of simulating temporal bone surgery for beginners in otology. Its production is simple and economical, so it can be popularized on a large scale.

Simulation Framework for Vehicle Platooning and Car-following Behaviors Under Connected-vehicle Environment

As embodied intelligence emerges as a core frontier in artificial intelligence research, simulation platforms must evolve beyond low-level physical interactions to capture complex, human-centered social behaviors. We introduce FreeAskWorld, an interactive simulation framework that integrates large language models (LLMs) for high-level behavior planning and semantically grounded interaction, informed by theories of intention and social cognition. Our framework supports scalable, realistic human-agent simulations and includes a modular data generation pipeline tailored for diverse embodied tasks.To validate the framework, we extend the classic Vision-and-Language Navigation (VLN) task into a semantically enriched Direction Inquiry setting, wherein agents can actively seek and interpret navigational guidance. We present and publicly release FreeAskWorld, a large-scale benchmark dataset comprising reconstructed environments, six diverse task types, 16 core object categories, 63,429 annotated sample frames, and more than 17 hours of interaction data to support training and evaluation of embodied AI systems. We benchmark VLN models, and human participants under both open-loop and closed-loop settings. Experimental results demonstrate that models fine-tuned on FreeAskWorld outperform their original counterparts, achieving enhanced semantic understanding and interaction competency. These findings underscore the efficacy of socially grounded simulation frameworks in advancing embodied AI systems toward sophisticated high-level planning and more naturalistic human-agent interaction.

A major concern of many government agencies is to predict and control the behavior of crowds in different situations. Many times such gatherings are legal, legitimate, and peaceful. But there are times when they can turn violent, run out of control, result in material damages and even casualties. It then becomes the duty of governments to bring them under control using a variety of techniques, including non-lethal and lethal weapons, if necessary. In order to aid decision makers on the course of action in crowd control, there are modeling and simulation tools that can provide guidelines by giving programmed rules to computer animated characters and to observe behaviors over time in appropriate scenarios. A crowd is a group of people attending a public gathering, with some joint purpose, such as protesting government or celebrating an event. In some countries these kinds of activities are the only way to express public's displeasure with their governments. The governments' reactions to such activities may or may not be tolerant. For these reasons, such situations must be eliminated by recognizing when and how they occur and then providing guidelines to mitigate them. Police or military forces use non-lethal weapons (NLWs), such as plastic bullets or clubs, to accomplish their job. In order to simulate the results of such actions in a computer, there is a need to determine the physical effects of NLWs over the individuals in the crowd. In this dissertation, a fuzzy logic based crowd injury model for determining the physical effects of NLWs is proposed. Fuzzy logic concepts can be applied to a problem by using linguistic rules, which are determined by problem domain experts. In this case, a group of police and military officers were consulted for a set of injury model rules and those rules were then included in the simulation platform. As a proof of concept, a prototype system was implemented using the Repast Simphony agent based simulation toolkit. Simulation results illustrated the effectiveness of the simulation framework.

The human inspector performing visual inspection of an aircraft is the backbone of the aircraft inspection process, a vital element in assuring safety and reliability of an air transportation system. Training is an effective strategy for improving their inspection performance. A drawback of present-day on-the-job (OJT) training provided to aircraft inspectors is the limited exposure to different defect types. Previous studies have shown offline feedback training using virtual reality (VR) simulators to be effective in improving visual inspection performance. This research aims at combining the advantages of VR technology that includes exposure to a wide variety of defects and the one-on-one tutoring approach of OJT by implementing a collaborative virtual training environment. In an immersive collaborative virtual environment (CVE), avatars are used to represent the co-participants. In a CVE, information of where the trainer is pointing can be provided to a trainee as visual deictic reference (VDR). This study evaluates the effectiveness of simulating on-the-job training in a CVE for aircraft inspection training, providing VDR slaved to a 3D mouse used by the trainer for pointing. The results of this study show that the training was effective in improving inspection performance.