Unscented Kalman Filter Based Sensor Fusion for Robust Optical and Electromagnetic Tracking in Surgical Navigation

Electromagnetic (EM) tracking systems are often used for real time navigation of medical tools in an Image Guided Therapy (IGT) system. They are specifically advantageous when the medical device requires tracking within the body of a patient where line of sight constraints prevent the use of conventional optical tracking. EM tracking systems are however very sensitive to electromagnetic field distortions. These distortions, arising from changes in the electromagnetic environment due to the presence of conductive ferromagnetic surgical tools or other medical equipment, limit the accuracy of EM tracking, in some cases potentially rendering tracking data unusable. We present a mapping method for the operating region over which EM tracking sensors are used, allowing for characterization of measurement errors, in turn providing physicians with visual feedback about measurement confidence or reliability of localization estimates. In this instance, we employ a calibration phantom to assess distortion within the operating field of the EM tracker and to display in real time the distribution of measurement errors, as well as the location and extent of the field associated with minimal spatial distortion. The accuracy is assessed relative to successive measurements. Error is computed for a reference point and consecutive measurement errors are displayed relative to the reference in order to characterize the accuracy in near-real-time. In an initial set-up phase, the phantom geometry is calibrated by registering the data from a multitude of EM sensors in a non-ferromagnetic ("clean") EM environment. The registration results in the locations of sensors with respect to each other and defines the geometry of the sensors in the phantom. In a measurement phase, the position and orientation data from all sensors are compared with the known geometry of the sensor spacing, and localization errors (displacement and orientation) are computed. Based on error thresholds provided by the operator, the spatial distribution of localization errors are clustered and dynamically displayed as separate confidence zones within the operating region of the EM tracker space.

One of the major challenges related to percutaneous radiofrequency ablation (RFA) of liver tumors is the exact placement of the instrument within the lesion. Previous studies have shown the benefit of computer-assisted needle insertion based on optical tracking of both the instrument and internal fiducials used for registration. However, the concept has not been accepted for clinical use. This may in part be attributed to the line-of-sight constraint imposed by optical tracking systems which results in the use of needles thick enough to avoid bending. Electromagnetic (EM) tracking systems allow the localization of medical instruments without line-of-sight requirements, but are known to be less robust to the influence of metallic and/or ferromagnetic objects. In this paper, we apply a previously introduced fiducial-based system for navigated needle insertion with an EM tracking system and assess the overall targeting error using a static phantom in two different settings: in a non-metallic environment (REF) and on a CT stretcher (CT). While accurate needle insertion could be achieved in the reference environment (REF: 2.6±0.7 mm), targeting errors dropped drastically in the presence of metal (CT: 10.4±6.1 mm). For accurate and robust computer-assisted needle insertion, EM field distortions should thus either be avoided by assuring a suitable environment or by using methods for shielding or error compensation.

Positioning of dental implants using computer-aided navigation and an optical tracking system: case report and presentation of a new method

A navigation system for flexible endoscopes equipped with ultrasound (US) scan heads is presented. In contrast to similar systems, abdominal 3D-US is used for image fusion of the pre-interventional computed tomography (CT) to the endoscopic US. A 3D-US scan, tracked with an optical tracking system (OTS), is taken pre-operatively together with the CT scan. The CT is calibrated using the OTS, providing the transformation from CT to 3D-US. Immediately before intervention a 3D-US tracked with an electromagnetic tracking system (EMTS) is acquired and registered intra-modal to the preoperative 3D-US. The endoscopic US is calibrated using the EMTS and registered to the pre-operative CT by an intra-modal 3D-US/3D-US registration. Phantom studies showed a registration error for the US to CT registration of 5.1 mm ± 2.8 mm. 3D-US/3D-US registration of patient data gave an error of 4.1 mm compared to 2.8 mm with the phantom. From this we estimate an error on patient experiments of 5.6 mm.

4d cone-beam computed tomography (CBCT) scans are usually reconstructed by extracting the motion information from the 2d projections or an external surrogate signal, and binning the individual projections into multiple respiratory phases. In this "after-the-fact" binning approach, however, projections are unevenly distributed over respiratory phases resulting in inefficient utilization of imaging dose. To avoid excess dose in certain respiratory phases, and poor image quality due to a lack of projections in others, the authors have developed a novel 4d CBCT acquisition framework which actively triggers 2d projections based on the forward-predicted position of the tumor. The forward-prediction of the tumor position was independently established using either (i) an electromagnetic (EM) tracking system based on implanted EM-transponders which act as a surrogate for the tumor position, or (ii) an external motion sensor measuring the chest-wall displacement and correlating this external motion to the phase-shifted diaphragm motion derived from the acquired images. In order to avoid EM-induced artifacts in the imaging detector, the authors devised a simple but effective "Faraday" shielding cage. The authors demonstrated the feasibility of their acquisition strategy by scanning an anthropomorphic lung phantom moving on 1d or 2d sinusoidal trajectories. With both tumor position devices, the authors were able to acquire 4d CBCTs free of motion blurring. For scans based on the EM tracking system, reconstruction artifacts stemming from the presence of the EM-array and the EM-transponders were greatly reduced using newly developed correction algorithms. By tuning the imaging frequency independently for each respiratory phase prior to acquisition, it was possible to harmonize the number of projections over respiratory phases. Depending on the breathing period (3.5 or 5 s) and the gantry rotation time (4 or 5 min), between ∼90 and 145 projections were acquired per respiratory phase resulting in a dose of ∼1.7-2.6 mGy per respiratory phase. Further dose savings and decreases in the scanning time are possible by acquiring only a subset of all respiratory phases, for example, peak-exhale and peak-inhale only scans. This study is the first experimental demonstration of a new 4d CBCT acquisition paradigm in which imaging dose is efficiently utilized by actively triggering only those projections that are desired for the reconstruction process.

This paper presents the development of an electromagnetic (EM) tracking system that consists of a pyramid field generator (PFG) of five excitation coils and a sensing coil for location and orientation estimation. The system measures the induced electromotive forces (EMF) of the sensing coil with the change of the magnetic field contributed by the PFG. The proposed modeling distributed magnetic flux (DMF) method that divides the multi-turns sensing coil into multiple layers and concentric circles for the distributed elements is utilized to determine EMF induced in the sensing coils for investigating the effects of translational and angular displacement. The finite element analysis numerically verifies the DMF method and validates the Artificial Neural Network (ANN)-based estimation models. The calibration and ANN-based estimation process are applied to inversely estimate the location and orientation of the sensing coil based on the DMF method and the experimental EM tracking system. The proposed design and a prototype of the EM tracking system have experimentally verified the DMF method and showed the validity of the location and orientation estimation.

Neuronavigation is integral to modern neurosurgery. Clinical studies demonstrate its effectiveness. The primary tracking modalities in neurosurgical navigation are optical tracking systems (OTS) and electromagnetic tracking systems (EMTS). OTS remains the gold standard due to its accuracy and reliability. However, inherent inaccuracies due to brain deformation, image resolution, tool calibration, and registration errors can impact overall accuracy significantly, which differs from the system-declared accuracy. Augmented reality (AR) technologies solve traditional navigation challenges by integrating virtual information with the patient's anatomy, enhancing the surgeon's focus and cognitive load management. Head-mounted displays (HMDs) offer ergonomic benefits, although most AR-based neuronavigation studies have been limited to proof-of-concept trials. This study aims to evaluate VOSTARS, a novel hybrid video and optical see-through HMD designed for precision surgery, specifically in neurosurgical oncology for targeting supratentorial tumors. Previous in-vitro studies using patient-specific phantoms have shown promising results, with high accuracy in real-to-virtual target visualization and craniotomy trajectory tracing. With this work, we further assessed VOSTARS' targeting accuracy within a realistic neurosurgery clinical workflow and compared its performance to the commercial StealthStation system on a patient-specific phantom. Our results demonstrate that users achieved the same median accuracy, 2 mm (IQR: 1 mm), over 60 measurements with both VOSTARS and the StealthStation with no statistically significant difference between the systems, confirming the non-inferiority of the VOSTARS platform compared to a commercial optical tracking-based surgical navigator.

Patient-to-medical image registration is a crucial factor that affects the accuracy of image-guided ENT- and neurosurgery systems. In this study, a novel registration protocol that extracts the point cloud in the patient space using the contact approach was proposed. To extract the optimal point cloud in patient space, we propose a multi-step registration protocol consisting of augmentation of the point cloud and creation of an optimal point cloud in patient space that satisfies the minimum distance from the point cloud in the medical image space. A hemisphere mathematical model and plastic facial phantom were used to validate the proposed registration protocol. An optical and electromagnetic tracking system, of the type that is commonly used in clinical practice, was used to acquire the point cloud in the patient space and evaluate the accuracy of the proposed registration protocol. The SRE and TRE of the proposed protocol were improved by about 30% and 50%, respectively, compared to those of a conventional registration protocol. In addition, TRE was reduced to about 28% and 21% in the optical and electromagnetic methods, respectively, thus showing improved accuracy. The new algorithm proposed in this study is expected to be applied to surgical navigation systems in the near future, which could increase the success rate of otolaryngological and neurological surgery.

Online viewpoint optimization method for high-accuracy intraoperative optical measurement in orthopedic surgeries: An active navigation approach

The hand-wrist region is reported as the most common injury site in boxing. Boxers are at risk due to the amount of wrist motions when impacting training equipment or their opponents, yet we know relatively little about these motions. This paper describes a new method for quantifying wrist motion in boxing using an electromagnetic tracking system. Surrogate testing procedure utilising a polyamide hand and forearm shape, and in vivo testing procedure utilising 29 elite boxers, were used to assess the accuracy and repeatability of the system. 2D kinematic analysis was used to calculate wrist angles using photogrammetry, whilst the data from the electromagnetic tracking system was processed with visual 3D software. The electromagnetic tracking system agreed with the video-based system (paired t tests) in both the surrogate (< 0.2°) and quasi-static testing (< 6°). Both systems showed a good intraclass coefficient of reliability (ICCs > 0.9). In the punch testing, for both repeated jab and hook shots, the electromagnetic tracking system showed good reliability (ICCs > 0.8) and substantial reliability (ICCs > 0.6) for flexion–extension and radial-ulnar deviation angles, respectively. The results indicate that wrist kinematics during punching activities can be measured using an electromagnetic tracking system.

A preliminary study on surgical navigation for epiduroscopic laser neural decompression.

Navigation with optical tracking sometimes makes it difficult to establish a line-of-sight in cluttered operating theatres. On the other hand, the accuracy of electromagnetic tracking is influenced by ferromagnetic surgical equipment. We compared electromagnetic with optical tracking under controlled conditions for the lateral skull base. Six anatomical specimens were dissected to measure the target registration error (TRE) in and around the petrous bone in a wet laboratory to simulate an intraoperative setting. Specimens were registered with passive optical and electromagnetic tracking. Overall accuracy was better using optical tracking than electromagnetic tracking (0.22 mm; 0.07-0.48 vs 0.99 mm; 0.56-1.27 mm; median, lower and upper quartiles, respectively; p < 0.001). The accuracy of optical tracking was near the resolution of the camera system, whereas the accuracy of electromagnetic tracking was lower. Only optical tracking allows for an application accuracy of considerably less than 1 mm in high-resolution datasets.

Optical motion tracking systems are widely employed in virtual- and mixed reality applications enabling users to directly interact with 3D space. Important efforts have been made to make such technologies available to a larger group of users by significantly reducing their deployment costs. In this poster we introduce the result of a German-Brazilian cooperation: The first commercial low-cost marker-based optical tracking system developed in South-America. Despite using cheaper technologies and being low-cost which usually imply worse performance and accuracy, we developed a system with sufficient accuracy for professional use. There have been significant research efforts to develop cheaper tracking devices to provide interaction capabilities in virutal- and mixed reality. The Augmented Reality Tool Kit (ARToolkit) [1] is a good example of a very popular low-cost academic optical tracker working with any type of webcam. As a non-academic example of affordable tracking systems, Nintendo's latest video game console (Nintendo Wii) comes with the Wiimote interaction device containing a built-in motion tracker that allows for immersive interaction metaphors in home environments. However, most such available low-cost solutions lack in suitability for professional applications due to their usually insufficient accuracy, tracking range or latency time. The starting point of this project was PTrack [2], an academic marker-based single-camera tracker developed at Fraunhofer IGD. PTrack development has shown that an accurate optical tracking system can be built from components that are affordable for a much wider group of users, from the academic AR/VR community to companies’ developers. The performance and accuracy of PTrack were assessed through a comprehensive set of experiments which have proved the suitability of the system for professional applications. One of the main topics raised by PTrack as further research was an extension of the system into a multiple stereo camera configuration, which led to the present work.

This project aimed to create an immersive digital twin laboratory that incorporates advanced tracking and visualization capabilities. In collaboration with Fort Lewis College, NREL designed a state-of-the-art physical visualization laboratory, developed a software platform to enable interaction with tracked physical objects in the laboratory, and provided proof-of-concept curricula that included manipulating these tracked objects. The project was initiated to address the growing need for innovative educational tools in engineering education. As renewable energy systems, particularly solar installations, become more complex, there is a pressing need to bridge the gap between theoretical knowledge and practical application. Traditional methods of teaching solar engineering concepts often fall short of providing students with a comprehensive, hands-on understanding. This immersive digital twin laboratory was conceived to fill that gap by creating a safe, non-energized setting where students can interact with augmented solar installation objects, gaining valuable insights into system performance, design, and maintenance. The project utilized extended reality (XR) technologies, including head-mounted displays (HMDs) and a whole-room optical motion tracking system, to connect physical objects with their digital twins in real time. The laboratory was equipped with MagicLeap 2 HMDs, supported by a Vicon Vero 2.2 Optical Tracking System, which provided precise 6-degrees-of-freedom (6-DOF) tracking. We developed a software platform to manage the interaction between the tracked physical objects and their virtual counterparts, enabling real-time data synchronization, object recognition, and virtual overlays. We designed the system to be flexible and extendable, allowing for future integration of additional objects and curriculum. This research advances the field of engineering education by demonstrating the potential of immersive digital twin environments. The laboratory provides a dynamic learning space where students can experiment, collaborate, and learn without the risks associated with live experimentation. The ability to simulate and manipulate solar installation objects under various conditions has broad implications for workforce development, particularly in renewable energy. The project also highlights the economic feasibility of using XR technologies in educational settings, offering a cost-effective solution for institutions looking to enhance their curriculum. By fostering a deeper understanding of solar energy systems, this work contributes to the broader goal of supporting the global energy transition and preparing the next generation of engineers and technicians.

Electromagnetic tracking systems (EMTS) are increasingly used in computer assisted surgery systems, motion capture units and military systems. The main application in the medical domain represents the support of minimally invasive surgeries. However, those electromagnetic tracking systems suffer from low accuracy and systematic errors caused by conductive and ferromagnetic media. In this paper we present a method for the calibration of electromagnetic tracking systems, which could be adopted in the development of novel electromagnetic tracking devices. The proposed method was evaluated on an experimental setup of a tracking system.