Augmented Reality Calibration with Stereo Image Registration for Surgical Navigation

The present study investigated the impact teaching academic vocabulary through an augmented reality (AR) approach had on acquisition and retention as compared to using a more traditional flash card based approach. 29 high school students ranging from 9th to 11th grade followed the pattern of a 10 word vocabulary pre-test, being taught those words through respective alternating AR and traditional vocabulary methods, and post-tested one week from the date of teaching. At the end of the testing and teaching process, the unit scores were compared to find if any differences occurred. Analysis revealed that there is a significant difference between the two teaching methods. Participants scored better in the AR unit as compared to the traditional unit. Furthermore, participants also reported greater satisfaction with the AR method because it held their attention, gave them something to focus on, and was overall more engaging. Comparatively, participants found the traditional method not engaging, however, they were comfortable with the traditional method because it is what they are comfortable with in the classroom. Given the curiosity of using AR in the classroom, more research could be conducted to find patterns in participant performance across all academic content areas as well as other uses for AR in the classroom. 

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Recupera algunos de los elementos menos estudiados y reconocidos en el ensayo latinoamericano: su profunda vocación de reflexión, diálogo y mediación intelectual; su plasticidad y apertura a la búsqueda del sentido; su vínculo con la sociabilidad, la conversación y la generación de un espacio simbólico de encuentro; su capa cidad de establecer una nueva retroalimentación entre lectura y escritura.

Spatial data visualization technology allows users to understand Geographic Information System (GIS) applications. Unlike traditional visualization methods, Augmented Reality (AR) inserts virtual objects and information directly into digital representations of the real world, which makes these objects and data more easily understood and interactive. However, effective AR-GIS systems and rich spatial information visualization is still a challenging task. In addition, indoor AR-GIS systems are further impeded by the limited capacity of these systems to detect and display geometry and semantic information. To address this problem, a novel AR and indoor map fusion method is proposed that automatically registers spatial information onto a live camera view of a mobile phone. We fused Bluetooth low energy (BLE) and pedestrian dead reckoning (PDR) localization techniques to track the camera positions. The proposed algorithm extracts and matches a bounding box of the indoor map to a real world scene. We render the indoor map and semantic information into the real world, based on the real-time computed spatial relationship between the indoor map and live camera view. Experimental results demonstrate that our approach accurately and richly visualizes spatial information. Our augmented reality and indoor map fusion technique effectively links rich indoor spatial information to real world scenes in AR integrated GIS.

Workflow Caveats in Augmented Reality–Assisted Pedicle Instrumentation: Cadaver Lab

Augmented reality (AR) is a new human-computer interaction technology that combines virtual reality, computer vision, and computer networks. With the rapid advancement of the medical field towards intelligence and data visualisation, AR systems are becoming increasingly popular in the medical field because they can provide doctors with clear enough medical images and accurate image navigation in practical applications. However, it has been discovered that different display types of AR systems have different effects on doctors' perception of the image after virtual-real fusion during the actual medical application. If doctors cannot correctly perceive the image, they may be unable to correctly match the virtual information with the real world, which will have a significant impact on their ability to recognise complex structures. This paper uses Citespace, a literature analysis tool, to visualise and analyse the research hotspots when AR systems are used in the medical field. A visual analysis of the 1163 articles retrieved from the Web of Science Core Collection database reveals that display technology and visualisation technology are the key research directions of AR systems at the moment. This paper categorises AR systems based on their display principles, reviews current image perception optimisation schemes for various types of systems, and analyses and compares different display types of AR systems based on their practical applications in the field of smart medical care so that doctors can select the appropriate display types based on different application scenarios. Finally, the future development direction of AR display technology is anticipated in order for AR technology to be more effectively applied in the field of smart medical care. The advancement of display technology for AR systems is critical for their use in the medical field, and the advantages and disadvantages of various display types should be considered in different application scenarios to select the best AR system.

Analysing and manipulating 3D objects are key competences for product designers. Although there is a substantial amount of research on review tools based on immersive technologies related to industrial practices, a comparison with traditional design reviews on an educational level is less prominent in scientific literature. This paper consists of a comparison between reviewing a CAD model on a 2D laptop screen and a more immersive method, using an augmented reality (AR) headset. The participants are twelve industrial design students. They were asked to review two products and look for ergonomic and technical design errors. Based on the evaluation of different parameters, the presented AR method scored high on spatial insight and contextual readability. Also, it was uncovered that the AR method was more demanding both physically and mentally. The findings state that immersive technologies can positively influence the industrial design process, anticipating that future AR technology becomes mature enough.

Medical care-computerized change could be an incredible forward leap for the clinical business. Executing the correct arrangements will bring a ton of favorable circumstances, and a few illnesses could be treated more viably and a lot faster. Progressive change in medical care is not just about innovations at the same time; to be more exact, it is somewhat about the best approach to utilize accessible instruments, programming, advanced technologies to take full advantage of it for individuals’ well-being. Computerized or digital medical service is a relatively new field, yet on account of the experience of some created nations; presently, we can exploit the accompanying arrangements: cause online physical checkups; to make electronic clinical records; make brought together stages for information trade; use chatbots to give essential data on the Web, including electronic diagnostic cards and other similar methods that include all patients’ records and their prescriptions. Advanced and digital change in medical services is a structure square of a patient-centered way to deal with medical services. It will help medical care suppliers’ smooth out activities, comprehend what the patient requires, fabricate dedication and trust, and offer a superior client experience. Thus, today computerized medical care innovation advances various public arena changes and specifically in the medical services field. Thus, these techniques are commonly used to enhance the facilities and industry offered in the medical field. Artificial intelligence (AI) technology provides better diagnostics as it has great potential for data analysis; augmented reality (AR) is a technique to increase efficiency and helps in surgery cost-wise. Cloud computing can store and share data on all platforms; blockchain technology makes it easier for another doctor to treat a patient because it allows the doctor to see its medication history. Internet of Things (IoT), wearable gadgets, and assisting applications gather ongoing information that could help physicians have complete information about the patient’s condition; this information could likewise be investigated and give valuable bits of knowledge to various explores. We ended up in an unpleasant period loaded with vulnerability and capriciousness. However, the pandemic is uncommon, while a medical services emergency brought about by the jumps in understanding flood is occasional, unsurprising, and controllable. Computerized medical services change one of the significant strides to monitor expanding the weight and forestall the emergency medical service. Computerized change or digital transformation is a progressing cycle, and new patterns are arising in the medical services industry as time passes. When seeking digital transformation in medical care, we have to think past the innovation expected to drive development.

Augmented Reality (AR) is the process of integrating virtual elements in reality, often by mixing computer graphics into a live video stream of a real scene. It requires registration of the target object with respect to the cameras. To this end, some approaches rely on dedicated hardware, such as magnetic trackers or infra-red cameras, but they are too expensive and cumbersome to reach a large public. Others are based on specifically designed markers which usually look like bar-codes. However, they alter the look of objects to be augmented, thereby hindering their use in application for which visual design matters. Recent advances in Computer Vision have made it possible to track and detect objects by relying on natural features. However, no such method is commonly used in the AR community, because the maturity of available packages is not sufficient yet. As far as deformable surfaces are concerned, the choice is even more limited, mainly because initialization is so difficult. Our main contribution is therefore a new AR framework that can properly augment deforming surfaces in real-time. Its target platform is a standard PC and a single webcam. It does not require any complex calibration procedure, making it perfectly suitable for novice end-users. To satisfy to the most demanding application designers, our framework does not require any scene engineering, renders virtual objects illuminated by real light, and let real elements occlude virtual ones. To meet this challenge, we developed several innovative techniques. Our approach to real-time registration of a deforming surface is based on wide-baseline feature matching. However, traditional outlier elimination techniques such as RANSAC are unable to handle the non-rigid surface's large number of degrees of freedom. We therefore proposed a new robust estimation scheme that allows both 2–D and 3–D non-rigid surface registration. Another issue of critical importance in AR to achieve realism is illumination handling, for which existing techniques often require setup procedures or devices such as reflective spheres. By contrast, our framework includes methods to estimate illumination for rendering purposes without sacrificing ease of use. Finally, several existing approaches to handling occlusions in AR rely on multiple cameras or can only deal with occluding objects modeled beforehand. Our requires only one camera and models occluding objects at runtime. We incorporated these components in a consistent and flexible framework. We used it to augment many different objects such as a deforming T-shirt or a sheet of paper, under challenging conditions, in real-time, and with correct handling of illumination and occlusions. We also used our non-rigid surface registration technique to measure the shape of deformed sails. We validated the ease of deployment of our framework by distributing a software package and letting an artist use it to create two AR applications.

We present a novel method for augmented reality in endoscopic endonasal surgery. Our method does not require the use of external tracking devices and can show hidden anatomical structures relevant to the surgical intervention. Our method registers a preoperative 3D model of the nasal cavity to an intraoperative 3D model by estimating a scaled-rigid transformation. Registration is based on a two-stage ICP approach on the reconstructed nasal cavity. The hidden structures are then transferred from the preoperative 3D model to the intraoperative one using the estimated transformation, projected and overlaid into the endoscopic images to obtain the augmented reality. We performed qualitative and quantitative validation of our method on 12 clinical cases. Qualitative results were obtained from an ENT surgeon from visual inspection of the hidden structures in the augmented images. Quantitative results were obtained by measuring a target registration error using a novel transillumination-based approach. The results show that the hidden structures of interest are augmented at the expected locations in most cases. Our method was able to augment the endoscopic images in a sufficiently precise manner when the intraoperative nasal cavity did not deform considerably with respect to its preoperative state. This is a promising step towards trackerless augmented reality in endonasal surgery.

This paper presents an experimental method and apparatus of augmented reality (AR) for nDimensional (nD) building information modeling (BIM). BIM allows nD information to be visualized simultaneously by architects, engineers and constructors to gain a synchronized understanding viewing from different perspectives. However, BIM is conventionally being operated on a desktop-based computer which makes collaboration less flexible, and it also creates an isolation gap between the model and reality. This isolation gap does not severely affect experienced and skilled professionals, as they can bridge the isolation gap with their intuition developed over the years. Nevertheless, users who are lack of such experience will feel the isolation gap between the digital realm and practicable reality, which could be the hurdle in project participation and decision making. AR allows virtual content to be mixed with real environment for user experience. In the context of our study, AR is functional to present the nD information of BIM, at the same time retaining users' connection with the reality. It is not just being utilized solely for presentation, but also to maximize the potential for communication, interaction and experience. This pilot study investigates effective technological approach of using AR as an effective collaboration technology combining with BIM through proposed key aspects of contextualization, customization and curation. Contextualization is significant to enable users to understand the AR content by making the presented information meaningful to the target audience, implemented thru the means of 2D annotations, animations and options comparison. This study compares both AR BIM with and without contextualization. Customization can generate unique virtual environment and content for different level of users tailored to their needs and preference to create intuitive interaction with AR BIM. Curation is crucial to provide users with a reliable experience, and to formulate a continually improving AR BIM thru log data and users' feedback. All in all, this paper explores the major aspects of contextualization, customization and curation, to distinguish effective approach in the currently “free for all” AR BIM development. Finally, an implication is provided for future study in terms of balance in information sufficiency and complexity for AR BIM.

Augmented reality imaging technology for orthopaedic surgery

The immersive augmented reality (AR) system necessitates precise depth registration between virtual objects and the real scene. Prior studies have emphasized the efficacy of surface texture in providing depth cues to enhance depth perception across various media, including the real scene, virtual reality, and AR. However, these studies predominantly focus on black-and-white textures, leaving a gap in understanding the effectiveness of colored textures. To address this gap and further explore texture-related factors in AR, a series of experiments were conducted to investigate the effects of different texture cues on depth perception using the perceptual matching method. Findings indicate that the absolute depth error increases with decreasing contrast under black-and-white texture. Moreover, textures with higher color contrast also contribute to enhanced accuracy of depth judgments in AR. However, no significant effect of texture density on depth perception was observed. The findings serve as a theoretical reference for texture design in AR, aiding in the optimization of virtual-real registration processes.

The perceptual issues in Augmented Reality (AR) systems, is drawing more and more attention these days. Among the perceptual issues, the incorrect depth interpretations degrade user experience significantly. One way to improve the depth interpretations in AR systems is to improve the consistency of the real scene and the virtual objects by using the depth cue method based on blurring effect. In the research of [Okumura 2006], a blur rendering method based on measuring the defocusing on the AR marker from the captured real scene has been proposed. However, the blur effect could only be rendered on the marker position where the blur in the real scene is measured. This limits the practicality of the system since in many applications the virtual objects are out of the marker's range or even moving. In our work, a new method to estimate the degree of blur is proposed. This method makes it possible to calculate the degree of blur that varies spatially with the scene.

Augmented Reality (AR) technology offers innovative ways in order to visualize and manipulate a 3D model of an object by superimposing computer-generated images onto another object interactively. The ability to interact with digital and spatial information in real-time offers new opportunities to manipulate and process medical data easily and efficiently. During surgical interventions, surgeons face various challenges dealing with digital patient data. Several methods are used to visualize the operative areas, such as fluoroscopy and ultrasound techniques. These techniques have several limitations. Thus, the augmented reality technique could serve as a better alternative to project a three-dimensional model of the target organ into the surgeon's perspective and field of view to improve the accuracy and efficiency of the medical intervention intraoperatively. In this paper, a new AR method is proposed in order to visualize and simulate the biomechanical model of the liver organ during open hepatic surgery. In this regard, the 3D model based on the patient's preoperative CT scans is first reconstructed. Then, the reconstructed model is projected using the AR headset. After that, the biomechanical model is generated and prepared for the simulation. The proposed approach is validated using acquired CT scans of the human organ.