Game-based evacuation drill using augmented reality and head-mounted display

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Recupera algunos de los elementos menos estudiados y reconocidos en el ensayo latinoamericano: su profunda vocación de reflexión, diálogo y mediación intelectual; su plasticidad y apertura a la búsqueda del sentido; su vínculo con la sociabilidad, la conversación y la generación de un espacio simbólico de encuentro; su capa cidad de establecer una nueva retroalimentación entre lectura y escritura.

Inherent to the field of computer-aided surgery (CAS) is the necessity to handle sophisticated equipment in the operating room; an undesired side-effect of this development is the fact that the surgeon’s attention is drawn from the operating field since surgical progress is partially monitored on the computer’s screen. The overlay of computer-generated graphics over a real-world scene, also referred to as augmented reality (AR), provides a possibility to solve this problem. The considerable technical problems associated with this approach such as viewing of the scenery within a common focal range on the head-mounted display (HMD) or latency in display on the HMD have, however, kept AR from widespread usage in CAS. In this paper, the concept of the Varioscope AR, a lightweight head-mounted operating microscope used as a HMD, is introduced. The registration of the patient to the preoperative image data as well as preoperative planning take place on VISIT, a surgical navigation system developed at our hospital. Tracking of the HMD and stereoscopic visualisation take place on a separate POSIX.4 compliant realtime operating system running on PC hardware. While being in a very early stage of laboratory testing, we were able to overcome the technical problems described above; our work resulted in an AR visualisation system with an update rate of 6 Hz and a latency below 130 ms. First tests with 2D/3D registration have shown a match between 3D world coordinates and 2D HMD display coordinates in the range of 1.7 pixels. It integrates seamlessly into a surgical navigation system and provides a common focus for both virtual and real world objects. On the basis of our current results, we conclude that the Varioscope AR with the realtime visualisation unit is a major step towards the introduction of AR into clinical routine.

Efficient evaluation of misalignment between real and virtual objects for HMD-Based AR assembly assistance system

Augmented reality imaging technology for orthopaedic surgery

In today’s world, people increasingly rely on mobile apps to do their day-today activities, like checking their Instagram feed and online shopping from websites like Amazon and Flipkart. People are depending on WhatsApp and Instagram stories to communicate with local businesses and to leverage the said platforms for online advertising. Using Google Maps to find their way when they travel and finding out the immediate road and traffic conditions with digital banners around the road, has obviously led to a boom in advertising and marketing. Recently, internet users have increasingly desired to immerse themselves in a Metaverse-like platform where they can interact and socialize. Meta’s Metaverse is a tightly connected network of 3D digital spaces that allow users to escape into a virtual world. It is designed to change the way you socialize, work, shop, and connect with the real and virtual world around you. These platforms are not fully submerged in the real world; they are inclined toward virtual spaces only, making it obvious to fill this gap. Thus, the proposed framework in this chapter would be a new kind of system that may develop a socio-meta platform, powered by augmented reality and other technologies like photogrammetry and LiDAR. Augmented reality provides an interactive way of experiencing the real world, where the objects of the natural world are enhanced by computer-generated perceptual vision. One of the significant problems with augmented reality is the process of building virtual 3D objects that can be augmented into real spaces, which could be solved with photogrammetry and LiDAR. Photogrammetry is the technique of producing 3D objects using 2D images of a physical object taken from different angles and orientations. LiDAR, on the other hand, is another 3D reconstruction technology used widely by Apple’s eco-system. The functioning of LIDAR is very similar to sonar and radar, and the detection and ranging part are where it stands out from the others. The idea behind this platform is to open tons of virtual dimensions in the real world using the principles of mixed reality and geographic mapping tools such as Google Maps, MapBox, and GeoJSON; it would consequently transition the way people spend their time on social media by opening a portal for generating 3D objects that can be augmented to the real-world location using photogrammetry and cloud anchors by just a few 2D digital photographs taken from their camera.

The Covid-19 pandemic has hit Indonesia since the beginning of 2020 and is still continuing. The education sector has experienced a considerable impact, face-to-face learning must be replaced with all-online activities. Many problems arise from online learning, such as inadequate infrastructure, an ineffective learning system and difficulties in understanding the material. Medical education has an impact on the current situation because most of the learning uses infrastructure and direct practice, it is necessary to have alternative learning methods that can improve the quality of learning. Currently, video conference media technology can assist learning activities so that instructors can still explain material to students through virtual media. For medical students who need practice and infrastructure facilities, this technology is still not enough to support their learning media. The idea came up “Application of Learning Module in Health Field based on Extended Reality (sub chapter: Augmented Reality)” This application combines Virtual Reality, Augmented Reality, and Mixed Reality. Augmented reality is a technology that is suitable to be implemented in medical learning because students do not need to prepare anything to perform simulations, just devices and internet networks. For feeding and drinking to the elderly, some items are needed to start the practicum, with this augmented reality there is no need to prepare all of these items, and also reduce contact with other people during a pandemic like this. Users can connect to each other in different devices and environments according to their respective devices. In this application, Augmented Reality acts as a technology that combines the real and virtual worlds by adding virtual objects into the real world. Users can use devices that have cameras such as cellphones or laptops so they can see virtual objects. This virtual object can be seen by all users and each user is able to interact with it in real time. As for the temporary test results, it was found that Augmented Reality has been tested and integrated with other immersive technologies with Web-XR. Interaction in Augmented Reality on Web-XR using a touch screen as well as several nursing SOP have been created and simulated using Augmented Reality technology. As for the final test in this final project, Augmented Reality technology based on WebXR has been running smoothly according to the specifications of the device used to open WebXR. If the device used to access WebXR has a high specification, then WebXR will run optimally, otherwise when the device specification is low, WebXR will not run optimally. The interaction used in WebXR-based Augmented Reality is touchscreen. The author tested this system to the user, the user here is a medical student. testing proves that this system is very helpful in terms of efficiency, users don't need to prepare tools repeatedly when they want to do simulations, just open WebXR and users can do the simulations provided. In testing the system, the author uses the PIECES framework (Performance, Information, Economics, Control and Security, Efficiency, Service) and produces an average score of 4.1. Then the user is categorized as satisfied with the system that was built.

Augmented and virtual reality have the potential to improve various aspects of everyday life such as tourism. An elementary part of tourism is the organization and execution of guided city tours. In case of tours for virtual cities, such as in the case of no longer existing ancient settlements, a combination of augmented and virtual reality may render high benefit for collaboration between guide and visitors. Immersive technology has been shown suitable for collaborative systems by previous research, as they can provide a better experience than other technologies. However, recent literature analyses showed that there is a lack of research regarding the role of augmented reality supporting the active/leading user role in such scenarios. Usually, augmented reality is used for the users who are passively experiencing the virtual environment and not for the one controlling it. However, we think that augmented reality has a large potential to improve the workflow of such a primary user. To investigate this research question, we present a collaborative system in this work, which implements a virtual city tour. In this system, the guide is using augmented reality and the visitor is immersed in the virtual city using virtual reality glasses. To implement a valid research prototype, the system has been developed in close feedback with experts in the field of guided city tours.

In this article, we present a client/server augmented reality (AR) system for viewing complex assembly models on mobile phones. Complex model information is located on a PC which takes care of all of the heavy AR tracking and rendering computation. A camera phone is used as a client to show this information, augmented on still images as animated view. The mobile phone interface also supports correct masking of the assembly pieces as they come together, as well as a graphics only viewing mode intended for better understanding of the assembly process. In addition to describing the mobile augmented assembly system we present results from two pilot user studies evaluating elements of the user interface. INTRODUCTION In order to remain competitive, today’s industry is employing a growing number of product variations with shorter and shorter life cycles, requiring efficient ways to handle the production planning and processing. Assembly line workers face more and more complex tasks and an increasing memory load is posed by the new production needs. Experienced workers may be able to handle several product variants at the same time, but new workers require training and guidance to handle the work tasks. Related activities such as maintenance and repair also require a lot of know-how of complex products. Guidance in assembly tasks is commonly provided by using printed blueprints or other documents. However, these are mainly two dimensional images or text while the real assembly task is three dimensional, making the relationship between parts hard to understand. Computer based technologies have been developed such as electronic guides, multimedia training material and interactive documents. However, these typically require the user to turn away from the assembly task to study a computer screen separate from the workspace. In our research we explore how augmented reality (AR) technology can be used to provide more intuitive guides to assist with assembly, repair and training tasks. With AR technology, synthetic objects can be merged with the user’s view of the real world so that the user can perform the assembly task without needing to look at a separate screen. The virtual objects appear in the correct position so that AR enables the creation of realistic looking 3D animated assembly “manuals”. Other researchers have developed prototype AR systems for supporting assembly tasks. These are typically based on head mounted displays (HMDs) connected to desktop or wearable computers. However, wearable computers are relatively expensive, they have a short battery life, and taking them into a harsh industrial environment is not always possible. Head mounted displays can also be bulky, they involve safety issues, and many workers are reluctant to wear them. To overcome these disadvantages, we are interested in developing a reliable mobile phone based AR assembly system. Today’s mobile phones have fast processors, significant memory, 3D graphics support, and high resolution displays. They contain various accessories that can be used for developing mobile AR applications such as good quality cameras, connectivity by Bluetooth, WLAN and GPRS, and various SAM.700.003.1155

An optical see-through head-mounted display for augmented reality is designed in this paper. Considering the factors, such as the optical performance, the utilization ratios of energy of real world and virtual world, the feelings of users when he wears it and etc., a structure of the optical see-through is adopted. With the characteristics of the particular negative dispersive and the power of realizing random-phase modulation, the diffractive surface is helpful for optical system of reducing weight, simplifying structure and etc., and a diffractive surface is introduced in our optical system. The optical system with 25 mm eye relief, 12 mm exit pupil and 20° (H)x15.4° (V) field-of-view is designed. The utilization ratios of energy of real world and virtual world are 1/4 and 1/2, respectively. The angular resolution of display is 0.27 mrad and it less than that of the minimum of human eyes. The diameter of this system is less than 46mm, and it applies the binocular. This diffractive-refractive optical system of see-through head-mounted display not only satisfies the demands of user’s factors in structure, but also with high resolution, very small chromatic aberration and distortion, and satisfies the need of augmented reality. In the end, the parameters of the diffractive surface are discussed.

Information media continues to evolve with the advancement of time. It is needed at all times because through information media, people can access various types of current and emerging information, as well as exchange ideas and interact with one another. Numerous technologies have been developed to support the dissemination of information, one of which is Augmented Reality (AR). Augmented Reality (AR) can be defined as an environment that merges two realms: the virtual world and the real world. AR allows information to be embedded into the virtual world and displayed in the real-world using markers or barcodes via smartphones or other devices. Augmented Reality (AR) has been widely implemented across various sectors, one of which is promotional media. In the business world, AR can serve as a powerful promotional tool. Therefore, with the advancement of Augmented Reality (AR) technology, a promotional media application was developed using AR for the housing brochures of Cipta Bangun Khatulistiwa Ltd. This AR-based promotional media is expected to assist current and prospective consumers in gaining a more realistic view of Cipta Bangun Khatulistiwa Ltd. housing details through specially designed brochures, without the need to travel to the development site, which is relatively far from the city center. From the results of the system test, 80% mastery and very good assessment were obtained.

Two main concepts of Head Mounted Displays (HMD) for augmented reality (AR) visualization exist, the optical and video-see through type. Several research groups have pursued both approaches for utilizing HMDs for computer aided surgery. While the hardware requirements for a video see through HMD to achieve acceptable time delay and frame rate seem to be enormous the clinical acceptance of such a device is doubtful from a practical point of view. Starting from previous work in displaying additional computer-generated graphics in operating microscopes, we have adapted a miniature head mounted operating microscope for AR by integrating two very small computer displays. To calibrate the projection parameters of this so called Varioscope AR we have used Tsai's Algorithm for camera calibration. Connection to a surgical navigation system was performed by defining an open interface to the control unit of the Varioscope AR. The control unit consists of a standard PC with a dual head graphics adapter to render and display the desired augmentation of the scene. We connected this control unit to a computer aided surgery (CAS) system by the TCP/IP interface. In this paper we present the control unit for the HMD and its software design. We tested two different optical tracking systems, the Flashpoint (Image Guided Technologies, Boulder, CO), which provided about 10 frames per second, and the Polaris (Northern Digital, Ontario, Canada) which provided at least 30 frames per second, both with a time delay of one frame.

Computer-aided surgery (CAS), the intraoperative application of biomedical visualization techniques, appears to be one of the most promising fields of application for augmented reality (AR), the display of additional computer-generated graphics over a real-world scene. Typically a device such as a head-mounted display (HMD) is used for AR. However, considerable technical problems connected with AR have limited the intraoperative application of HMDs up to now. One of the difficulties in using HMDs is the requirement for a common optical focal plane for both the realworld scene and the computer-generated image, and acceptance of the HMD by the user in a surgical environment. In order to increase the clinical acceptance of AR, we have adapted the Varioscope (Life Optics, Vienna), a miniature, cost-effective head-mounted operating binocular, for AR. In this paper, we present the basic design of the modified HMD, and the method and results of an extensive laboratory study for photogrammetric calibration of the Varioscope's computer displays to a real-world scene. In a series of 16 calibrations with varying zoom factors and object distances, mean calibration error was found to be 1.24 +/- 0.38 pixels or 0.12 +/- 0.05 mm for a 640 x 480 display. Maximum error accounted for 3.33 +/- 1.04 pixels or 0.33 +/- 0.12 mm. The location of a position measurement probe of an optical tracking system was transformed to the display with an error of less than 1 mm in the real world in 56% of all cases. For the remaining cases, error was below 2 mm. We conclude that the accuracy achieved in our experiments is sufficient for a wide range of CAS applications.

Medical simulations bridge theory and practice in immersive settings. During this unique form of education, learners experience challenging medical scenarios without risking patient harm. The effectiveness of simulations stem from their ability to induce autonomic stress responses, which potentiate memory formation. Educators have adopted virtual reality (VR) and augmented reality (AR) to supplement traditional, mannequin-based simulations. In this Pro-Con commentary article, we discuss the competitive advantages and limitations of VR and AR simulation for medical training. VR uses computer-generated images (CGI) within head-mounted displays (HMDs) to enhance learning. Gamification of medical procedures in VR improves acquisition of complex skills, positioning VR simulation as a pivotal training tool. Participant embodiment while in VR develops non-technical skills such as improved communication and enhanced empathy. The placement of participants within CGI worlds that would otherwise be difficult or impossible to recreate situate VR as the ideal next step in simulation evolution. VR is superior to AR given its lower hardware costs, expansive software libraries, and complete immersion in CGI compared to limited holographic fields of view within AR. By displaying holograms within the participant's natural world, AR is primed to replace screen and VR technologies as the premier learning adjunct. The integration of real-world instruments with AR holograms provides innumerable methods for teaching procedural skills. In addition, AR enhances pre- and intra-operative planning by overlaying holographic, radiographic patient images over their own bodies. Because AR has the unique feature of maintaining direct eye contact between participants, scenarios that improve communication skills are easily created with holograms in the real world, eliminating the need for an entirely CGI VR environment. Immersive simulation efforts should skip the costs of creating VR scenarios and redirect efforts to developing AR simulation software. Although both VR and AR have limitations related to cost and ergonomics, immersive technologies are relatively more affordable and versatile than traditional mannequins and physical simulation centers. As adjuncts to enhance traditional simulation curricula, we advocate for continued research to demonstrate the validity and utility of these immersive technologies.

This chapter introduces design and principles of head mounted displays (HMDs), as well as their state-of-the-art examples, for augmented reality (AR). Section 2 introduces a brief history of head mounted displays, human vision system, and application examples of see-through HMDs. Section 3 describes designs and principles of HMDs, such as typical configurations of optics, typical display elements, and major categories of HMDs. Section 4 gives typical characteristics of HMDs, such as resolution, field of view, and distortion. Section 5 describes human perceptual and health issues such as depth perception and safety. Finally, Section 6 gives conclusions with future challenges and prospects. For researchers, learners and HMD developers, this chapter is a good start point to learn basics, state of the art technologies, and future research directions of HMDs. For system developers and end-users, this chapter will give a good insight to HMDs to choose a suitable HMD for their purposes.