Augmented reality for the surgeon: Systematic review.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

This paper reviews diverse 3D visualisation technologies such as virtual reality (VR), augmented reality (AR), and mixed reality (MR). It was prompted by recent research and breakthroughs in AR technology and its biomedical embedded applications in the wearable electronics area, especially in the domain of head-up displays (HUD). Wearable AR technologies are being used to help elderly generations and people with dementia, visual impairment and hearing impairment according to a growing body of research. Moreover, the surgical community and the biomedical education have benefited from these technologies. In addition to discussing the development of novel wearable gadgets and systems, the paper details clinical uses of wearable technology that are currently being evaluated.

Augmented Reality (AR) is a technology that can overlap virtual elements over the real world in real time. This research focuses on studying how different AR elements can help forklift operators locate pallets as quickly as possible in a warehouse environment. We have developed a simulated AR environment to test Egocentric or Exocentric virtual navigation cues. The virtual elements were displayed to the user in a HUD (head-up display) on the forklift windshield, fixed place in front of the user operator, or in a HMD (head-mounted display), where the virtual cues are attached to the head of the user. A user study found that the Egocentric AR view was preferred over the Exocentric condition and performed better while the HUD and HMD viewing methods produced no difference in performance.

Impairment of autonomous way-finding subsequent to a multitude of neurodegenerative and other diseases impedes independence of older persons and their everyday activities. It was the goal to use augmented reality to aid autonomous way-finding in a community setting. A spatial map and directional information were shown via head-up display to guide patients from the start zone on the hospital campus to a bakery in the nearby community. Hospital campus and nearby community. Patients with mild cognitive impairment (age 63 to 89). A head-up display was used to help patients find their way. Time needed to reach goal and number of assists needed. With use of augmented reality device, patients preceded along the correct path in 113 out of 120 intersections. Intermittent reassurance was needed for most patients. Patients affirmed willingness to use such an augmented reality device in everyday life if needed or even pay for it. Augmented reality guided navigation is a promising means to sustain autonomous way-finding as a prerequisite for autonomy of older persons in everyday activities. Thus, this study lays ground for a field trial in the community using assistive technology for older persons with cognitive impairment.

In traditional driving, drivers need to frequently look down at the instrument panel to check information such as speed, RPM, and fuel level. This process causes the driver's gaze to leave the road ahead, increasing the risk of accidents.The automotive head‐up display (HUD) was introduced by General Motors in 1988 to prevent driver distraction. Augmented reality (AR) technology has made the HUD more attractive for ADAS visualization, with the ability to overlay driving and sensed information on the road. [1] AR-HUD technology has been applied to project important driving information in the driver's line of sight. By The head-up-display (HUD), which reflects driving information into the windshield has the goal to lower driving effort from the information uptake and thereby, increase our safety by reducing risks associated to e.g., fatigue and stress. [2] The specific implementation principle is that the windscreen HUD uses the optical combining characteristics of the HUD interlayer (i.e. the PVB interlayer sandwiched in a laminated windscreen) to provide a single focused virtual image appearing near the front of the car. [3] HUDs have evolved from traditional Combiner HUDs (C-HUD) to Windshield HUDs (W-HUD) and then to AR-HUDs. AR-HUDs achieve immersive interaction of "human-car-environment" through sensor fusion and connected vehicle data. However, the current AR-HUD interface design, with its fixed information priority and single visual presentation form, struggles to meet the differentiated needs and safety guarantees in dynamic environments. To address this issue, we dynamically adjust information priority and presentation forms to enhance the visual focus of key driving information, optimize the spatial organization logic of the information architecture to improve information recognition efficiency in complex scenarios, and adapt interface element contrast and layout clarity to environmental features to improve user visual experience. We have established an environmental adaptability design model and proposed an interface innovation strategy of "context awareness-information layering-visual optimization." The feasibility of this method was evaluated through user research. Subjective feedback on the "jarring sensation" of the interface information decreased by 78%, and spatial perception deviation reduced by 45%. This design significantly improved the effectiveness of information transmission in multiple environmental scenarios, reduced the driver's cognitive load, and verified the core role of interface design in the environmental adaptability of automotive AR-HUDs, providing theoretical and practical support for optimizing human-machine interaction in intelligent driving.

Digital Electro-optics Platform is the main concept of Jasper Display Corp. (JDC) to develop various applications. These applications are based on our X-on-Silicon technologies, for example, X-on-Silicon technologies could be used on Liquid Crystal on Silicon (LCoS), Micro Light-Emitting Diode on Silicon (μLEDoS), Organic Light-Emitting Diode on Silicon (OLEDoS), and Cell on Silicon (CELLoS), etc. LCoS technology is applied to Micro Display, Spatial Light Modulator (SLM), Dynamic Optics, Wavelength Selective Switch (WSS), Holographic Display, Microscopy, Bio-tech, 3D Printing and Adaptive Optics, etc. In addition, μLEDoS technology is applied to Augmented Reality (AR), Head Up Display (HUD), Head-mounted Display (HMD), and Wearable Devices.

An ongoing research problem in Augmented Reality (AR) is to improve tracking and display technology in order to minimize registration errors. However, perfect registration is not always necessary for users to understand the intent of an augmentation. This paper describes the results of an experiment to evaluate the effects of graphical context in a Lego block placement task when the graphics are located outside of the task area. Four conditions were compared: fully registered AR; non-registered AR; a heads-up display (HUD) with the graphics always visible in the field of view; and a HUD with the graphics not always visible in the field of view. The results of this experiment indicated that registered AR outperforms both non-registered AR and graphics displayed on a HUD. The results also indicated that non-registered AR does not offer any significant performance advantages over a HUD, but is rated as less intrusive and can keep non-registered graphics from cluttering the task space.

This paper studies the applicability of Augmented Reality Head-Up Displays (HUD) to car racing applications. HUD have been used for years in avionics to effectively represent environmental cues to the pilot without the need to look at the instruments. Recent advances in technology claim that similar systems can be developed for cars. HUD have been available for cars for some time, but have been exploited only as virtual dashboards. This paper looks into the potential benefits and limitations of truly Augmented Reality for car racing. In particular, the objective is to highlight the curbs of the track so to help the driver plan the best trajectory. We develop the algorithms needed to detect the curb and project its position on the HUD. We show that, in order to provide a precise projection of the cue, the attitude of the vehicle needs to be estimated and the information from a camera needs to be merged with a map of the racing track. We test the approach on an experimental vehicle rigged to emulate a HUD. The results indicate that, while the algorithms can be designed to be robust and precise, they require a very large Field of View.

Today, the automotive industry is focused on human-vehicle interaction related with safety and convenience while driving. One of those is head-up display (HUD) to implement augmented reality (AR) in a vehicle for information offering [Park et al. 2013]. In this study, we discuss a vehicle Augmented Reality Information System (vARIS) to offer information such as collision warning and path guide through augmented reality (AR) in a vehicle. This system realizes AR on windshield using projection type HUD and provides AR information on 50 inch HUD display for safety and convenience of a driver. A currently commercialized HUD has 3~5 inch display.

Problem definition: Firms increasingly use augmented reality (AR) devices to improve their production ramp-up processes. These devices appear useful, yet little is known about their broader impact on worker productivity and behavior. Academic/practical relevance: Efficient production ramp-ups are particularly important when product life cycles are short. An ongoing debate among academics and practitioners pertains to how Industry 4.0, and AR devices in particular, can accelerate the ramp-up. The current study provides empirical evidence related to AR in the production ramp-up context, examines the strengths and weaknesses of AR, and tests four hypotheses, leading to a more nuanced view of AR use in the manufacturing ramp-up. Methodology: A framed field experiment in a manufacturing plant provides a test of how quickly workers can perform new tasks with and without AR support and how the use of AR affects their ability to suggest process improvements. Results: When faced with a new task, workers instructed by AR smart glasses use 43.8% less time to complete the task compared with a control group that relies on paper-based instructions. However, workers that use AR glasses consistently use 23% more time than the control group when both groups repeat the task without either AR or paper-based instructions. Task difficulty moderates this relationship; workers assigned to a more difficult task benefit the most from AR instructions. After the devices are removed, workers instructed based on paper improve their productivity faster through learning than those instructed by AR. In addition, the former group suggests better process improvements than the latter one. Managerial implications: Although these results indicate substantially higher productivity resulting from AR devices, they also support the view that, once instructed through AR devices, workers come to rely on this new technology without fully internalizing the task. This failure to internalize their task then leads workers to suggest less useful process improvements.

Traditional workplace studies focus on the bodily integration of familiar technologies like the desktop computer and smartphone. To better understand augmented reality (AR) technologies that provide new ways to perform daily tasks, this article turns to an overlooked yet critical forerunner of AR: the head-up display (HUD). Extensive archival research and oral histories show that the HUD caused pilots to depend on a single interface, curtailing their autonomy for integrating disparate information that could inform critical, often lethal decision-making processes. This history provides evidence that once a single device subsumes many functions of a work environment, adaptive articulation work can drive information workers to undertake increasingly insulated information practices. The HUD's isolating and insulating perceptual legacy suggests that AR will similarly train users in many contexts how to trust interactive data overlays in ways that preclude cross-checking alternative data sources that could inform decision-making processes.

Effective rendering technique of auditory information for an augmented reality (AR) device has been investigated. Researchers and industries are searching for new applications incorporating AR and Virtual Reality (VR) technologies, which provide enhanced user-experience of interactivity and intimacy. To utilize these benefits best, homogeneous integration between visual and auditory information is important. To date, the binaural technology based on head-related transfer function (HRTF) has been used to create immersive and three-dimensional audio objects for VR and AR devices. In this study, we compared the HRTF method with a stereophonic panning method that only controls Inter-aural Level Difference (ILD) for an AR device (a smart see-through glass) and investigated the precision of auditory information required for a coherent representation with a target visual image at the locations of 0 degree, -5 degrees, and -10 degrees in a counter clockwise. Auditory stimuli were rendered to have target locations in the horizontal plane from + 45 degrees to -45 degrees with a 5-degree interval. In the subjective evaluation, each participant answered whether a randomly given auditory location was matched to a target location of the visual image. The results showed that two audio rendering methods did not produce significant difference in creating integrated perception in an AR device.

The automotive industry recognizes the vibrant development of augmented reality (AR) technology and identifies that it will be a great enhancement for the business from design, manufacturing, safety, and user experience perspectives. Recently, AR head-up displays (HUDs) are showcased with fascinating and useful applications. Several automotive OEMs advertise their intent to implement AR HUDs in future programs. In this presentation, we will focus on HUDs, provide an overview of AR HUD use cases that enhance driver experience, the requirements to create AR experience, and review the enabling technology. Specifically, we will review and describe the key elements of phase holography and the use of this powerful technology to achieve a compact optical system design.

Recently, there is an increasing trend of adopting digital holography in practical applications such as projectors, head-up display (HUD), and near-eye augmented reality (AR) devices. These demands have led to numerous research works in the past decade, focusing on developing low cost and compact holographic display system. Amongst these research works, investigation on generation of phase-only holograms has become an area of immense interest. A phase-only hologram (POH) has two major advantages. First it can be displayed with a single phase-only SLM. Second, the reconstructed image is brighter, and free from the de-focused twin image. In this tutorial, a number of methods for the generation of POH, will be reviewed. Emphasis will be placed on methods that are non-iterative and computationally efficient so that they can be applied to generate phase-only hologram at high frame-rate. Moreover, these works also have the advantage that the POHs generated can be displayed directly on the phase-only SLM with little or no optical accessories, hence simplifying the complexity and cost of the optical setup.

Purpose: This study aims to investigate the accuracy of a cross platform augmented reality (AR) system for percutaneous needle interventions irrespective of operator error. In particular, we study the effect of the relative position and orientation of the AR device and the marker, the location of the target, and the angle of needle on the overlay accuracy. Method: A needle guidance AR platform developed using Unity and Vuforia SDK platforms was used to display a planned needle trajectory for targets via mobile and wearable devices. To evaluate the system accuracy, a custom phantom embedded with metal fiducial markers and an adjustable needle guide was designed to mimic different relative position and orientation scenarios of the smart device and the marker. After segmenting images of CT-visible fiducial markers as well as different needle trajectories, error was defined by comparing them to the corresponding augmented target/needle trajectory projected by smartphone and smartglasses devices. Results: The augmentation error for targets and needle trajectories were reported as a function of marker position and orientation, as well as the location of the targets. Overall, the image overlay error for needle trajectory was 0.28±0.32° (Max = 0.856°) and 0.41±0.23° (Max = 0.805°) using the iPhone and HoloLens glasses, respectively. The overall image overlay error for targets was 1.75±0.59 mm for iPhone, and 1.74±0.86 mm for HoloLens. Conclusions: The image overlay error caused by different sources can be quantified for different AR devices.