A Cloud Based Augmented Reality Framework - Enabling User-Centered Interactive Systems Development

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Augmented reality: An ecological blend

Introduction: Augmented Reality (AR) systems are systems in which users view and interact with virtual objects overlaying the real world. AR systems are used across a variety of disciplines, i.e., games, medicine, and education to name a few. Optical See-Through (OST) AR displays allow users to perceive the real world directly by combining computer-generated imagery overlaying the real world. While perception of depth and visibility of objects is a widely studied field, we wanted to observe how color, luminance, and movement of an object interacted with each other as well as external luminance in OST AR devices. Little research has been done regarding the issues around the effect of virtual objects’ parameters on depth perception, external lighting, and the effect of an object’s mobility on this depth perception.Methods: We aim to perform an analysis of the effects of motion cues, color, and luminance on depth estimation of AR objects overlaying the real world with OST displays. We perform two experiments, differing in environmental lighting conditions (287 lux and 156 lux), and analyze the effects and differences on depth and speed perceptions.Results: We have found that while stationary objects follow previous research with regards to depth perception, motion and both object and environmental luminance play a factor in this perception.Discussion: These results will be significantly useful for developers to account for depth estimation issues that may arise in AR environments. Awareness of the different effects of speed and environmental illuminance on depth perception can be utilized when performing AR or MR applications where precision matters.

In recent years, hardware and software advances in digital technology have prompted not only the commercial use of augmented reality (AR) systems, but its implementation in education. Marker-based AR systems allow users to input pre-designed markers, and generate new scenes blending together the real world and computer-generated 3D images. Nevertheless, few AR systems permit simultaneous interactions of multiple markers. This study aims to construct an interactive multimedia learning system. The story script showcases the growth of flowers by applying a series of markers, where interactions with 3D objects can be achieved through AR technology. Learners can use a camera to capture the information of multiple AR markers, and 3D objects and media can be generated on-screen. With variations in AR markers, and time changes of the Arduino Clock, one can learn the growth of plants during day and night through a mixed-reality virtual experience that brings fun to learning. Accordingly, this e-learning system is capable of boosting students' learning interests and improving their learning effectiveness.

The rapid development of the Augmented Reality (AR) technology has brought new possibilities regarding the enhancement of user experiences in various industries, and the retail will be one of the first to receive it. In this paper, the research questions are put regarding the application of AR in shopping malls to provide a hotel and interactive shopping experience. The AR technology applied to mobile applications and smart devices will assist customers with visualization of the products in 3D and information superimposition in real-time, and give AR instructions that will guide the customers through the intricate plans of the mall floor. The proposed system will also enable retailers to find new methods of interactive marketing, advertise customized promotion and devise methods of customer engagement. An AR system prototype was developed then tested in a simulated mall environment to assess the enhancement of user experience, navigational outcomes, and product engagement. The findings suggest the system is beneficial to enhancing user satisfaction in the mall, navigating better, and entail greater promotion content engagement. This research demonstrates how AR can improve user experience in malls, create more foot traffic into a mall, and provide insights through data for shoppers, retailers, and mall operators. Future research will be conducted on scalability options for the AR system, integration with wearable AR devices for real-time analytics that will improve the experience of a smart retail environment.

There is a growing interest for the broad use of Augmented Reality (AR) and Virtual Reality (VR) in the fields of bioinformatics and cheminformatics to visualize complex biological and chemical structures. AR and VR technologies allow for stunning and immersive experiences, offering untapped opportunities for both research and education purposes. However, preparing 3D models ready to use for AR and VR is time-consuming and requires a technical expertise that severely limits the development of new contents of potential interest for structural biologists, medicinal chemists, molecular modellers and teachers. Herein we present the RealityConvert software tool and associated website, which allow users to easily convert molecular objects to high quality 3D models directly compatible for AR and VR applications. For chemical structures, in addition to the 3D model generation, RealityConvert also generates image trackers, useful to universally call and anchor that particular 3D model when used in AR applications. The ultimate goal of RealityConvert is to facilitate and boost the development and accessibility of AR and VR contents for bioinformatics and cheminformatics applications. http://www.realityconvert.com. dfourch@ncsu.edu. Supplementary data are available at Bioinformatics online.

The demand for advanced photonic technologies to enhance augmented reality (AR) devices has led to a significant interest in new optical materials suitable for operation in sometimes challenging environmental conditions, such as elevated temperatures levels. Holographic optical lenses (HOLs), fabricated using holographic recording, offer several advantages over traditional refractive lenses, including greater versatility, lower cost, lower weight, and increased adaptability. In this study, we investigated the optical and thermal performance of HOLs recorded on four different photosensitive materials: Bayfol HX200, Bayfol HX120, a PVA/acrylamide-based photopolymer (AA/PVA), and a photopolymerisable hybrid sol–gel (PHSG). The lenses were exposed to thermal cycles 20 ∘ C–100 ∘ C–20 ∘ C to simulate real-world conditions in AR systems, where heat generation from processors, sensors, displays, and the proximity to poweful LED light sources can affect their optical performance. The results highlight that Bayfol HX200 showed the highest thermal stability, while the PHSG demonstrated good recovery properties after thermal stress. Optical and image quality were evaluated using parameters such as root mean square wavefront error, Strehl ratio and spatial resolution, which revealed significant variations in lens performance as a function of temperature. These results provide valuable information on the material selection for durable, high-performance holographic lenses in AR applications.

Augmented reality (AR) systems can be effectively used to enhance manufacturing and industrial processes. However, not all the existing prototypes of AR systems can be used in an industrial environment due to heavy constraints such as low robustness or cumbersome equipment. Our AR system relies purely on passive techniques to solve the real-time registration problem, and it can run on a portable computer. We combined a powerful virtual reality (VR) component-based simulation framework with computer vision techniques, turning it into an AR system. The resulting system allows us to produce complex rendering and animation of avatars, and blend them into the real world. The system tracks the 3D camera position by means of a natural features tracker, which, given a rough CAD model, can deal with complex 3D objects. The tracking method is robust and can handle large camera displacements and aspect changes. The target applications of our AR system are industrial maintenance, repair and training. The tracking robustness makes the AR system able to work in real environments, such as industrial facilities, and not only in the laboratory.

Virtual reality (VR) and augmented reality (AR) systems are increasingly available to neurosurgeons. These systems may provide opportunities for technical rehearsal and assessments of surgeon performance. The assessment of neurosurgeon skill in VR and AR environments and the validity of VR and AR feedback has not been systematically reviewed. A systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines was conducted through MEDLINE and PubMed. Studies published in English between January 1990 and February 2021 describing the use of VR or AR to quantify surgical technical performance of neurosurgeons without the use of human raters were included. The types and categories of automated performance metrics (APMs) from each of these studies were recorded. Thirty-three VR studies were included in the review; no AR studies met inclusion criteria. VR APMs were categorized as either distance to target, force, kinematics, time, blood loss, or volume of resection. Distance and time were the most well-studied APM domains, although all domains were effective at differentiating surgeon experience levels. Distance was successfully used to track improvements with practice. Examining volume of resection demonstrated that attending surgeons removed less simulated tumor but preserved more normal tissue than trainees. More recently, APMs have been used in machine learning algorithms to predict level of training with a high degree of accuracy. Key limitations to enhanced-reality systems include limited AR usage for automated surgical assessment and lack of external and longitudinal validation of VR systems. VR has been used to assess surgeon performance across a wide spectrum of domains. The VR environment can be used to quantify surgeon performance, assess surgeon proficiency, and track training progression. AR systems have not yet been used to provide metrics for surgeon performance assessment despite potential for intraoperative integration. VR-based APMs may be especially useful for metrics that are difficult to assess intraoperatively, including blood loss and extent of resection.

Endoscopic surgery causes less tissue injury compared to open surgical techniques, thus promoting more rapid recuperation and reduced post-operative pain. Endoscopy, however, allows the surgeon to visualise only the anatomical surface of the surgical site, with a relatively narrow field of view. Moreover, the 2D video captured by the conventional endoscope does not provide depth perception of the surgical scene. In this study, these limitations have been addressed with the development of an augmented reality (AR) system with stereoscopic visualisation. A phantom and its 3D CT model were used, respectively, to form the real and virtual parts of the AR. The virtual environment camera pose was tracked using algorithms for image feature detection, feature matching and Perspective-n-Point applied on the endoscopic image and the 3D virtual model-rendered image. The endoscope video frame- and the virtual model-rendered images were superimposed to form the AR composite view. The depth buffer (z-buffer) of the rendering window was further used to make a stereo pair of the AR image. The AR system produced a stereo composite view having well-aligned real and virtual components. The RMS error of the real and virtual image contours registration was 9.6 ± 6.7 mm. Correlation coefficients between the depth map from z-buffer and a depth camera was between 0.60 and 0.96 (p < 0.05). The AR system requires further improvement to be applicable at a higher frame rate of endoscope image acquisition. It also needs to include motion and deformation models when applied to animals or patients.

The combination of technology Augmented Reality (AR) with educational content aims to create new ideas that can improve the effectiveness and attractiveness of teaching and learning for students in real-life scenarios. The use of AR technology can make theprocess of teaching and learning more enjoyable, in which students' interest in learning can be stimulated. The AR technology has three characteristics. The first feature is that AR technology is a combination of virtual objects with the real world. The second feature is that AR operates in real time and interactively. The last feature is that it has 3 dimensions (3D) for virtual objects. In this study, the ADDIE model includes five phases, namely Analysis, Design, Development, Implementation and Evaluation, which are utilised to develop applications based on AR technology in education for the topic of Current and Voltage Division. To identify the effectiveness of the application of AR technology, a survey using questionnaire was conducted among pre-serviceteachers to know their perceptions of the development of AR applications namely as Voltrent AR. As this is a quantitative study where questionnaires were distributed to 73 students from the class of Bachelor of Technology with Education (Electric and Electronics) with Honours and Bachelor of Technology with Education (Living Skills) with Honours at Universiti Teknologi Malaysia (UTM) Skudai, Johor. Most respondents gave very positive feedback on the use of AR in education, especially for the topic of Current and Voltage Division. AR is viewed as a medium of instruction that is innovative, interesting and effective. Voltrent AR application development is successfully developed based on the ADDIE model, and the application development of the constructivism theory was applied in this AR application. Thus, the AR application development has reached the desired objective of producing application-based Augmented Reality as a teaching aid.

Problem definition: Firms increasingly use augmented reality (AR) devices to improve their production ramp-up processes. These devices appear useful, yet little is known about their broader impact on worker productivity and behavior. Academic/practical relevance: Efficient production ramp-ups are particularly important when product life cycles are short. An ongoing debate among academics and practitioners pertains to how Industry 4.0, and AR devices in particular, can accelerate the ramp-up. The current study provides empirical evidence related to AR in the production ramp-up context, examines the strengths and weaknesses of AR, and tests four hypotheses, leading to a more nuanced view of AR use in the manufacturing ramp-up. Methodology: A framed field experiment in a manufacturing plant provides a test of how quickly workers can perform new tasks with and without AR support and how the use of AR affects their ability to suggest process improvements. Results: When faced with a new task, workers instructed by AR smart glasses use 43.8% less time to complete the task compared with a control group that relies on paper-based instructions. However, workers that use AR glasses consistently use 23% more time than the control group when both groups repeat the task without either AR or paper-based instructions. Task difficulty moderates this relationship; workers assigned to a more difficult task benefit the most from AR instructions. After the devices are removed, workers instructed based on paper improve their productivity faster through learning than those instructed by AR. In addition, the former group suggests better process improvements than the latter one. Managerial implications: Although these results indicate substantially higher productivity resulting from AR devices, they also support the view that, once instructed through AR devices, workers come to rely on this new technology without fully internalizing the task. This failure to internalize their task then leads workers to suggest less useful process improvements.

With Generation Z becoming the primary group of nursing learners and the increasing shortage of nursing education resources, augmented reality and mixed reality based on head-mounted displays are being used more and more in nursing education. However, the current application landscape and the proper usage of these concepts remain unclear. Therefore, this study aims to conduct a scoping review to explore the current applications of head-mounted display-based augmented reality and mixed reality in nursing education and to clarify the definitions and usage of the concepts of augmented reality and mixed reality, ultimately providing directions for future applications and research. Based on the five-stage framework and PRISMA-ScR guidelines, a comprehensive collection and summarization of evidence regarding the application of head-mounted display-based augmented reality and mixed reality in nursing education were conducted. The databases retrieved include CNKI, Wanfang Database, VIP, CBM, PubMed, Cochrane Library, Embase, CINAHL, Web of Science, Scopus, IEEE Xplore, ACM Digital Library, and Ei. The languages of the included literature are Chinese and English. The retrieval was up to October 2024. A total of 44 studies were included in this review, covering three types of head-mounted displays: immersive head-mounted displays, smart glasses, and smartphone-based head-mounted displays. The main application areas were skills training and knowledge acquisition. Most of the studies were feasibility studies, though they also included some efficacy studies and research on personal experiences. In addition, these studies often employed vague or inconsistent definitions of augmented reality and mixed reality. Despite various explorations in the application of head-mounted display-based augmented reality and mixed reality in nursing education, there is still room for improvement in the relevant theory and utilization of this technology. In the future, interventions should use the three dimensions (observation of reality, real - virtual interaction, and fidelity of virtuality) described in detail, rather than simply employing the concepts of augmented reality or mixed reality. Efforts should be concentrated on developing and implementing head-mounted displays combined with other technologies that boast enhanced performance and cost - effectiveness, and further validating their effectiveness.

While manual visual inspection of structures has the advantage of being relatively simple and low cost, it is usually time consuming, labor intensive and highly subjective. Augmented reality (AR), because of its ability to provide the user with additional information about the working environment in real-time, has been used in the past to address some of the limitations of manual visual inspection by supporting human workers during the inspection process. The paper presents the development of an efficient deep learning (DL) based augmented reality (AR) system for identifying critical departures from the pristine state of the structure with focus on two anomaly categories- corrosion and fatigue cracks. Most of the common AR devices usually come with a built-in camera for capturing image/video data, a storage and a microprocessor. However, due to the limited processing power, the underlying deep learning (DL) model has to be first trained externally and a suitable version of the trained model is then deployed locally on the device. The model then outputs information for identifying critical departures from the pristine state of the structure e.g., highlighting corroded regions, fatigue cracks and/or combination of both. This information is overlaid real time over the current field of view through either a headmounted or a hand-held AR device in order to augment the human vision. The worker can then focus on the highlighted region for a more detailed inspection. The feasibility of the proposed AR system is demonstrated using laboratory inspection of common mechanical components likes pipes, plates etc. In order to enable the model to keep learning based on the inputs from the AR glasses, a strategy for federated learning is introduced towards the end of the paper.

The rise of ubiquitous computing has led to the rapid development of intelligent systems such as augmented reality (AR). AR is being employed for various usage including common tasks such as web browsing, navigating and controlling other internet of things (IoT) devices. In these types of system usage, similar to mobile phones, the presentation of notifications is inevitable. However, studies with regards to the effective delivery of notification signals within the AR environment is still lacking. Thus, the present study aims to investigate the effects of different output modalities (visual, auditory and multimodal) for presenting notifications during simple search and selection tasks within the AR system. Twenty individuals participated in a user study where they were asked to perform a simple search and selection task while using the AR device. Participants were asked to confirm the notifications that were presented at random in each trial. Results of the user study showed that multimodal notification signals (both visual and auditory) yielded a higher notification recognition rate and were rated as the most appropriate modality and most preferred among the conditions tested. Also, it was found that speech input modality was used more frequently in response to notifications, regardless of the output type. Several design implications were derived from the findings of the study.