An Efficient Augmented Reality (AR) System for Enhanced Visual Inspection

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Concepts for user interfaces are developed in order to enable humans exchanging information and having a better understanding of different processes. With the rise of technology, concepts of enriching our environments with computer-generated graphics appeared in the 1960s. The expansion of visual perception of the surrounding is the core of Augmented Reality (AR). Many AR applications have been appearing in different areas, opening the door for revolutionizing and changing interactivity models and the way people perform tasks. Even though AR systems’ structures are well defined, applications are not yet widespread. One of the main reasons for this is that AR system developers face difficulties with developing their applications and adapting them to different hardware. Each hardware set (i.e. AR device) requires different software development kits and different technical requirements. That leaves AR concepts hard to be scaled and implemented in real life scenarios. One more additional challenge facing the implementing of AR is that AR mostly relies on complex vision-based algorithms. Based on that, this paper explores a lightweight, modular, and cross-platform solution. It presents a novel framework based on redesigning the AR pipeline, introducing a prototype of a solution where AR can operate within the cloud, converting all AR devices into thin clients with high performance capability. This also contributes directly to the Industry 4.0 aim of preparing the way for digitalized and optimized processes in a connected and integrated network.

Augmented reality: An ecological blend

Problem definition: Firms increasingly use augmented reality (AR) devices to improve their production ramp-up processes. These devices appear useful, yet little is known about their broader impact on worker productivity and behavior. Academic/practical relevance: Efficient production ramp-ups are particularly important when product life cycles are short. An ongoing debate among academics and practitioners pertains to how Industry 4.0, and AR devices in particular, can accelerate the ramp-up. The current study provides empirical evidence related to AR in the production ramp-up context, examines the strengths and weaknesses of AR, and tests four hypotheses, leading to a more nuanced view of AR use in the manufacturing ramp-up. Methodology: A framed field experiment in a manufacturing plant provides a test of how quickly workers can perform new tasks with and without AR support and how the use of AR affects their ability to suggest process improvements. Results: When faced with a new task, workers instructed by AR smart glasses use 43.8% less time to complete the task compared with a control group that relies on paper-based instructions. However, workers that use AR glasses consistently use 23% more time than the control group when both groups repeat the task without either AR or paper-based instructions. Task difficulty moderates this relationship; workers assigned to a more difficult task benefit the most from AR instructions. After the devices are removed, workers instructed based on paper improve their productivity faster through learning than those instructed by AR. In addition, the former group suggests better process improvements than the latter one. Managerial implications: Although these results indicate substantially higher productivity resulting from AR devices, they also support the view that, once instructed through AR devices, workers come to rely on this new technology without fully internalizing the task. This failure to internalize their task then leads workers to suggest less useful process improvements.

ObjectiveThe aim of this study was to investigate the feasibility of using augmented reality (AR) glasses in central line simulation by novice operators and compare its efficacy to standard central line simulation/teaching.DesignThis was a prospective randomized controlled study enrolling 32 novice operators. Subjects were randomized on a 1:1 basis to either simulation using the augmented virtual reality glasses or simulation using conventional instruction.SettingThe study was conducted in tertiary-care urban teaching hospital.SubjectsA total of 32 adult novice central line operators with no visual or auditory impairments were enrolled. Medical doctors, respiratory therapists, and sleep technicians were recruited from the medical field.Measurements and main resultsThe mean time for AR placement in the AR group was 71±43 s, and the time to internal jugular (IJ) cannulation was 316±112 s. There was no significant difference in median (minimum, maximum) time (seconds) to IJ cannulation for those who were in the AR group and those who were not (339 [130, 550] vs 287 [35, 475], p=0.09), respectively. There was also no significant difference between the two groups in median total procedure time (524 [329, 792] vs 469 [198, 781], p=0.29), respectively. There was a significant difference in the adherence level between the two groups favoring the AR group (p=0.003).ConclusionAR simulation of central venous catheters in manikins is feasible and efficacious in novice operators as an educational tool. Future studies are recommended in this area as it is a promising area of medical education.

The 10-m walk test (10MWT) is a stopwatch-based clinical mobility assessment. To better understand mobility limitations, 10MWT test completion times may be complemented with gait parameters like step length. State-of-the-art augmented reality (AR) glasses can potentially do this given their unique 3D-positional data from which gait parameters may be derived. We examined the test-retest reliability, concurrent validity, and face validity of gait parameters derived from AR glasses during a 10MWT in 20 people with Parkinson's disease, performed at self-selected comfortable and fast-but-safe walking speeds. AR-derived 10MWT completion times and gait parameters (mean step length, cadence, and maximal gait speed) were compared across repetitions and with lab-based (Interactive Walkway) and clinical (stopwatch) reference systems. Good-to-excellent test-retest reliability statistics were observed for test completion times and gait parameters for all systems and conditions alike. Concurrent validity was demonstrated between AR, lab-based, and clinical references for test completion times (good-to-excellent agreement: ICC > 0.879) and gait parameters (excellent agreement: ICC > 0.942). Face validity was confirmed by significant differences in test completion times and gait parameters between speed conditions in a-priori expected directions. These findings support the conclusion that gait parameters can be derived reliably and validly from AR glasses in people with Parkinson's disease.

This paper discusses the development of Virtual Control Interface (VCI), a Virtual Reality (VR) system for simulating and exploring multimodal context-sensitive interactions between Augmented Reality (AR) glasses and Internet of Things (IoT) devices.Current trends in controlling IoT devices focus on using existing control interfaces such as smartphones. Parallel technological trends indicate that AR glasses are likely to become the next consumer terminal. Therefore, there is a need to explore a new paradigm for IoT control interfaces that leverages the benefits of AR.Towards developing future-proof AR-IoT interfaces, we observed the HCI community’s challenges in using existing hardware to explore the full potential and the limitations of future AR-IoT interfaces. That became the foundation for developing VCI.This paper provides an overview of the advantages and shortcomings of developing such interfaces on smartphones, state-of-the-art AR/Mixed Reality (MR) glasses, and our proposed VR simulated AR environment. This overview outlines how our proposed system could be a superior choice for researchers to investigate, challenge assumptions, prototype, develop, and experimentally test AR-IoT interactions and interfaces with currently available technologies.Furthermore, we present the three pillars we followed to guide the design and development of VCI to make it resemble an actual AR-IoT framework as much as possible while abstracting parts that have little to no effect on AR-IoT HCI research. These pillars are: design transferable architecture, use viable multimodal interactions and explore user interfaces in real-life scenarios.These three principles informed the VCI’s system architecture built upon three main subsystems: handling multi-modal user input, IoT device emulation and connectivity, and managing the AR user interface (UI).Finally, we provide a heuristic evaluation and future work for VCI.KeywordsSmart homeInternet of ThingsAugmented realityVirtual realityAR-IoTMultimodal interactionVirtual Control Interface

Introduction: Augmented Reality (AR) systems are systems in which users view and interact with virtual objects overlaying the real world. AR systems are used across a variety of disciplines, i.e., games, medicine, and education to name a few. Optical See-Through (OST) AR displays allow users to perceive the real world directly by combining computer-generated imagery overlaying the real world. While perception of depth and visibility of objects is a widely studied field, we wanted to observe how color, luminance, and movement of an object interacted with each other as well as external luminance in OST AR devices. Little research has been done regarding the issues around the effect of virtual objects’ parameters on depth perception, external lighting, and the effect of an object’s mobility on this depth perception.Methods: We aim to perform an analysis of the effects of motion cues, color, and luminance on depth estimation of AR objects overlaying the real world with OST displays. We perform two experiments, differing in environmental lighting conditions (287 lux and 156 lux), and analyze the effects and differences on depth and speed perceptions.Results: We have found that while stationary objects follow previous research with regards to depth perception, motion and both object and environmental luminance play a factor in this perception.Discussion: These results will be significantly useful for developers to account for depth estimation issues that may arise in AR environments. Awareness of the different effects of speed and environmental illuminance on depth perception can be utilized when performing AR or MR applications where precision matters.

Reality check

The development and structuring of an augmented reality (AR) platform integrating image processing, deep learning, and spatial tracking techniques have yielded promising results. AR technology, which enhances real-world objects with computer-generated information, holds vast potential for education, entertainment, and industry. Image processing enables real-time object detection, recognition, and tracking within the user's environment, while deep learning models enhance pattern recognition accuracy. Spatial tracking techniques ensure seamless integration of virtual content with the physical world, providing immersive AR experiences. Ant colony optimization (ACO) algorithms further enhance the AR platform's functionality by optimizing object placement and interaction. ACO mimics ant behavior to find optimal paths through graphs, effectively optimizing spatial layout and user interaction patterns in dynamic environments. The integration of ACO into the AR platform facilitates efficient pathfinding for augmented objects and enhances the system's responsiveness to user inputs. The successful implementation of the AR platform underscores its potential to revolutionize various industries, including healthcare, education, and marketing. By leveraging AI-driven algorithms and optimization techniques, AR systems can deliver personalized and engaging experiences that blur the boundaries between the digital and physical worlds. These results pave the way for future advancements in AR technology, driving innovation and reshaping human-computer interaction paradigms.

The rise of ubiquitous computing has led to the rapid development of intelligent systems such as augmented reality (AR). AR is being employed for various usage including common tasks such as web browsing, navigating and controlling other internet of things (IoT) devices. In these types of system usage, similar to mobile phones, the presentation of notifications is inevitable. However, studies with regards to the effective delivery of notification signals within the AR environment is still lacking. Thus, the present study aims to investigate the effects of different output modalities (visual, auditory and multimodal) for presenting notifications during simple search and selection tasks within the AR system. Twenty individuals participated in a user study where they were asked to perform a simple search and selection task while using the AR device. Participants were asked to confirm the notifications that were presented at random in each trial. Results of the user study showed that multimodal notification signals (both visual and auditory) yielded a higher notification recognition rate and were rated as the most appropriate modality and most preferred among the conditions tested. Also, it was found that speech input modality was used more frequently in response to notifications, regardless of the output type. Several design implications were derived from the findings of the study.

This paper primarily focuses on the research of industrial component surface defect detection algorithms using deep learning techniques. Taking the example of defect detection in the injector valve seat of automotive engine components, currently, the inspection of such small components heavily relies on manual visual inspection or traditional machine vision methods. Manual visual inspection is inefficient and cannot guarantee the speed and accuracy of detection. Traditional machine vision methods, while efficient, are sensitive to harsh environmental conditions and can be influenced by factors such as lighting, camera positioning, and background, resulting in poor robustness and limited feature extraction capabilities. To address these issues, this paper conducts research on a deep learning-based surface defect detection algorithm for injector valve seats.

Effective rendering technique of auditory information for an augmented reality (AR) device has been investigated. Researchers and industries are searching for new applications incorporating AR and Virtual Reality (VR) technologies, which provide enhanced user-experience of interactivity and intimacy. To utilize these benefits best, homogeneous integration between visual and auditory information is important. To date, the binaural technology based on head-related transfer function (HRTF) has been used to create immersive and three-dimensional audio objects for VR and AR devices. In this study, we compared the HRTF method with a stereophonic panning method that only controls Inter-aural Level Difference (ILD) for an AR device (a smart see-through glass) and investigated the precision of auditory information required for a coherent representation with a target visual image at the locations of 0 degree, -5 degrees, and -10 degrees in a counter clockwise. Auditory stimuli were rendered to have target locations in the horizontal plane from + 45 degrees to -45 degrees with a 5-degree interval. In the subjective evaluation, each participant answered whether a randomly given auditory location was matched to a target location of the visual image. The results showed that two audio rendering methods did not produce significant difference in creating integrated perception in an AR device.

Augmented reality (AR) systems can be effectively used to enhance manufacturing and industrial processes. However, not all the existing prototypes of AR systems can be used in an industrial environment due to heavy constraints such as low robustness or cumbersome equipment. Our AR system relies purely on passive techniques to solve the real-time registration problem, and it can run on a portable computer. We combined a powerful virtual reality (VR) component-based simulation framework with computer vision techniques, turning it into an AR system. The resulting system allows us to produce complex rendering and animation of avatars, and blend them into the real world. The system tracks the 3D camera position by means of a natural features tracker, which, given a rough CAD model, can deal with complex 3D objects. The tracking method is robust and can handle large camera displacements and aspect changes. The target applications of our AR system are industrial maintenance, repair and training. The tracking robustness makes the AR system able to work in real environments, such as industrial facilities, and not only in the laboratory.

Virtual reality (VR) and augmented reality (AR) systems are increasingly available to neurosurgeons. These systems may provide opportunities for technical rehearsal and assessments of surgeon performance. The assessment of neurosurgeon skill in VR and AR environments and the validity of VR and AR feedback has not been systematically reviewed. A systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines was conducted through MEDLINE and PubMed. Studies published in English between January 1990 and February 2021 describing the use of VR or AR to quantify surgical technical performance of neurosurgeons without the use of human raters were included. The types and categories of automated performance metrics (APMs) from each of these studies were recorded. Thirty-three VR studies were included in the review; no AR studies met inclusion criteria. VR APMs were categorized as either distance to target, force, kinematics, time, blood loss, or volume of resection. Distance and time were the most well-studied APM domains, although all domains were effective at differentiating surgeon experience levels. Distance was successfully used to track improvements with practice. Examining volume of resection demonstrated that attending surgeons removed less simulated tumor but preserved more normal tissue than trainees. More recently, APMs have been used in machine learning algorithms to predict level of training with a high degree of accuracy. Key limitations to enhanced-reality systems include limited AR usage for automated surgical assessment and lack of external and longitudinal validation of VR systems. VR has been used to assess surgeon performance across a wide spectrum of domains. The VR environment can be used to quantify surgeon performance, assess surgeon proficiency, and track training progression. AR systems have not yet been used to provide metrics for surgeon performance assessment despite potential for intraoperative integration. VR-based APMs may be especially useful for metrics that are difficult to assess intraoperatively, including blood loss and extent of resection.