Augmented Reality for Building Maintenance and Operation

This article examines the impact of augmented reality (AR) visualizations on users’ sense of physical presence, knowledge gain, and perceptions of the authenticity of journalistic visuals. In a mixed experimental design, 79 participants were randomly assigned to view three The New York Times articles on a mobile phone featuring one of three viewing modalities: (1) AR visualizations, (2) interactive (non-AR) visualizations, or (3) non-interactive, static visualizations. AR induced a greater sense of physical presence compared to the other modalities. The findings suggest that immersive properties of AR can contribute to journalism’s goal of engaging the audience. However, AR was not a superior medium for informing the participants, and the viewing modality did not have an effect on the perceived authenticity of the visuals. The findings indicate a need for more efficient ways to relay information through journalistic AR visualizations while keeping the user engaged in an immersive experience.

Team communication is crucial in multi-domain operations (MDOs) that require teammates to collaborate on complex tasks synchronously in dynamic unknown environments. In order to enable effective communication in human-robot teams, the human teammate must have an intuitive interface that supports and satisfies the time-sensitive nature of the task for communicating information to and from their robot teammate. Augmented Reality (AR) technologies can provide just such an interface by providing a medium for both active and passive robot communication. In this paper we propose a new Virtual Reality (VR) based framework for authoring AR visualizations, and demonstrate the use of this framework to produce AR visualizations that help facilitate high task performance in synchronized, time-dominant human-robot teaming. In this paper we propose a new framework that uses a virtual reality (VR) simulation environment for developing AR strategies as well as present a AR solution for maximizing task performance in synchronized, time-dominant human-robot teaming. The framework utilizes a Unity-based VR simulator that is run from the first person point of view of the human teammate and overlays AR features to virtually imitate the use of an AR headset in human-robot teaming scenarios. Then, we introduce novel AR visualizations that support strategic communication within teams by collecting information from each teammate and presenting it to the other in order to influence their decision making. Our proposed design framework and AR solution has the potential to impact any domain in which humans conduct synchronized multi-domain operations alongside autonomous robots in austere environments, including search and rescue, environmental monitoring, and homeland defense.

Software visualization is an ongoing research area with the goal to make software systems more tangible. In this paper we present our visualization system that allows visual analysis of graph structures representing software systems in both virtual and augmented reality. The provided visual analysis process is demonstrated on two existing software systems. We propose the transition towards visualization in augmented reality and we present developed interaction techniques to explore and examine software visualizations in augmented reality.

Augmented Reality (AR) and Internet of Things (IoT) devices can converge, we call it AR + IoT. The integration of AR and IoT has promising opportunities for future product research and development. IoT is a tool for communicating, monitoring, and remotely controlling with connected products, while AR is a means to see and find out what is happening with connected products based on visualization/interface based on additional information. The mapping review method was used to carry out this research. The result is that as many as 43 papers from 2014-2020 are the main targets of review. The largest publication database is in the IEEE Xplore Digital Library, the application dominates in the field of social life, the research objectives are varied, and the recommended future work is to maximize the potential for integration between AR and IoT or even with other promising devices. Then comfort, convenience, and user experience are also recommended in implementing AR + IoT

AR/VR assisted integrated framework of autonomous disassembly system for industrial products

Virtual Reality (VR) and Augmented Reality (AR) technologies have emerged as powerful tools with the potential to revolutionize the field of education. This study provides an overview of the impact of VR and AR in education, drawing from existing research and analytical applications. The impact of VR in education is evident in its ability to create immersive and engaging learning experiences. Students can be transported to virtual environments that facilitate experiential learning, from exploring historical landmarks to simulating complex scientific experiments. The result is enhanced student engagement, improved retention of knowledge, and the development of practical skills. AR, on the other hand, overlays digital information onto the real world, enriching traditional educational materials. Interactive textbooks and guided field trips using AR provide students with dynamic and context-rich learning experiences. Language learners benefit from instant translations and pronunciation guides, while complex data becomes tangible through AR visualizations. Both VR and AR contribute to the promotion of collaborative learning, enabling students and educators to interact in virtual environments. These technologies offer new opportunities for special needs education, as they can be customized to cater to the unique requirements of individual learners. The adoption of VR and AR in education comes with challenges, including cost considerations, the need for suitable technical infrastructure, content development complexities, and the necessity to address safety concerns. Integrating VR and AR into the existing curriculum and ensuring alignment with educational objectives are also ongoing challenges. The study aim is to analyse the impact of VR and AR in education.

This study presents the development and evaluation of ARTechGuide, an Augmented Reality (AR)-based application designed for technical training of operators in an automotive industrial environment. The research was conducted using the Design Science Research (DSR) methodology, with application and validation in a real factory setting. The system uses mobile devices and visual recognition to provide audiovisual instructions in five critical operational scenarios. Compared to traditional methods, a 57 % reduction in setup time, a 25 % increase in execution accuracy, and a 75 % decrease in requests for help were observed. The system achieved an average of 9.25 on the User Experience (UX) scale and 4.5 on nine technical indicators evaluated by experts. Unlike commercial AR platforms, the solution adopts a local architecture, independent of constant connectivity, facilitating its adoption in environments with operational constraints. These benefits are aligned with the Industry 5.0 proposal, which values customization according to human needs.

Correctly identifying and classifying food is decisive in food safety. The food sector is constantly evolving, and one of the technologies that stands out is augmented reality (AR). During practical studies at Companhia de Entreposto e Armazéns Gerais de São Paulo (CEAGESP), responsible for the largest food storage in South America, difficulties were identified in classifying aromatic herbs due to the large number of species. The project aimed to create an innovative AR application called ARomaticLens to solve the challenges associated with identifying and classifying aromatic herbs using the design science research (DSR) methodology. The research was divided into five stages according to the DSR methodology, from surveying the problem situation at CEAGESP to validating the application through practical tests and an experience questionnaire carried out by CEAGESP specialists. The result of the study presented 100% accuracy in identifying the 18 types of aromatic herbs studied when associated with the application’s local database without the use of an Internet connection, in addition to a score of 8 on a scale of 0 to 10 in terms of the usability of the interface as rated by users. The advantage of the applied method is that the app can be used offline.

Today's sailing visualization instruments struggle to cope with the increasing number of onboard sensors, automation, artificial intelligence, and the high dynamism of the crew. Current solutions scatter multiple displays all over the boat, both inside and outside, potentially reducing usability and increasing costs. This work presents a systematic review of augmented reality (AR) as an integral solution for sailing data visualization, which revealed four scientific papers and eight commercial products. We analyzed the publication type, the AR hardware, what and how information is presented using AR, the validation method (if present), and the technological readiness. We defined the technical requirements needed for the AR device for sailing and distinguished a first generation of commercial solutions based on head-up displays from a second one based on proper augmentation with stereo head-mounted displays. The displayed information elements are limited in number and are commonly 2-D graphics (e.g., text and symbols) with a screen-relative frame of reference (as opposed to body- or world-relative). The most visualized elements are heading (10) followed by wind direction (8), boat speed (7) compass (7), and wind speed (7). We also found that most of the solutions lack critical evaluation. We conclude that AR has the potential to integrate sailing data from different systems and to improve accessibility, situation awareness, and safety for a large group of users. However, the main limitations are the lack of AR head-mounted displays suitable or adaptable for sailing conditions, an extensive exploration of 3-D interface elements, and an adequate number of usability studies in the scientific literature.

Image-guided surgery (IGS) has allowed for more minimally invasive procedures, leading to better patient outcomes, reduced risk of infection, less pain, shorter hospital stays and faster recoveries. One drawback that has emerged with IGS is that the surgeon must shift their attention from the patient to the monitor for guidance. Yet both cognitive and motor tasks are negatively affected with attention shifts. Augmented reality (AR), which merges the realworld surgical scene with preoperative virtual patient images and plans, has been proposed as a solution to this drawback. In this work, we studied the impact of two different types of AR IGS set-ups (mobile AR and desktop AR) and traditional navigation on attention shifts for the specific task of craniotomy planning. We found a significant difference in terms of the time taken to perform the task and attention shifts between traditional navigation, but no significant difference between the different AR set-ups. With mobile AR, however, users felt that the system was easier to use and that their performance was better. These results suggest that regardless of where the AR visualisation is shown to the surgeon, AR may reduce attention shifts, leading to more streamlined and focused procedures.

For the past two decades, medical Augmented Reality visualization has been researched and prototype systems have been tested in laboratory setups and limited clinical trials. Up to our knowledge, until now, no commercial system incorporating Augmented Reality visualization has been developed and used routinely within the real-life surgical environment. In this paper, we are reporting on observations and analysis concerning the usage of a commercially developed and clinically approved Freehand SPECT system, which incorporates monitor-based Mixed Reality visualization, during real-life surgeries. The workflow-based analysis we present is focused on an atomic sub-task of sentinel lymph node biopsy. We analyzed the usage of the Augmented and Virtual Reality visualization modes by the surgical team, while leaving the staff completely uninfluenced and unbiased in order to capture the natural interaction with the system. We report on our observations in over 100 Freehand SPECT acquisitions within different phases of 52 surgeries.

For the past two decades, medical Augmented Reality visualization has been researched and prototype systems have been tested in laboratory setups and limited clinical trials. Up to our knowledge, until now, no commercial system incorporating Augmented Reality visualization has been developed and used routinely within the real-life surgical environment. In this paper, we are reporting on observations and analysis concerning the usage of a commercially developed and clinically approved Freehand SPECT system, which incorporates monitor-based Mixed Reality visualization, during real-life surgeries. The workflow-based analysis we present is focused on an atomic sub-task of sentinel lymph node biopsy. We analyzed the usage of the Augmented and Virtual Reality visualization modes by the surgical team, while leaving the staff completely uninfluenced and unbiased in order to capture the natural interaction with the system. We report on our observations in over 100 Freehand SPECT acquisitions within different phases of 52 surgeries.

Accompanying the advent of wireless networking and the Internet of Things (IoT), traditional augmented reality (AR) systems to visualize virtual 3D models of the real world are evolving into smart and interactive AR related to the context of things for physical objects. We propose the integration of AR and IoT in a complementary way, making AR scalable to cover objects everywhere with an acceptable level of performance and interacting with IoT in a more intuitive manner. We identify three key components for realizing such a synergistic integration: (1) distributed and object-centric data management (including for AR services); (2) IoT object-guided tracking; (3) seamless interaction and content interoperability. We survey the current state of these respective areas and herein discuss research on issues about realizing a future smart and interactive living environment.

Digital transformation has become a buzzword across many industries, and the dental field is no exception. Based on electronic health data, digital transformation is acknowledged as one of the biggest game-changers of the twenty-first century, tackling both current and future challenges in dental and oral healthcare. A novel method for addressing today’s major healthcare issues, such as an ageing population with a higher incidence of chronic diseases and higher lifetime treatment costs, is offered by the utilization of digital tools and apps. Health care providers may improve patient satisfaction, develop loyalty and trust, and streamline operations with the aid of digitalization. The trend of digitalization has also been influenced and nurtured by the social and cultural habits of civilised society in industrialised nations. These behaviours include urbanism, centralization, mobility, and constant accessibility through the use of smartphones, tablets, and the internet of things (IoT). In order to ensure transparency for all parties involved—including patients, healthcare providers, universities and research institutions, the medtech industry, insurance, the public media, and state policy— digital dentistry necessitates managing expectations in a more pragmatic and realistic manner. It is not to be construed that digital smart data technology will eventually take the place of people who can provide dental competence and patient empathy. The dental team in charge of digital applications is still crucial to patient care and will always be so. Many difficulties arise in the process of gathering, storing, and analysing digital biological patient data. Safeguarding patient data for optimal safety requires not just technical considerations for managing massive volumes of data, but also adherence to globally established norms and ethical guidelines.1 There are four main categories that best describe the advantages of digital dental technologies in Prosthodontics. The first and foremost is improved communication. Dental laboratory staff, patients, dentists, and other stakeholders may all communicate clearly with the help of computerized patient records. Furthermore, digital radiographs and pictures depicting intraoral conditions improve the information exchange between medical professionals and patients. One of the main benefits of integrated electronic patient records is error-free, real-time communication. Enhanced record keeping, data fidelity, workflow efficiency, and therapeutic outcomes are among the benefits of increased quality. Real-time clinical improvement is made possible by intraoral scanning of tooth preparations that are examined in highly contrasted, magnified fields on a computer screen while the patient is in direct view. Data archiving for specific patients is the third benefit. The following are some benefits of using 3-D archived diagnostic casts: a) long-lasting images without causing damage or loss to the original casts; b) allowing the images to be interfaced with other images for analysis using cutting-edge analytical and design tools; c) removing human error; and d) reduced costs for storage. The fourth and most significant advantage of digital technology in Prosthodontics is its favourable effect on patient satisfaction. More advanced treatment plans are driven by the enhanced diagnostic data. Several factors, such as time constraints, IT support costs, a lack of basic computer skills, disruptions to workflow, privacy and security concerns, interprofessional and intersystem connections, and technical and expert support, are impeding the adoption of digital technology in Prosthodontics.2,3 Several digital processes for production processing in Prosthodontics have already been incorporated into treatment procedures, particularly in the quickly expanding fields of rapid prototyping (RP) and computer-aided design/computer-aided manufacture (CAD/ CAM). Artificial intelligence (AI) and machine learning (ML) have created new opportunities for automated processing in radiological imaging. Furthermore, the technology underlying the superimposition of various imaging files to create virtual dentistry patients and non-invasive simulations comparing various outcomes before any clinical intervention is known as augmented and virtual reality, or AR/VR. These exciting new technologies—whose potential applications are still up in the air—have been made conceivable by increased IT capability.1,2,3 The process of rapidly and autonomously creating three-dimensional (3D) models of a finished product or a component of a whole using 3D printers is known as rapid prototyping. Complex 3D geometries can be produced at a reasonable cost with minimal material waste, thanks to the additive manufacturing technique. The workpiece is virtually sliced into multiple two dimensional layers. The tool-path is then generated by the AM machine in both the x and y dimensions. A three-dimensional component is formed by sequentially depositing each material layer on top of the other. The foundation of this novel approach is the slicing of a three-dimensional CAD model into numerous thin layers, which are then built one after the other by manufacturing machinery using the geometric data. Dental technology can benefit greatly from RP’s mass manufacture of dental models and its ability to fabricate implant surgical guides. Large-scale, simultaneous production in a repeatable, standardized manner is highly advantageous from an economic perspective.4 Augmented reality, or AR, is an interactive technology that uses computer-animated perceptual data to enhance a real-world experience. Stated differently, augmented reality is the addition of virtual content to the physical world. Usually, it involves superimposing extra digital data on real-time pictures or movies. In contrast, virtual reality relies solely on artificial, non-reality-connected computerized settings. Every imaginable form of sensation, primarily visual, aural, and haptic, can be employed alone or in any combination, depending on the technique. In addition to several fascinating advancements for patients and healthcare professionals, AR/VR technologies are currently finding a growing number of applications in the field of Prosthodontics as a whole.5,6 Artificial Intelligence has come a long way in the last ten years. The field of Prosthodontics is about to benefit from the most intriguing AI applications that are just around the corner. Though AI is developing quickly, it will never be able to fully replace human intelligence, skill, or capacity to make decisions. Artificial Intelligence (AI) in prosthodontics is growing exponentially. The implementation’s results are on par with, and sometimes even better than, those of humans. AI can be seen as a potential tool in every area, including the identification of marginal lines, the classification of denture fixtures and maxillofacial prosthesis, and the reduction of human error in implant cementation. Furthermore, AI cannot take the role of human knowledge, skill, or treatment planning; it can only support clinicians in carrying out their responsibilities in a professional manner. AI is generally recognized as a great tool for Prosthodontists, despite the fact that there are still obstacles to be addressed, including data collection, interpretation, computing power, and ethical issues. With careful design and long-term clinical validation, AI can be transparent, unbiased, repeatable, and user-friendly.7,8 Future research should emphasize the connection between oral and general health in order to concentrate on patient-centered outcomes and personalized therapy. Research in Prosthodontics ought to be useful to society in this context. It shouldn’t only produce papers for scientific journals; instead, it should aim to improve clinical protocols. Research and development in material science and related technical applications aim to preserve tooth structures with early diagnosis, repair of dental conditions to attain aesthetics, function with high degree of predictability, along with fewer appointments. Digital technology has a significant impact on patient motivation, clinical aspects, laboratory procedures, student training, practice management, and research.

Chapter Nine - Envisioning big data in IoT with augmented and virtual reality: challenges,opportunities, and potential solutions