Behavioral Patterns and Learning Performance of Collaborative Knowledge Construction on an Augmented Reality System

The major challenges of augmented reality (AR) systems and virtual reality (VR) systems are varifocal images for vergence-accommodation conflict (VAC) and vision corrections. In this paper, we design a liquid crystal (LC) lens set consisting of three LC lenses for varifocal images and vision corrections in AR and VR. Four operating modes of such a LC lens set present three electrically tunable lens powers: 0, -0.79 diopters, -2 diopters, and -3.06 diopters by means of manipulation of polarization of incident light using electrically tunable half-wave-plates. The response time is fast(< 50 ms). We also demonstrate AR and VR systems by adopting the LC lens set to exhibit functions of varifocal images and vision corrections which enable to solve VAC as well as vision problem in AR and VR.

Augmented reality (AR) systems can be effectively used to enhance manufacturing and industrial processes. However, not all the existing prototypes of AR systems can be used in an industrial environment due to heavy constraints such as low robustness or cumbersome equipment. Our AR system relies purely on passive techniques to solve the real-time registration problem, and it can run on a portable computer. We combined a powerful virtual reality (VR) component-based simulation framework with computer vision techniques, turning it into an AR system. The resulting system allows us to produce complex rendering and animation of avatars, and blend them into the real world. The system tracks the 3D camera position by means of a natural features tracker, which, given a rough CAD model, can deal with complex 3D objects. The tracking method is robust and can handle large camera displacements and aspect changes. The target applications of our AR system are industrial maintenance, repair and training. The tracking robustness makes the AR system able to work in real environments, such as industrial facilities, and not only in the laboratory.

Teaching, learning and training in assembly or disassembly as well as maintenance operation to new users, employees, engineers or technicians is important for various education sector and industries for them to acquire new skills. However to train those users to efficiently perform new skills is become challenging. It is important to reduce the user's training and working times to avoid cost issues in training, material, unavailable expertise and avoid actual part damage. Therefore, this study is conducted to design and develop a Virtual Reality (VR) and Augmented Reality (AR) system for learning and training platform. The valve assembly system was selected as a task for mechanical assembly system. This study was designed to find the influence of interactive learning method using AR system. The outcome from this study showed that learning and training in AR and VR system resulted in better to user skill performance. User learn and trained in a ARVR group gives positive feedback; very easy, practical and as a solution of the training process without using the actual one. While learning using interactive learning method such as AR resulted with better knowledge with less assembly error.

1 IntroductionNowadays, the institutes already use in the process of the advantages of mobile technology, such as portability (the students have easy access at any time to information), the use of digital assets (audio and video files) and huge amount of textual information. This study proposes a new system approach for improving the process by Mobile Augmented Reality. The features of this technology stimulate the environment and consider that the student's motivation is the most important factor in learning.The existing literature [1] highlights two different approaches for augmented reality (AR). Firstly, the authors consider that AR is a system which has the following characteristics: Combines real and virtual; Interactive in real time; Registered in 3-D. Secondly [1] considers Augmented Reality part of Virtual Continuum. This context is composed of two elements: Real Environment - which contains all representations from real life and Virtual Environment - which combines different elements from virtual life. ARs are considered closer to reality than virtual environment because this type of systems are supposed to extend the real objects with virtual elements.According to [2] Augmented Reality is a technology that extends the real world elements with 2D or 3D computer generated objects and lets the users interact with them. Also, the authors present three types of devices which can be used to view the result of AR technology: a Head Mounted Display (HMD), a mobile device - with at least one back camera and a desktop or a laptop computer.Similarly, [2] considers mobile technology as an alternative teaching resource because the users have easy access in real time to information. Also, mobile devices in the process two main problems of traditional are solved: the location and time. Likewise, [1] considers that mobile is a type of eLearning without restrictions of time and space and this technology improves the quality of classical environments, but does not replace them.Other researches, such as [3] and [4] highlight the benefits of a Mobile Augmented Reality system based on tracking objects. In [3] is proposed a mobile augmented reality system based on visual recognition. According to the authors the visual recognition consists into tracking objects identified in real world with the camera of the mobile device and augmenting the scene with virtual objects created by MAR technology. Similarly, [4] presents a Mobile Augmented Reality system bases on 3D registration technology. According to the authors the 3D registration represents the tracking of the position and pose of the mobile device in the real time and real life and finally use these information to insert in the real world the virtual scene created by MAR. The 3D registration technology is based on two type of approaches: markers and natural features. Recently, the 3D registration for MAR uses GPS and sensors from mobile devices to provide different information about position and pose of the camera and combine them with computer vision to improve the speed and accuracy of the system.Other studies, such as [5] propose a makerbased Mobile Augmented Reality solution. The application is used by a Vocational Educational and Training (VET) program for supporting the maintenance process of a car. The authors define VET as education programs that are designed for learners to acquire the knowledge, skills and competencies specific to a particular occupation, trade or class of occupations or trades(p. 49-50). Moreover, in [5] is presented an AR application used by VET students for basic mathematics skills, which is based on videogames.More studies, such as [6] present a Mobile Augmented Reality solution for French language which is based on gamification didactic method. The author, Bernadette Perry, describes gamification as using gamebased mechanics, aesthetics and game thinking to engage people, motivate action, and promote learning (p. …

This article presents a mobile Augmented Reality system, called DiedricAR, aimed at the learning of Descriptive Geometry. Thanks to its ability to recreate virtual models in real space, Augmented Reality is a technology suitable for making Descriptive Geometry comprehension and interpretation easier. The DiedricAR application allows students to learn in autonomously way by using their own mobile devices (smartphones and tablets), that work as Augmented Reality displays over training material (DiedricAR exercise workbook) specially designed for the new learning model defined by the European Higher Education System. Compared to some of the existing Augmented Reality systems used to learn Descriptive Geometry, DiedricAR offers the advantage of being specifically developed for mobile devices giving the students the possibility of using ubiquitous learning to its ultimate extent by interacting with the didactical content (i.e. showing the desired intermediate step when solving dihedral exercises). The presentation of DiedricAR is completed by exploring some key items such as the potential benefits for students' spatial ability, the relationship between application design and user experience, and software performance on several mobile devices.

Augmented Reality Systems (ARS) attempt to enhance humans’ perception of their indoors and outdoors working and living environments and understanding of tasks that they need to carry out. The enhancement is effected by complementing the human senses with virtual input. For example, when the human visual sense is enhanced, an ARS allows virtual objects to be superimposed on a real world by projecting the virtual objects onto real objects. This provides the human user of the ARS with additional information that he/she could not perceive with his/her senses. In order to receive the virtual input and sense the world around them augmented with real time computer-generated features, users of an ARS need to wear special equipment, such as head-mounted devices or wearable computing gears. Tracking technologies are very important in an ARS and, in fact, constitute one challenging research and development topic. Tracking technologies involve both hardware and software issues, but in this chapter we focus on tracking computation. Tracking computation refers to the problem of estimating the position and orientation of the ARS user’s viewpoint, assuming the user to carry a wearable camera. Tracking computation is crucial in order to display the composed images properly and maintain correct registration of real and virtual worlds. This tracking problem has recently become a highly active area of research in ARS. Indeed, in recent years, several approaches to vision-based tracking using a wearable camera have been proposed, that can be classified into two main categories, namely “marker-based tracking” and “marker-less tracking.” In this chapter, we provide a concise introduction to vision-based tracking for mobile ARS and present an overview of the most popular approaches recently developed in this research area. We also present several practical examples illustrating how to conceive and to evaluate such systems.

Augmented reality (AR) is a new human-computer interaction technology that combines virtual reality, computer vision, and computer networks. With the rapid advancement of the medical field towards intelligence and data visualisation, AR systems are becoming increasingly popular in the medical field because they can provide doctors with clear enough medical images and accurate image navigation in practical applications. However, it has been discovered that different display types of AR systems have different effects on doctors' perception of the image after virtual-real fusion during the actual medical application. If doctors cannot correctly perceive the image, they may be unable to correctly match the virtual information with the real world, which will have a significant impact on their ability to recognise complex structures. This paper uses Citespace, a literature analysis tool, to visualise and analyse the research hotspots when AR systems are used in the medical field. A visual analysis of the 1163 articles retrieved from the Web of Science Core Collection database reveals that display technology and visualisation technology are the key research directions of AR systems at the moment. This paper categorises AR systems based on their display principles, reviews current image perception optimisation schemes for various types of systems, and analyses and compares different display types of AR systems based on their practical applications in the field of smart medical care so that doctors can select the appropriate display types based on different application scenarios. Finally, the future development direction of AR display technology is anticipated in order for AR technology to be more effectively applied in the field of smart medical care. The advancement of display technology for AR systems is critical for their use in the medical field, and the advantages and disadvantages of various display types should be considered in different application scenarios to select the best AR system.

Abstract. The attempt to work out the geomatic workflow of transforming low-level aerial imagery obtained with unmanned aerial vehicles (UAV) into a digital terrain model (DTM) and implementing the 3D model into the augmented reality (AR) system constitutes the main problem discussed in this article. The authors suggest the following workflow demonstrated in Fig. 1.The series of pictures obtained by means of UAV equipped with a HD camera was the source of data to be worked out in the final stage of the geovisualization. The series was then processed and a few point clouds were isolated from it, being later used for generating test 3D models.The practical aim of the research conducted was to work out, on the basis of the UAV pictures, the 3D geovisualization in the AR system that would depict the heap of the natural aggregate of irregular shape. The subsequent aim was to verify the accuracy of the produced 3D model. The object of the study was a natural aggregate heap of irregular shape and denivelations up to 11 meters.Based on the obtained photos, three point clouds (varying in the level of detail) were generated for the 20 000-meter-square area. The several-centimeter differences observed between the control points in the field and the ones from the model might corroborate the usefulness of the described algorithm for creating large-scale DTMs for engineering purposes. The method of transformation of pictures into the point cloud that was subsequently transformed into 3D models was employed in the research, resulting in the scheme depicting the technological sequence of the creation of 3D geovisualization worked out in the AR system. The geovisualization can be viewed thanks to a specially worked out mobile application for smartphones.

Virtual reality (VR) and augmented reality (AR) systems are increasingly available to neurosurgeons. These systems may provide opportunities for technical rehearsal and assessments of surgeon performance. The assessment of neurosurgeon skill in VR and AR environments and the validity of VR and AR feedback has not been systematically reviewed. A systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines was conducted through MEDLINE and PubMed. Studies published in English between January 1990 and February 2021 describing the use of VR or AR to quantify surgical technical performance of neurosurgeons without the use of human raters were included. The types and categories of automated performance metrics (APMs) from each of these studies were recorded. Thirty-three VR studies were included in the review; no AR studies met inclusion criteria. VR APMs were categorized as either distance to target, force, kinematics, time, blood loss, or volume of resection. Distance and time were the most well-studied APM domains, although all domains were effective at differentiating surgeon experience levels. Distance was successfully used to track improvements with practice. Examining volume of resection demonstrated that attending surgeons removed less simulated tumor but preserved more normal tissue than trainees. More recently, APMs have been used in machine learning algorithms to predict level of training with a high degree of accuracy. Key limitations to enhanced-reality systems include limited AR usage for automated surgical assessment and lack of external and longitudinal validation of VR systems. VR has been used to assess surgeon performance across a wide spectrum of domains. The VR environment can be used to quantify surgeon performance, assess surgeon proficiency, and track training progression. AR systems have not yet been used to provide metrics for surgeon performance assessment despite potential for intraoperative integration. VR-based APMs may be especially useful for metrics that are difficult to assess intraoperatively, including blood loss and extent of resection.

Introduction: Augmented Reality (AR) systems are systems in which users view and interact with virtual objects overlaying the real world. AR systems are used across a variety of disciplines, i.e., games, medicine, and education to name a few. Optical See-Through (OST) AR displays allow users to perceive the real world directly by combining computer-generated imagery overlaying the real world. While perception of depth and visibility of objects is a widely studied field, we wanted to observe how color, luminance, and movement of an object interacted with each other as well as external luminance in OST AR devices. Little research has been done regarding the issues around the effect of virtual objects’ parameters on depth perception, external lighting, and the effect of an object’s mobility on this depth perception.Methods: We aim to perform an analysis of the effects of motion cues, color, and luminance on depth estimation of AR objects overlaying the real world with OST displays. We perform two experiments, differing in environmental lighting conditions (287 lux and 156 lux), and analyze the effects and differences on depth and speed perceptions.Results: We have found that while stationary objects follow previous research with regards to depth perception, motion and both object and environmental luminance play a factor in this perception.Discussion: These results will be significantly useful for developers to account for depth estimation issues that may arise in AR environments. Awareness of the different effects of speed and environmental illuminance on depth perception can be utilized when performing AR or MR applications where precision matters.

Endoscopic surgery causes less tissue injury compared to open surgical techniques, thus promoting more rapid recuperation and reduced post-operative pain. Endoscopy, however, allows the surgeon to visualise only the anatomical surface of the surgical site, with a relatively narrow field of view. Moreover, the 2D video captured by the conventional endoscope does not provide depth perception of the surgical scene. In this study, these limitations have been addressed with the development of an augmented reality (AR) system with stereoscopic visualisation. A phantom and its 3D CT model were used, respectively, to form the real and virtual parts of the AR. The virtual environment camera pose was tracked using algorithms for image feature detection, feature matching and Perspective-n-Point applied on the endoscopic image and the 3D virtual model-rendered image. The endoscope video frame- and the virtual model-rendered images were superimposed to form the AR composite view. The depth buffer (z-buffer) of the rendering window was further used to make a stereo pair of the AR image. The AR system produced a stereo composite view having well-aligned real and virtual components. The RMS error of the real and virtual image contours registration was 9.6 ± 6.7 mm. Correlation coefficients between the depth map from z-buffer and a depth camera was between 0.60 and 0.96 (p < 0.05). The AR system requires further improvement to be applicable at a higher frame rate of endoscope image acquisition. It also needs to include motion and deformation models when applied to animals or patients.

Augmented reality simulations aims to provide realistic blending between real world and virtual objects. One of the important factors for realistic augmented reality is correct illumination simulation. Mobile augmented reality systems is one of the best options for introducing augmented reality to the mass market due to its low production cost and ubiquitousness. In mobile augmented reality systems, the ability to correctly simulate in realtime the illumination direction that matches the illumination direction of the real world is limited. Developing a mobile augmented reality systems with the ability to estimate illumination direction presents a challenge due to low computation power and dynamically changing environment. In this paper, we described a new method that we have developed for realtime illumination direction estimation for mobile augmented reality systems, using analysis of shadow produced by a reference object that doubles as a 3D augmented reality marker. The implementation of the method could estimate the direction of a single strong light source in a controlled environment with a very good degree of accuracy, with angular error averaging lower than 0.038 radians. The current implementation achieved 2.1 FPS performance in a low-end Android mobile device, produced proper estimation within 15 seconds using a uniform surface, and demonstrated scalability potential.

INTRODUCTION: Augmented Reality (AR) offers the benefit of improved hand-eye coordination by accurately superimposing 3D stereoscopic displays for surgical instruments onto anatomic landmarks. METHODS: 120 pedicle screws were implanted using an AR system (OnPoint AR, OnPoint Surgical, Bedford, MA) (thoracic 50, lumbosacral 70; pre-op CT 40, O-arm 80). Heary grading was applied by an independent radiologist assessing the degree of pedicle breaches. 3D measurements of positional error (PE) and angular error (AE) were performed by comparing post-operative CT scans of implanted screws with planned trajectories. Statistical analyses compared PE and AE for OnPoint AR with the available values for other systems using an unequal variance t-test method. RESULTS: 113 screws placed demonstrated Heary grade 1 accuracy. 6 screws had a larger diameter than the pedicle and were planned with in-out-in technique with Heary grade 2 accuracy. One screw demonstrated a lateral breach (grade 2) caused by a loose connection to the screwdriver. PE and AE (mean ± 1SD) for OnPoint AR were the following: for intra-operative spin 1.1mm±0.5mm and 1.3°±0.5°; for pre-operative CT 1.9mm±1.2mm and 2.2°±1.3°. Percent increment in PE and AE for Medtronic Stealth Station, Brainlab navigation, Stryker nav3i, Medtronic Mazor X robot, Globus Excelsius robot, Augmedics Xvision, and Novarad VisAR compared to OnPoint AR ranged between 40–80%, with differences being highly significant for all comparisons. CONCLUSIONS: PE and AE for placing pedicle screws are statistically significantly smaller for the novel AR system compared to existing navigation, robotic, and AR systems. A positional error of 1.1mm and angular error of 1.3° makes this technology particularly suited for small pedicles, challenging anatomy, and MIS techniques.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

In recent years, hardware and software advances in digital technology have prompted not only the commercial use of augmented reality (AR) systems, but its implementation in education. Marker-based AR systems allow users to input pre-designed markers, and generate new scenes blending together the real world and computer-generated 3D images. Nevertheless, few AR systems permit simultaneous interactions of multiple markers. This study aims to construct an interactive multimedia learning system. The story script showcases the growth of flowers by applying a series of markers, where interactions with 3D objects can be achieved through AR technology. Learners can use a camera to capture the information of multiple AR markers, and 3D objects and media can be generated on-screen. With variations in AR markers, and time changes of the Arduino Clock, one can learn the growth of plants during day and night through a mixed-reality virtual experience that brings fun to learning. Accordingly, this e-learning system is capable of boosting students' learning interests and improving their learning effectiveness.