Learning Calculus with Augmented Reality and Virtual Environments

Three-dimensional sound's effectiveness in virtual reality (VR) environments has been widely studied. However, due to the big differences between VR and augmented reality (AR) systems in registration, calibration, perceptual difference of immersiveness, navigation, and localization, it is important to develop new approaches to seamlessly register virtual 3-D sound in AR environments and conduct studies on 3-D sound's effectiveness in AR context. In this paper, we design two experimental AR environments to study the effectiveness of 3-D sound both quantitatively and qualitatively. Two different tracking methods are applied to retrieve the 3-D position of virtual sound sources in each experiment. We examine the impacts of 3-D sound on improving depth perception and shortening task completion time. We also investigate its impacts on immersive and realistic perception, different spatial objects identification, and subjective feeling of human presence and collaboration. Our studies show that applying 3-D sound is an effective way to complement visual AR environments. It helps depth perception and task performance, and facilitates collaborations between users. Moreover, it enables a more realistic environment and more immersive feeling of being inside the AR environment by both visual and auditory means. In order to make full use of the intensity cues provided by 3-D sound, a process to scale the intensity difference of 3-D sound at different depths is designed to cater small AR environments. The user study results show that the scaled 3-D sound significantly increases the accuracy of depth judgments and shortens the searching task completion time. This method provides a necessary foundation for implementing 3-D sound in small AR environments. Our user study results also show that this process does not degrade the intuitiveness and realism of an augmented audio reality environment

Augmented reality sickness questionnaire (ARSQ): A refined questionnaire for augmented reality environment

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Augmented reality environments face significant challenges in achieving context and semantic awareness, which are relevant for enhancing user experiences. Although machine learning has been extensively applied to areas such as object and action recognition, there is a knowledge gap in the integration of these techniques to improve contextual understanding in AR environments. This study explores how machine learning enhances object and action recognition, improving context awareness in augmented reality environments. A systematic review of fifteen studies revealed advancements in deep learning, semantic web, knowledge graphs, and multimodal interfaces, which collectively improved the capabilities of augmented reality systems. The findings include innovations in aspects such as 3D hand pose estimation, semantic gaze analysis through eye tracking, and hybrid approaches that optimize real-time processing. These developments improve augmented and mixed reality environments by enabling more realistic, interactive, and responsive applications. However, persistent challenges such as latency, scalability, and environmental adaptation underscore the need for more research. These results suggest a guide for the development of highly immersive and efficient AR/MR systems, with the potential to improve user interaction and operational capabilities.

The purpose of the research is to compare the impact of students’ learning performance between an augmented reality (AR) environment and in-class environment. To create an efficient AR environment, we used Microsoft HoloLens, which is the next generation of a see-through holographic computer. We developed the AR learning module: ergonomic guidelines for manual material handling (MMH). We hypothesize that AR changes the way students understand the concepts of MMH. Our analysis includes a careful evaluation of student experimental skills during the learning activities. This new AR environment could allow students to engage hands-on training of MMH and strengthen their understanding. Student test score was used as metrics for performance assessment. We found a significant improvement on student understanding of MMH lecture after they used the AR module. The findings of this study indicated the potential benefits of using AR environment interfaces in engineering education and training.

Recupera algunos de los elementos menos estudiados y reconocidos en el ensayo latinoamericano: su profunda vocación de reflexión, diálogo y mediación intelectual; su plasticidad y apertura a la búsqueda del sentido; su vínculo con la sociabilidad, la conversación y la generación de un espacio simbólico de encuentro; su capa cidad de establecer una nueva retroalimentación entre lectura y escritura.

Virtual characters are now widely used in games, computer-generated (CG) movies, virtual reality (VR), and communication media. The continued technological innovations in motion capture mean that a more natural representation of a three-dimensional character’s motion should be achievable. Many researchers have investigated how virtual characters interact with their surrounding environment through spatial relationships, which were introduced for adapting and preserving character motion. However, technical problems should be resolved to enable the control of characters in augmented reality (AR) environments that combine with the real world, and this can be achieved by adapting motion to environmental differences using original motion datasets. In this paper, we investigate a novel method for preserving automatic motion adaptation for a virtual character in AR environments. We used specific object (e.g., puddle) recognition and the spatial properties of the user’s surrounding space, e.g., object types and positions, and ran validation experiments to provide accurate motion to improve the AR experience. Our experimental study showed positive results in terms of smooth motion in AR configurations. We also found that the participants using AR felt a greater sense of co-presence with the character through adapted motion.

Smart building is a rising interdisciplinary research field that aims to improve the monitoring, management, and maintenance of buildings. The authors present an innovative solution for combining building information modeling (BIM) data with ambient information collected by heterogeneous devices deployed in the building. To collect environmental information, they exploit a distributed software architecture. It enables interoperability between heterogeneous data sources--either physical devices, such as sensor nodes, or third-party software such as Archibus--where building information resides. On top of this infrastructure, they developed an Android-based application that presents environmental building information integrated with BIM data in an augmented and virtual reality environment. The proposed solution gives users awareness about building conditions and energy consumption.

The paper describes a technique that allows measuring and annotating real objects in an Augmented Reality (AR) environment. The technique is based on the marker tracking, and aims at enabling the user to define the three-dimensional position of points, within the AR scene, by selecting them directly on the video stream. The technique consists in projecting the points, which are directly selected on the monitor, on a virtual plane defined according to the bi-dimensional marker, which is used for the tracking. This plane can be seen as a virtual depth cue that helps the user to place these points in the desired position. The user can also move this virtual plane to place points within the whole 3D scene. By using this technique, the user can place virtual points around a real object with the aim of taking some measurements of the object, by calculating the minimum distance between the points, or in order to put some annotations on the object. Up to date, these kinds of activities can be carried out by using more complex systems or it is needed to know the shape of the real object a priori. The paper describes the functioning principles of the proposed technique and discusses the results of a testing session carried out with users to evaluate the overall precision and accuracy.

This paper presents an intuitive and natural gesture-based methodology for solid modelling in the Augmented Reality (AR) environment. The framework of Client/Server (C/S) is adopted to design the AR-based computer aided design (CAD) system. The method of creating random or constraints- based points using gesture recognition is developed to support modelling. In addition, a prototype system of product 3D solid modelling has been successfully developed, we have compared it with traditional CAD systems through several basic design modeling tasks. Finally, analysis of questionnaire feedback survey shows the intuitiveness and effectiveness of the system, and user studies demonstrate the advantage of helping accomplish the product early design and creating and manipulating the 3D model in the AR environment.

Objective. Brain-controlled robotic arms have shown broad application prospects with the development of robotics, science and information decoding. However, disadvantages, such as poor flexibility restrict its wide application. Approach. In order to alleviate these drawbacks, this study proposed a robotic arm asynchronous control system based on steady-state visual evoked potential (SSVEP) in an augmented reality (AR) environment. In the AR environment, the participants were able to concurrently see the robot arm and visual stimulation interface through the AR device. Therefore, there was no need to switch attention frequently between the visual stimulation interface and the robotic arm. This study proposed a multi-template algorithm based on canonical correlation analysis and task-related component analysis to identify 12 targets. An optimization strategy based on dynamic window was adopted to adjust the duration of visual stimulation adaptively. Main results. Experimental results of this study found that the high-frequency SSVEP-based brain–computer interface (BCI) realized the switch of the system state, which controlled the robotic arm asynchronously. The average accuracy of the offline experiment was 94.97, whereas the average information translate rate was 67.37 ± 14.27 bits·min−1. The online results from ten healthy subjects showed that the average selection time of a single online command was 2.04 s, which effectively reduced the visual fatigue of the subjects. Each subject could quickly complete the puzzle task. Significance. The experimental results demonstrated the feasibility and potential of this human-computer interaction strategy and provided new ideas for BCI-controlled robots.

Recent work in human-machine interaction and audiology has indicated the effectiveness of using spatialized bone conduction (BC) hearing in augmented reality (AR) environments. For broader applications, individualization methods are preferred, as they require simpler computation and measurement, despite the fact that spatial hearing for BC is not yet well investigated at individual levels. Accordingly, this study examines different individualization techniques for spatial audio using BC applied to a navigation task with a moving virtual sound object in external noise, which simulates a common situation in AR environments. The tested methods include one based on individually measured head-related transfer functions (HRTFs), generic HRTFs, and a modified version based on these. Subjects are instructed to follow a virtual sound source presented from the BC headphones while walking. The virtual sound source is updated so that the source is located ahead of the subject according to the subject’s location and head rotation tracked by the motion capture system. The subjects’ performance are evaluated as the error of the subjects’ actual path from the simulated pathway. Results are discussed on the comparison between individualization methods and in regards to recent reports on sound localization in BC hearing without the listener’s body movement.

ABSTRACTPainting is a complex and creative process that involves the use of various drawing skills to create artworks. The concept of training artificial intelligence models to imitate this process is referred to as neural painting. To enable ordinary people to engage in the process of painting, we propose PainterAR, a novel interface that renders any paintings stroke‐by‐stroke in an immersive and realistic augmented reality (AR) environment. PainterAR is composed of two components: the neural painting model and the AR interface. Regarding the neural painting model, unlike previous models, we introduce the Kullback–Leibler divergence to replace the original Wasserstein distance existed in the baseline paint transformer model, which solves an important problem of encountering different scales of strokes (big or small) during painting. We then design an interactive AR interface, which allows users to upload an image and display the creation process of the neural painting model on the virtual drawing board. Experiments demonstrate that the paintings generated by our improved neural painting model are more realistic and vivid than previous neural painting models. The user study demonstrates that users prefer to control the painting process interactively in our AR environment.

This study describes the implementation of Problem-Based Learning (PBL) using Ethno-STEM-based e-modules, assisted by Augmented Reality (AR), in the Integral Calculus course, and examines students' responses to the learning process and the development of critical thinking skills. The study employed a descriptive qualitative method with 37 Mathematics Education as subjects. The learning was designed by integrating local cultural contexts into AR-based STEM activities. Data were collected through critical thinking tests and student response questionnaires. The results showed that students' critical thinking skills were in the fair to good category, especially in the aspects of focus identification and reasoning, while the inference and reflection aspects still needed improvement. Students' responses to the learning process were very positive, especially in motivation, collaboration, and learning experiences. The integration of PBL, Ethno-STEM, and AR created contextual and meaningful learning. Keywords: Problem-Based Learning, Ethno-STEM, Augmented Reality

An increasingly large amount of data from the physical world has been collected, digitized, and stored. To communicate such data to general public effectively, visual data-driven storytelling has been widely used. Yet, the vast majority of data communication occurs on desktop computers separated from the physical world the data originates in and refers to. Recent advances in Augmented Reality (AR) have shed new light on data-driven storytelling, offering exciting possibilities for telling engaging, in-situ, and immersive stories by embedding the data in the real-world context. However, although creating such kind of AR data stories is demanding and requires considerable knowledge and skills from different fields (e.g., data visualization, computer graphics, computer vision, and human-machine interaction), prior research has rarely investigated the effective way to create visual data-driven stories in AR environments. This thesis aims to fill this gap by exploring the approaches to facilitate the authoring of infographics, a popular format for data-driven storytelling, in AR. Given that AR devices are still evolving rapidly, this thesis focuses on the interaction between reality and virtuality, an essence of AR that is independent of specific devices. The first system, MARVisT, leverages physical properties (e.g., size, positions) from real-world objects to assist non-experts in creating 3D infographics in mobile AR. Given the limited interaction capabilities of mobile devices, in the second work, LassoNet is proposed to facilitate the selection of 3D objects in a 2D screen based on a deep neural network. Besides augmenting the physical world, the third research studies augmenting the semantic content of real-world infographics in AR and introduces PapARVis Designer to allow designers to create augmented static visualizations. Finally, the fourth work explores automating the creation of timeline infographics in AR by developing a deep-learning based method that automatically extracts extensible templates from real timeline infographics to generate virtual timelines with new data in AR. The core idea of this thesis is to allow visualization designers to go beyond the desktop platform to the engaging, immersive, and promising AR platform that is seen to be the next-generation human-machine interaction platform. The resulting systems and techniques blaze a trail toward futuristic visual storytelling.