Application of sensors in Augmented Reality based interactive learning environments

Widespread adoption of mobile augmented reality (AR) and virtual reality (VR) applications depends on their smoothness and immersiveness. Modern AR applications applying computationally intensive computer vision algorithms can burden today's mobile devices, and cause high energy consumption and/or poor performance. To tackle this challenge, it is possible to offload part of the computation to nearby devices at the edge. However, this calls for smart task placement strategies in order to efficiently use the resources of the edge infrastructure. In this paper, we introduce Nimbus — a task placement and offloading solution for a multi-tier, edge-cloud infrastructure where deep learning tasks are extracted from the AR application pipeline and offloaded to nearby GPU-powered edge devices. Our aim is to minimize the latency experienced by end-users and the energy costs on mobile devices. Our multifaceted evaluation, based on benchmarked performance of AR tasks, shows the efficacy of our solution. Overall, Nimbus reduces the task latency by <inline-formula><tex-math notation="LaTeX">$\sim 4\times$</tex-math></inline-formula> and the energy consumption by <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 77% for real-time object detection in AR applications. We also benchmark three variants of our offloading algorithm, disclosing the trade-off of centralized versus distributed execution.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Computational thinking (CT) skills are increasingly important in education to prepare students for the challenges of the digital age. Augmented Reality (AR) and Virtual Reality (VR) have been introduced as immersive technologies that have the potential to enhance CT skills through more interactive learning experiences. However, there is still a gap in understanding the effectiveness of these technologies in supporting the development of CT, particularly in different levels of education and disciplines. Although several studies have highlighted the benefits of AR and VR in education, no systematic review integrates these findings to identify advantages, challenges, and opportunities for further implementation. Therefore, this study conducted a systematic review based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines by analyzing 25 empirical studies (AR=17, VR=8) obtained from the Scopus database (2008-2024). The analysis addresses four key research questions: (1) the current state of AR/VR in CT development, (2) their advantages, (3) implementation challenges, and (4) future research directions. The results show that AR is more widespread than VR at various levels of education, with dominance in higher education followed by secondary and primary schools. Computer science is the main field of application of AR and VR, while AR is also widely applied in mathematics to increase interest and problem-solving. A total of 11 studies reported significant impacts of these technologies on CT, with AR being superior in increasing student motivation and engagement, as well as aiding in problem-solving and debugging. In contrast, VR provides a more immersive learning experience by strengthening concept understanding, especially in programming and recursion. However, several obstacles in the application of AR and VR, such as hardware limitations, costs, and user skills, affect the effectiveness of these technologies in the learning environment. This study also identified potential future research, including the exploration of VR in primary and kindergarten education, the application of VR in non-computer science fields, and the efficient use of these technologies in supporting the CT process. This study provides more precise insights into the optimal ways of utilizing AR and VR in developing CT skills. It is a reference for educators, policymakers, and researchers in supporting CT learning.

Practical exercises are a crucial part of many curricula. Even simple exercises can improve the understanding of the underlying subject. Most experimental setups require special hardware. To carry out e. g. a lens experiments the students need access to an optical bench, various lenses, light sources, apertures and a screen. In our previous publication we demonstrated the use of augmented reality visualization techniques in order to let the students prepare with a simulated experimental setup. Within the context of our intended blended learning concept we want to utilize augmented or virtual reality techniques for stationary laboratory exercises. Unlike applications running on mobile devices, stationary setups can be extended more easily with additional interfaces and thus allow for more complex interactions and simulations in virtual reality (VR) and augmented reality (AR). The most significant difference is the possibility to allow interactions beyond touching a screen. The LEAP Motion controller is a small inexpensive device that allows for the tracking of the user’s hands and fingers in three dimensions. It is conceivable to allow the user to interact with the simulation’s virtual elements by the user’s very hand position, movement and gesture. In this paper we evaluate possible applications of the LEAP Motion controller for simulated experiments in augmented and virtual reality. We pay particular attention to the devices strengths and weaknesses and want to point out useful and less useful application scenarios. © (2016) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.

Scalable Cloudlet-based Mobile Computing Model

The finite and rather small battery energy capacity in today's mobile devices has limited the functionality that can be integrated into these platforms or the performance and quality of applications that can be delivered to the users. In the last few years, there is a trend toward offloading certain computation-intensive and latency-tolerant local applications and service requests to a mobile cloud computing (MCC) system so as to save the precious battery life while providing the services requested by the users. Each mobile application can be thought of as a sequence of tasks that are executed locally or remotely. In this paper, the problem of optimal task dispatch, transmission, and execution onto the MCC system is considered. To achieve a good balance between the application execution time and power consumption, dynamic voltage and frequency scaling (DVFS) is applied to the local processor in the mobile device, while the transmitter can choose among multiple modulation schemes and bit rates. The rate capacity effect of a battery and power conversion losses in the mobile device are also accounted for so as to have a more realistic model of the remaining battery life. The mobile device is modeled as a semi-Markov decision process (SMDP) and the optimization problem to set the DVFS level and the transmission rate is effectively solved by linear programming combined with a one-dimensional heuristic search. Experimental results show that the proposed algorithm consistently outperforms some baseline algorithms.

In the course of digitalization, new technologies and innovations are continuously introduced to the educational sector. For instance, augmented reality (AR) is increasingly applied in science teaching in both school and higher education. Combining real and virtual content potentially enhances interactivity and understanding of the learning process. This teaching and learning approach can positively impact various learning outcomes, such as learning gains and motivation. This paper aims to investigate the positive learning effects of AR using a game-based AR learning environment: “Beat the Beast”. In line with the concept of an education for sustainable development (ESD) topic, microplastics, this learning environment follows an interdisciplinary approach, combining the subjects of biology, chemistry, and engineering. To determine and distinguish the effects of implementing the technology AR in science learning environments as well as the principles of game-based learning, we contrasted the learning environment into two factors: one with AR and one without AR and the other with a game and without a game. A quasi-experimental design with 203 pre-service teachers of the first semester of all subjects was chosen to evaluate the four different types of settings with questionnaires on motivation, technology acceptance, user engagement, cognitive load, computer self-efficacy, knowledge, and ESD outcomes in higher education. Our research demonstrates that although augmented reality (AR) imposes a relatively elevated cognitive load, it does not negatively affect learning effects. In spite of the increased cognitive load, learners in AR settings do not exhibit lesser knowledge acquisition compared to those in alternative environments. Moreover, our investigation highlights AR’s potential to amplify motivation and user engagement. Contrary to expectations, in the context of the selected subject matter and target audience, game-based adaptations of the educational environment fail to enhance learning outcomes. These versions actually underperform compared to other formats in both motivational and engagement metrics.

PurposeIn recent years, there has been significant interest in adopting XR (extended reality) technologies such as VR (virtual reality) and AR (augmented reality), particularly in retail. However, extending activities through reality-mediation is still mostly believed to offer an inferior experience due to their shortcomings in usability, wearability, graphical fidelity, etc. This study aims to address the research gap by experimentally examining the acceptance of metaverse shopping.Design/methodology/approachThis study conducts a 2 (VR: with vs. without) × 2 (AR: with vs. without) between-subjects laboratory experiment involving 157 participants in simulated daily shopping environments. This study builds a physical brick-and-mortar store at the campus and stocked it with approximately 600 products with accompanying product information and pricing. The XR devices and a 3D laser scanner were used in constructing the three XR shopping conditions.FindingsResults indicate that XR can offer an experience comparable to, or even surpassing, traditional shopping in terms of its instrumental and hedonic aspects, regardless of a slightly reduced perception of usability. AR negatively affected perceived ease of use, while VR significantly increased perceived enjoyment. It is surprising that the lower perceived ease of use appeared to be disconnected from the attitude toward metaverse shopping.Originality/valueThis study provides important experimental evidence on the acceptance of XR shopping, and the finding that low perceived ease of use may not always be detrimental adds to the theory of technology adoption as a whole. Additionally, it provides an important reference point for future randomized controlled studies exploring the effects of technology on adoption.

Educational technology can enhance learning by supporting the learning environment through various digital resources. There have been numerous emerging technologies which are able to bridge the resource gap in learning environments enabling students to get access to an abundance of resources on digital platforms. This paper presents a literature review, exploring the potential of using Mobile Augmented Reality (AR) and Virtual Reality (VR) technologies to support experiential learning in South African institutions. While there have been studies which aim to assess the use of AR and VR for educational purposes such as in mining safety education in South Africa, there is a need for studies that look at the potential of AR and VR in augmenting higher educational institutions such as universities and Technical and Vocational Education and Training (TVET) colleges which require students to complete an experiential learning component in their studies in order to complete their qualifications. The study aims to establish the potential role that AR and VR can provide in enhancing experiential learning by providing students with practical experience in various educational fields, leveraging augmented and virtual reality technologies to simulate such learning environments.

Much more than a telephone, today's mobile device has become an integral part of the way we interface with the world. Mobile devices have the computing capability, display, and battery power to become powerful medical devices that measure vital signs and provide intelligent interpretation or immediate transmission of information. The widespread adoption of mobile devices, even in low-resource settings, promises to make vital signs monitoring available anywhere and at low cost. This readily available computing power will also extend the utility of vital signs monitoring to new clinical indications, especially with the use of additional processing and integration of information. This review will focus on the universal promotion of pulse oximetry and advanced processing of plethysmography to assess variables such as respiratory rate, capillary refill time, and fluid responsiveness, and how these measurements may assist with perioperative monitoring, diagnosis, and management of pneumonia in children and preeclampsia in pregnancy when combined with mobile devices.

IntroductionThe introduction of augmented reality (AR) in medical education has been demonstrated to improve learning of medical students and young clinicians. Use of AR is often linked to Head Mounted Displays (HMD), whose high costs and expertise demands make them less widely applicable.MethodsThe open-source application Medical Imaging XR (MIXR), developed by Medicalholodeck™, was used to visualize computed tomography examinations in augmented reality on mobile devices, including smartphones and tablets. Cases were presented during Radiology lectures. Clinical cases relevant to each lecture topic—thorax, abdomen, gynecology, musculoskeletal system, and emergency medicine—were selected from our local picture archiving system. The cases were demonstrated on mobile devices during a radiological lecture, either in person or via video conference. To evaluate the educational experience, students provided feedback through a structured questionnaire.ResultsMedical students received a demonstration with AR in the lecture and were asked to answer a questionnaire. 47 students replied to the questionnaire. Students rated their experience with MIXR in a positive manner. 36.2% of students considered AR images to be very comprehensible, whereas only 21.3% considered conventional images as very comprehensible. 32 out of 47 considered that AR demonstration contribute to better understanding of the case as whole. 91.8% of students would like to continue using AR in the studies and 80.9% would like to have similar tools available for future interactions with patients.Conclusion/DiscussionIntroduction of AR in radiology lectures is well received by medical students, who would be interested in continuing using AR tools in their studies and future career. Using mobile device-based AR proved viable, contributing to democratization of AR, in a low-cost manner and with no need for extended expertise.

In this work, we describe an educational approach based on augmented reality (AR) and mobile technology that has been developed with the aim to engage students in a learning procedure regarding the electric forces as well as Coulomb’s Law. Initially, we present the AR learning environment and we explain its design principles with respect to the educational goals we aim to accomplish. The idea behind the present approach has been to present students with a novel learning environment that aligns well with their familiarization with mobile technology and the pleasure they take in making use of this technology, in order to stimulate their intrinsic motivation and learning performance. In addition, we demonstrate the early results of a classroom study performed in order to estimate students’ reaction to the AR approach and get an indication of their motivation levels and ease of use, with respect to the AR tool. Based on the study results, we conclude that students have demonstrated positive attitudes toward the AR learning approach and we claim that mobile AR may be incorporated for educational purposes in physics curricula.

1 IntroductionNowadays, the institutes already use in the process of the advantages of mobile technology, such as portability (the students have easy access at any time to information), the use of digital assets (audio and video files) and huge amount of textual information. This study proposes a new system approach for improving the process by Mobile Augmented Reality. The features of this technology stimulate the environment and consider that the student's motivation is the most important factor in learning.The existing literature [1] highlights two different approaches for augmented reality (AR). Firstly, the authors consider that AR is a system which has the following characteristics: Combines real and virtual; Interactive in real time; Registered in 3-D. Secondly [1] considers Augmented Reality part of Virtual Continuum. This context is composed of two elements: Real Environment - which contains all representations from real life and Virtual Environment - which combines different elements from virtual life. ARs are considered closer to reality than virtual environment because this type of systems are supposed to extend the real objects with virtual elements.According to [2] Augmented Reality is a technology that extends the real world elements with 2D or 3D computer generated objects and lets the users interact with them. Also, the authors present three types of devices which can be used to view the result of AR technology: a Head Mounted Display (HMD), a mobile device - with at least one back camera and a desktop or a laptop computer.Similarly, [2] considers mobile technology as an alternative teaching resource because the users have easy access in real time to information. Also, mobile devices in the process two main problems of traditional are solved: the location and time. Likewise, [1] considers that mobile is a type of eLearning without restrictions of time and space and this technology improves the quality of classical environments, but does not replace them.Other researches, such as [3] and [4] highlight the benefits of a Mobile Augmented Reality system based on tracking objects. In [3] is proposed a mobile augmented reality system based on visual recognition. According to the authors the visual recognition consists into tracking objects identified in real world with the camera of the mobile device and augmenting the scene with virtual objects created by MAR technology. Similarly, [4] presents a Mobile Augmented Reality system bases on 3D registration technology. According to the authors the 3D registration represents the tracking of the position and pose of the mobile device in the real time and real life and finally use these information to insert in the real world the virtual scene created by MAR. The 3D registration technology is based on two type of approaches: markers and natural features. Recently, the 3D registration for MAR uses GPS and sensors from mobile devices to provide different information about position and pose of the camera and combine them with computer vision to improve the speed and accuracy of the system.Other studies, such as [5] propose a makerbased Mobile Augmented Reality solution. The application is used by a Vocational Educational and Training (VET) program for supporting the maintenance process of a car. The authors define VET as education programs that are designed for learners to acquire the knowledge, skills and competencies specific to a particular occupation, trade or class of occupations or trades(p. 49-50). Moreover, in [5] is presented an AR application used by VET students for basic mathematics skills, which is based on videogames.More studies, such as [6] present a Mobile Augmented Reality solution for French language which is based on gamification didactic method. The author, Bernadette Perry, describes gamification as using gamebased mechanics, aesthetics and game thinking to engage people, motivate action, and promote learning (p. …

Inertial Measurement Units (IMUs) with gyroscopic sensors are standard in today's mobile devices. We show that these sensors can be co-opted for vibroacoustic data reception. Our approach, called VibroComm, requires direct physical contact to a transmitting (i.e., vibrating) surface. This makes interactions targeted and explicit in nature, making it well suited for contexts with many targets or requiring and intent. It also offers an orthogonal dimension of physical security to wireless technologies like Blue-tooth and NFC. Using our implementation, we achieve a transfer rate over 2000 bits/sec with less than 5% packet loss – an order of magnitude faster than prior IMU-based approaches at a quarter of the loss rate, opening new, powerful and practical use cases that could be enabled on mobile devices with a simple software update.

Today, proactive location-based services (LBS) are gaining momentum as mobile devices became able to track the users' position in the background without notable battery drain. They are used in several application areas like location-based marketing or gaming to deliver notifications, e.g. coupons, proactively to the user once a defined geographical area, also known as geofence, is entered. Owning to various reasons like the limited preciseness of positioning techniques for outdoor environments or the strict energy constraints of today's mobile devices, proactive LBS are still facing reliability issues. Either the user is burdened with non-relevant notifications in case the proactive LBS wrongly assumed the user to be located within a geofence, or notifications are not received at all although the user visited the geofence. This paper investigates how the day-by-day configuration of proactive LBS with geofences may influence the reliability of the service as a whole. Based on the findings, it introduces a web service which quantifies how reliable a proactive LBS will most probably be with respect to a given geofence in order to enable even non-experts to properly set up a proactive LBS with appropriate geofences.