Authentication Framework for Augmented Reality with Data-Hiding Technique

Hand-tracking technologies allow us to use our own hands to interact with real and virtual objects in Augmented Reality (AR) environments. This enables us to explore the interplay between hand-visualizations and hand-object interactions. We present a user study that examines the effect of different hand visualizations (invisible, transparent, opaque) on manipulation performance when interacting with real and virtual objects. For this, we implemented video-see-through (VST) AR-based virtual building blocks and hot wire tasks with real one-to-one counterparts that require participants to use gross and fine motor hand movements. To evaluate manipulation performance, we considered three measures: task completion time, number of collisions (hot wire task), and percentage of object displacement (building block task). Additionally, we explored the sense of agency and subjective impressions (preference, ease of interaction, successful and awkwardness) evoked by the different hand-visualizations. The results show that (1) manipulation performance is significantly higher when interacting with real objects compared to virtual ones, (2) invisible hands lead to fewer errors, higher agency, higher perceived success and ease of interaction during fine manipulation tasks with real objects, and (3) having some visualization of the virtual hands (transparent or opaque) overlayed on the real hands is preferred when manipulating virtual objects even when there are no significant performance improvements. Our empirical findings about the differences when interacting with real and virtual objects can aid hand visualization choices for manipulation tasks in AR.

In civil circulation, due to its digitalization, virtual objects that, unlike a real object of the surrounding world in its usual sense, have a digital form wihout a real shell, are widespread. The civil law regime of virtual objects is not defined in the current legislation. Meanwhile, in the context of the development of metaverse concepts, this problem and the search for ways to solve it is becoming increasingly urgent. Determining the legal regime of objects created by the user of the platform on which access to virtual reality is provided, augmented reality objects, as well as objects existing and created within the metaverse, where the user’s usual activities can be carried out, is ambiguous in scholarship and jurisprudence. This study was undertaken to determine the features of the civil law regime of objects created in virtual reality, augmented reality, and in the metaverse. It was concluded that the regimes of familiar civil law objects (property, results of intellectual activity, intangible benefits, services) can be extended to the objects under consideration. However, the material legal concept does not fully reflect the legal nature of the objects under study. Therefore, virtual objects that copy things of the physical world, in essence, can be recognized as other property rights. At the same time, the intellectual property rules can be applied to these objects only if they are created as a result of a person’s creative efforts.

In recent years, augmented reality (AR) technology has received attention. AR technology is expected to be widely used in various day-to-day activities. Many researchers have proposed methods that use AR technology for various purposes. However, there is a possibility that displayed AR objects cause anxiety or spoil scenic views for the user, as the objects can hinder the view and the appearances can change to several forms. Therefore, it is important to consider how we should display the AR objects depending on its purpose. In this study, we have investigated how participants behave when they are shown different AR objects to lead them to the left/right side on an outdoor road. As a result, we confirmed that we should use the AR object whose appearance blends with the surroundings when the object is a sign to lead people on outdoor.

Augmented Reality (AR) that can be considered as a form of location-based services is a technique utilizing position and orientation information and a location database of POIs to enrich video or photographs by adding on graphics or text objects. The AR technique can be used to develop Pedestrian Navigation Systems (PNSs), but high-precision positioning is needed for displaying AR objects at proper places. However, the required level of accuracy is far beyond current commercial positioning solutions, e.g., outdoor GPS and indoor RF systems. In this paper, we propose an AR-based positioning technique for AR users to position their locations. First, by utilizing a coarse positioning system, AR objects can be roughly displayed on the touch screen of a AR device. Then, AR objects can be matched with their images on the display via drag-and-drop operations by users. Thus, both the coordinates of the AR objects in the image and in the real world can be known. Based on the coordinates along with the knowledge of the camera focal length, the location at which the photograph was taken can be known. The location of the camera can be treated as the location of the user. The proposed positioning technique is very helpful in developing high-precision PNSs.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

We propose a novel wearable haptic device that can provide kinesthetic haptic feedback for stiffness rendering of virtual objects in augmented reality (AR). Rendering stiffness of objects using haptic feedback is crucial for realistic finger-based object manipulation, yet challenging particularly in AR due to the co-presence of a real hand, haptic device, and rendered AR objects in the scenes. By adopting passive actuation with a tendon-based transmission mechanism, the proposed haptic device can generate kinesthetic feedback strong enough for immersive manipulation and prevention of inter-penetration in a small-form-factor, while maximizing the wearability and minimizing the occlusion in AR usage. A selective locking module is adopted in the device to allow for the rendering of the elasticity of objects. We perform an experimental study of two-finger grasping to verify the efficacy of the proposed haptic device for finger-based manipulation in AR. We also quantitatively compare/articulate the effects of different types of feedbacks across haptic and visual sense (i.e., kinesthetic haptic feedback, vibrotactile haptic feedback, and visuo-haptic feedback) for stiffness rendering of virtual objects in AR for the first time.

Locating virtual objects (e.g., holograms) in head-mounted Augmented Reality (AR) can be an exhausting and frustrating task. This is mostly due to the limited field of view of current AR devices, which amplify the problem of objects receding from view. In previous work, EyeSee360 was developed to address this problem by visualizing the locations of multiple out-of-view objects. However, on small field of view devices such as the Hololens, EyeSee360 adds a lot of visual clutter that may negatively affect user performance. In this work, we compare three variants of EyeSee360 with different levels of information (assistance) to evaluate in how far they add visual clutter and thereby negatively affect search time performance. Our results show that variants of EyeSee360 with less assistance result into faster search times.

Consistent illumination of virtual and real objects in augmented reality (AR) is essential to achieve visual coherence. This paper presents a practical method for rendering with consistent illumination in AR in two steps. In the first step, a user scans the surrounding environment by rotational motion of the mobile device and the real illumination is captured. We capture the real light in high dynamic range (HDR) to preserve its high contrast. In the second step, the captured environment map is used to precalculate a set of reflection maps on the mobile GPU which are then used for real-time rendering with consistent illumination. Our method achieves high quality of the reflection maps because the convolution of the environment map by the BRDF is calculated accurately per each pixel of the output map. Moreover, we utilize multiple render targets to calculate reflection maps for multiple materials simultaneously. The presented method for consistent illumination in AR is beneficial for increasing visual coherence between virtual and real objects. Additionally, it is highly practical for mobile AR as it uses only a commodity mobile device.KeywordsAugmented RealityVirtual ObjectHigh Dynamic RangeLight ProbeBidirectional Reflectance Distribution FunctionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This study proposes a novel technology as a creative method to interact with an augmented reality (AR) system called PhoneCursor, which is incorporated onto head-mounted displays. PhoneCursor is designed to offer more intuitive, portable and natural interaction. Its technical realization combines the gyroscope and acceleration sensor based on mobile phone. In view of the target selection ability as a base function of PhoneCursor, it also enhances the performance in solving some difficult selection problems in AR, including far and small object selection, 3D occlusion, dense object selection, and batch selection problem. An experiment based on the ISO 9241-9 standard is conducted to investigate this ability, wherein the PhoneCursor technique is compared with the widely used and well-performing head-movement-based selection technique. Results show that PhoneCursor performs better in terms of movement time and throughput than the conventional technique. Some applications are further proposed to showcase the potential of using the PhoneCursor technique in AR scenarios.

We present several techniques for producing two visual and modeling effects in augmented reality. The first effect involves interactively calculating the occlusions between real and virtual objects. The second effect utilizes a collision detection algorithm to automatically move dynamic virtual objects until they come in contact with static real objects in augmented reality. All of the techniques utilize calibrated data derived from images of a real‐world environment.

This paper presents a markerless Augmented Reality (AR) framework which utilizes outstretched hand for registration of virtual objects in the real environment. Hand detection methods used in this work are purely based on computer vision algorithms to detect bare hand without assistance from markers or any other devices such as mechanical devices and magnetic devices. We use a stereo camera to capture video images, so that the depth information of the hand can be constructed. Skin color segmentation is employed to segment the hand region from the images. Instead of fiducial markers, the center of the palm and fingertips are tracked in real time. Incorporating the depth information computed, the 3D positions of these hand features are then used to estimate the 6DOF camera pose with respect to the hand, which in turn allows the virtual objects to be augmented onto the palm accurately. This method increases the ease of manipulation of AR objects. Users can inspect and manipulate the AR objects in an intuitive way by using their bare hands.

La Realidad Aumentada (RA) se configura como una tecnología de gran potencial en la formación universitaria, ofreciendo la posibilidad de combinar la información digital con la física en tiempo real a través de diferentes dispositivos tecnológicos (Tablet, smartphones, gafas,…). Nuestro estudio, perseguía un doble objetivo, por una parte conocer el grado de motivación que la utilización de objetos en formato RA despierta en los alumnos de medicina y por otra analizar si el género de los alumnos afecta positivamente en el grado de motivación mostrada. Para el análisis de la motivación despertada en los alumnos por la experiencia realizada con los objetos de RA elaborados se utilizó un cuestionario elaborado por Keller (2010), el “Instructional Material Motivational Survey” (IMMS). La muestra del estudio estuvo formada por alumnos de primer curso del Grado de Medicina de la Universidad de Sevilla (España), que cursaban la asignatura de “Anatomía Humana I”. Podemos destacar que la experiencia realizada llamó la atención de los alumnos, y que el material les despertó la curiosidad ayudándoles a mantener la atención y disfrutando por lo general con la experiencia realizada a la vez que el contenido del material presentado era relevante para sus intereses. Del estudio podemos sacar las siguientes conclusiones: El instrumento de diagnóstico se presenta como un buen predictor para explicar la motivación, atención, confianza, relevancia y satisfacción que despierta la interacción con los objetos de RA. La RA se presenta como una tecnología de gran utilidad para el aprendizaje de los alumnos de medicina, su incorporación a la enseñanza es fácil y el género de los estudiantes no determina el grado de aceptación que tienen de la RA.

BACKGROUND: The tenets of minimally invasive neurosurgery require small craniotomies that nevertheless afford sufficient surgical freedom to safely accomplish the operative goal. OBJECTIVE: To present a novel technique which couples surgical rehearsal in virtual reality (VR) with navigation-integrated, microscope-synchronized augmented reality (AR) to optimize minimally invasive openings. METHODS: A retrospective review of the senior author's early experience with this mixed reality (VR + AR) platform was conducted. Patient-specific VR models were rendered from standard computed tomography/MRI. Surgical rehearsal with these models created templates with the optimal opening, which were integrated with the AR system. In surgery, the templates were projected through a navigation-tracked microscope and appeared as AR objects onto the patient. The templates guided the planning of the incision and steps of the approach. RESULTS: Sixteen patients were included in this study, 11 women and 5 men (33-79 years; median: 59.5). Twelve operations were performed for mass lesions and 4 for microvascular decompression for trigeminal neuralgia. There were no intraoperative adverse events related to AR. For 15 patients (94%), the templates provided major advantage in optimizing the bony opening and guided the operation to the specific anatomic location chosen preoperatively. For 1 patient, the AR template guided the drilling away from the critical structures inside the temporal bone. In addition, 3-dimensional AR markers relieve the surgeon from ergonomically cumbersome monitor-based navigation (Video 1). CONCLUSION: This novel mixed reality platform combines surgical rehearsal in VR and navigation-integrated, microscope-synchronized overlay of AR templates to ensure that optimal exposure is efficiently obtained.

We conducted a fundamental user study to assess potential benefits of AR technology for immersive vocabulary learning. With the idea that AR systems will soon be able to label real-world objects in any language in real time, our within-subjects lab-based study explores the effect of such an AR vocabulary prompter on participants learning nouns in an unfamiliar foreign language, compared to a traditional flashcard-based learning approach. Our results show that the immersive AR experience of learning with virtual labels on real-world objects is both more effective and more enjoyable for the majority of participants, compared to flashcards. Specifically, when participants learned through augmented reality, they scored significantly better on both same-day and 4-day delayed productive recall tests than when they learned using the flashcard method. We believe this result is an indication of the strong potential for language learning in augmented reality, particularly because of the improvement shown in sustained recall compared to the traditional approach.

With the growing appeal of real-time interactions between physical views and virtual objects in augmented reality (AR) applications among contemporary users, optimizing power consumption is crucial for extending the battery life of mobile devices. This paper delves into methods for reducing power consumption specifically in mobile OLED devices when running AR applications, depending on user behaviors. Initially, we introduce a detection algorithm designed to accurately identify user status using cost-effective sensors. Subsequently, we present two dynamic configurations aimed at adjusting the display of physical views and virtual objects based on user visual attention of AR applications. The results of extensive experiments conducted on a commercial smartphone using an open-source AR application to assess the performance of the proposed power-saving design are highly promising.