Multi-User 3D Augmented Reality Anwendung für die gemeinsame Interaktion mit virtuellen 3D Objekten

Augmented Reality (AR) is the process of integrating virtual elements in reality, often by mixing computer graphics into a live video stream of a real scene. It requires registration of the target object with respect to the cameras. To this end, some approaches rely on dedicated hardware, such as magnetic trackers or infra-red cameras, but they are too expensive and cumbersome to reach a large public. Others are based on specifically designed markers which usually look like bar-codes. However, they alter the look of objects to be augmented, thereby hindering their use in application for which visual design matters. Recent advances in Computer Vision have made it possible to track and detect objects by relying on natural features. However, no such method is commonly used in the AR community, because the maturity of available packages is not sufficient yet. As far as deformable surfaces are concerned, the choice is even more limited, mainly because initialization is so difficult. Our main contribution is therefore a new AR framework that can properly augment deforming surfaces in real-time. Its target platform is a standard PC and a single webcam. It does not require any complex calibration procedure, making it perfectly suitable for novice end-users. To satisfy to the most demanding application designers, our framework does not require any scene engineering, renders virtual objects illuminated by real light, and let real elements occlude virtual ones. To meet this challenge, we developed several innovative techniques. Our approach to real-time registration of a deforming surface is based on wide-baseline feature matching. However, traditional outlier elimination techniques such as RANSAC are unable to handle the non-rigid surface's large number of degrees of freedom. We therefore proposed a new robust estimation scheme that allows both 2–D and 3–D non-rigid surface registration. Another issue of critical importance in AR to achieve realism is illumination handling, for which existing techniques often require setup procedures or devices such as reflective spheres. By contrast, our framework includes methods to estimate illumination for rendering purposes without sacrificing ease of use. Finally, several existing approaches to handling occlusions in AR rely on multiple cameras or can only deal with occluding objects modeled beforehand. Our requires only one camera and models occluding objects at runtime. We incorporated these components in a consistent and flexible framework. We used it to augment many different objects such as a deforming T-shirt or a sheet of paper, under challenging conditions, in real-time, and with correct handling of illumination and occlusions. We also used our non-rigid surface registration technique to measure the shape of deformed sails. We validated the ease of deployment of our framework by distributing a software package and letting an artist use it to create two AR applications.

Utilizing augmented reality for applications in medicine has been a topic of intense research for several years. A number of challenging tasks need to be addressed when designing a medical AR system. These include the import and management of medical datasets and preoperatively created planning data, the registration of the patient with respect to a global coordinate system, and accurate tracking of the camera used in the AR setup as well as the respective surgical instruments. Most research systems rely on specialized hardware or algorithms for realizing augmented reality in medicine. Such base technologies can be expensive or very time-consuming to implement. In this paper, we propose an alternative approach of building a surgical AR system by harnessing existing, commercially available equipment for image guided surgery (IGS). We describe the prototype of an augmented reality application, which receives all necessary information from a device for intraoperative navigation.

BackgroundSmartphone and tablet apps that deliver health care education have been identified as effective in improving patient knowledge and treatment adherence in asthma populations. Despite asthma being the most common chronic disease in pediatrics, there are few apps that are targeted specifically for children. Only half of children with asthma have acceptable control of their symptoms, and 40%-98% do not use their inhalers correctly. With children being increasingly connected to technology, there is an opportunity to improve asthma inhaler technique education by delivery via smartphone or tablet apps. Augmented reality (AR) technology was used in this study to capitalize on growing technological innovations. Digital health interventions that use a co-design process for development have the highest likelihood of successful uptake and effectiveness on their intended outcomes. Perceived usability also has been shown to improve the effectiveness of education as well as the acceptance of the intervention.ObjectiveThe aims of this study were to describe the co-design process, development, and design outcomes of a smartphone or tablet app that incorporates AR technology to deliver asthma inhaler technique education to children with asthma. This study also aimed to provide a usability evaluation, using the System Usability Scale to inform our work and future research, and recommendations for others performing similar work.MethodsThe development of the AR asthma inhaler technique education app was based on an iterative co-design process with likely end users (children with asthma, their caregivers, and health care professionals). This involved multiple stages: recruitment of end users for qualitative interviews and usability testing with a previously designed educational intervention, which used an AR-embedded smartphone or tablet app; ideation of content for a specific asthma inhaler technique education intervention with end users; development of the specific asthma inhaler intervention; and 2 further rounds of interviews and usability testing with the redesign of the initial prototype.ResultsWe included 16 participants aged 9-45 years. Using the co-design process, the AR asthma inhaler technique education app was designed, incorporating the preferences of end users. After iteration 1, animation was included based on the feedback provided. Iteration 2 feedback resulted in increased AR experiences and the removal of the requirement of a paper-based resource to trigger AR in the third iteration. Throughout all rounds, the ease of use of the app and the novel nature of the intervention were frequently described. The usability of the intervention overall was perceived to be excellent, and the mean System Usability Scale score of the intervention was found to be highest in the final round of evaluation (90.14).ConclusionsThe results from this co-design process and usability evaluation will be used to develop a final AR asthma inhaler technique educational intervention, which will be evaluated in the clinical setting.International Registered Report Identifier (IRRID)RR2-10.1177/16094069211042229

Computer visualization can effectively deliver instructions to a user whose task requires understanding of a real world scene. Consider the example of surgical telementoring, where a general surgeon performs an emergency surgery under the guidance of a remote mentor. The mentor guidance includes annotations of the operating field, which conventionally are displayed to the surgeon on a nearby monitor. However, this conventional visualization of mentor guidance requires the surgeon to look back and forth between the monitor and the operating field, which can lead to cognitive load, delays, or even medical errors. Another example is 3D acquisition of a real-world scene, where an operator must acquire multiple images of the scene from specific viewpoints to ensure appropriate scene coverage and thus achieve quality 3D reconstruction. The conventional approach is for the operator to plan the acquisition locations using conventional visualization tools, and then to try to execute the plan from memory, or with the help of a static map. Such approaches lead to incomplete coverage during acquisition, resulting in an inaccurate reconstruction of the 3D scene which can only be addressed at the high and sometimes prohibitive cost of repeating acquisition.Augmented reality (AR) promises to overcome the limitations of conventional out-of-context visualization of real world scenes by delivering visual guidance directly into the user's field of view, guidance that remains in-context throughout the completion of the task. In this thesis, we propose and validate several AR visual interfaces that provide effective visual guidance for task completion in the context of surgical telementoring and 3D scene acquisition.A first AR interface provides a mentee surgeon with visual guidance from a remote mentor using a simulated transparent display. A computer tablet suspended above the patient captures the operating field with its on-board video camera, the live video is sent to the mentor who annotates it, and the annotations are sent back to the mentee where they are displayed on the tablet, integrating the mentor-created annotations directly into the mentee's view of the operating field. We show through user studies that surgical task performance improves when using the AR surgical telementoring interface compared to when using the conventional visualization of the annotated operating field on a nearby monitor. A second AR surgical telementoring interface provides the mentee surgeon with visual guidance through an AR head-mounted display (AR HMD). We validate this approach in user studies with medical professionals in the context of practice cricothyrotomy and lower-limb fasciotomy procedures, and show improved performance over conventional surgical guidance. A comparison between our simulated transparent display and our AR HMD surgical telementoring interfaces reveals that the HMD has the advantages of reduced workspace encumbrance and of correct depth perception of annotations, whereas the transparent display has the advantage of reduced surgeon head and neck encumbrance and of annotation visualization quality. A third AR interface provides operator guidance for effective image-based modeling and rendering of real-world scenes. During the modeling phase, the AR interface builds and dynamically updates a map of the scene that is displayed to the user through an AR HMD, which leads to the efficient acquisition of a five-degree-of-freedom image-based model of large, complex indoor environments. During rendering, the interface guides the user towards the highest-density parts of the image-based model which result in the highest output image quality. We show through a study that first-time users of our interface can acquire a quality image-based model of a 13m $\times$ 10m indoor environment in 7 minutes.A fourth AR interface provides operator guidance for effective capture of a 3D scene in the context of photogrammetric reconstruction. The interface relies on an AR HMD with a tracked hand-held camera rig to construct a sufficient set of six-degrees-of-freedom camera acquisition poses and then to steer the user to align the camera with the prescribed poses quickly and accurately. We show through a study that first-time users of our interface are significantly more likely to achieve complete 3D reconstructions compared to conventional freehand acquisition. We then investigated the design space of AR HMD interfaces for mid-air pose alignment with an added ergonomics concern, which resulted in five candidate interfaces that sample this design space. A user study identified the aspects of the AR interface design that influence the ergonomics during extended use, informing AR HMD interface design for the important task of mid-air pose alignment.

PurposeIn recent years, there has been significant interest in adopting XR (extended reality) technologies such as VR (virtual reality) and AR (augmented reality), particularly in retail. However, extending activities through reality-mediation is still mostly believed to offer an inferior experience due to their shortcomings in usability, wearability, graphical fidelity, etc. This study aims to address the research gap by experimentally examining the acceptance of metaverse shopping.Design/methodology/approachThis study conducts a 2 (VR: with vs. without) × 2 (AR: with vs. without) between-subjects laboratory experiment involving 157 participants in simulated daily shopping environments. This study builds a physical brick-and-mortar store at the campus and stocked it with approximately 600 products with accompanying product information and pricing. The XR devices and a 3D laser scanner were used in constructing the three XR shopping conditions.FindingsResults indicate that XR can offer an experience comparable to, or even surpassing, traditional shopping in terms of its instrumental and hedonic aspects, regardless of a slightly reduced perception of usability. AR negatively affected perceived ease of use, while VR significantly increased perceived enjoyment. It is surprising that the lower perceived ease of use appeared to be disconnected from the attitude toward metaverse shopping.Originality/valueThis study provides important experimental evidence on the acceptance of XR shopping, and the finding that low perceived ease of use may not always be detrimental adds to the theory of technology adoption as a whole. Additionally, it provides an important reference point for future randomized controlled studies exploring the effects of technology on adoption.

PENS is a vocational education campus that is actively promoting the community to capture the interests of prospective students to register as students. This requires promotion of media innovation that can attract public interest. Multimedia technology, especially augmented reality can be used as an attractive alternative media for PENS to carry out promotions. This study aims to develop promotional media in the form of an augmented reality technology-based virtual catalog to introduce various study programs available in PENS. Catalogs that are generally in the form of physical media are converted into markers that can bring up 3D objects and videos that explain the contents to the user so that the user does not need to read because there is already an audio visual explanatory facility. The trial results show that the PENS virtual catalog can function well in viewing AR mascot, AR video and information.

Augmented Reality overlays computer generated images over the real world. This requires precise knowledge of the viewing projection of the headmounted display (HMD) and its position. Most of the previously published methods are complicated or use special equipment for the calibration process. We present a collection of calibration methods usable for fast and easy calibration of camera parameters, object/camera to tracker transformations, and image rectification, which do not need additional instrumentation or complicated procedures. They are applicable for both see-through and videobased HMDs and have already been successfully implemented in the Studierstube collaborative augmented environment.

The technology of augmented reality (AR), as a new user interface, introduces a completely new perspective for the design of technical manufacturing systems. This technique supports a face to face collaboration where users need to be able to easily cooperate with each other. As with typical construction sets like LEGO or Fischertechnik, the planning engineers model the future manufacturing system in their real environment. The components are taken from virtual construction sets and are positioned interactively in the manufacturing hall. Planning rules are used to assist the user and to prevents possible errors. This article describes the conception of a virtual construction set and the realization of its prototype. The description of the development of this construction set is supplemented by an illustration of the used hardware and software components.

Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding

Freehand gesture is an essential input modality for modern Augmented Reality (AR) user experiences. However, developing AR applications with customized hand interactions remains a challenge for end-users. Therefore, we propose GesturAR, an end-to-end authoring tool that supports users to create in-situ freehand AR applications through embodied demonstration and visual programming. During authoring, users can intuitively demonstrate the customized gesture inputs while referring to the spatial and temporal context. Based on the taxonomy of gestures in AR, we proposed a hand interaction model which maps the gesture inputs to the reactions of the AR contents. Thus, users can author comprehensive freehand applications using trigger-action visual programming and instantly experience the results in AR. Further, we demonstrate multiple application scenarios enabled by GesturAR, such as interactive virtual objects, robots, and avatars, room-level interactive AR spaces, embodied AR presentations, etc. Finally, we evaluate the performance and usability of GesturAR through a user study.

Grounded in the Technology Acceptance Model, Flow Theory, and Diffusion of Innovation, this paper explores the psychological indicators of Augmented Reality (AR)-enabled e-commerce adoption among Gen Z in India. It studies the direct effects of trust, compatibility, enjoyment, usefulness, ease of use, and AR perception on the intention to use such a platform. Additionally, we identify each variable's normalized relative relevance and the essential factors driving adoption. Criterion-based purposive sampling was employed to collect data from Gen Z e-commerce users across India. 658 usable responses were collected over three months. The dataset was analyzed using Partial Least Squares Structural Equation Modelling, Artificial Neural Networks, and Necessary Condition Analysis. The analysis revealed that enjoyment and compatibility are the strongest predictors of AR adoption, ranked first and second, respectively, and are a must-have with a medium effect size. Following that, ease of use and AR perception establish a significant relationship and are found to be moderately necessary. Remarkably, while trust and usefulness are statistically significant, they rank the lowest, indicating that they are foundational but not top predictors. Previous research focused on features specific to AR instead of fully grasping genuine user interaction. This research closes a significant gap in the academic discourse by providing a model for AR adoption in a developing market with factors that account for users' interaction with AR features. Investing in gamified, high-quality, and realistic AR experiences and alignment to users' lifestyles can drive adoption, leading to customer retention and loyalty.

Augmented reality (AR) adoption in archeological site museums has attracted growing attention; however, a notable gap remains in understanding how personal traits and technological perceptions interact to influence tourists’ choices. To address this gap, this study conducted at the Sanxingdui Archeological Site Museum employs the Technology Acceptance Model (TAM), a widely used framework for analyzing user acceptance of technology, alongside structural equation modeling to examine the nuanced interplay of Technology Readiness dimensions and their impact on perceptions. The findings reveal that optimism does not significantly influence attitudes toward AR, while innovativeness has a positive effect. Discomfort is negatively associated with AR attitudes, while perceived security concerns have a counterintuitive positive effect. Moreover, perceived ease of use strongly determines the perceived usefulness of AR, which, in turn, directly influences attitudes toward the technology. Perceived ease of use also plays a crucial role in shaping AR attitudes. Additionally, a positive attitude toward AR is strongly linked to future adoption intentions, which, in turn, significantly correlate with tourists’ intentions to visit destinations. These insights not only enhance academic discussions on TAM and AR in tourism but also offer valuable practical implications for the museum sector seeking seamless AR integration.

Controlling assistive robots can be challenging for some users, especially those lacking relevant experience. Augmented Reality (AR) User Interfaces (UIs) have the potential to facilitate this task. Although extensive research regarding legged manipulators exists, comparatively little is on their UIs. Most existing UIs leverage traditional control interfaces such as joysticks, Hand-Held (HH) controllers and 2D UIs. These interfaces not only risk being unintuitive, thus discouraging interaction with the robot partner, but also draw the operator’s focus away from the task and towards the UI. This shift in attention raises additional safety concerns, particularly in potentially hazardous environments where legged manipulators are frequently deployed. Moreover, traditional interfaces limit the operators’ availability to use their hands for other tasks. Towards overcoming these limitations, in this article, we provide a user study comparing an AR Head-Mounted Display (HMD) UI we developed for controlling a legged manipulator against off-the-shelf control methods for such robots. This user study involved 27 participants and 135 trials, from which we gathered over 405 completed questionnaires. These trials involved multiple navigation and manipulation tasks with varying difficulty levels using a Boston Dynamics’s Spot, a 7 df Kinova robot arm and a Robotiq 2F-85 gripper that we integrated into a legged manipulator. We made the comparison between UIs across multiple dimensions relevant to a successful human–robot interaction. These dimensions include cognitive workload, technology acceptance, fluency, system usability, immersion and trust. Our study employed a factorial experimental design with participants undergoing five different conditions, generating longitudinal data. Due to potential unknown distributions and outliers in such data, using parametric methods for its analysis is questionable, and while non-parametric alternatives exist, they may lead to reduced statistical power. Therefore, to analyse the data that resulted from our experiment, we chose Bayesian data analysis as an effective alternative to address these limitations. Our results show that AR UIs can outpace HH-based control methods and reduce the cognitive requirements when designers include hands-free interactions and cognitive offloading principles into the UI. Furthermore, the use of the AR UI together with our cognitive offloading feature resulted in higher usability scores and significantly higher fluency and Technology Acceptance Model scores. Regarding immersion, our results revealed that the response values for the AR Immersion questionnaire associated with the AR UI are significantly higher than those associated with the HH UI, regardless of the main interaction method with the former, i.e., hand gestures or cognitive offloading. Derived from the participants’ qualitative answers, we believe this is due to a combination of factors, of which the most important is the free use of the hands when using the HMD, as well as the ability to see the real environment without the need to divert their attention to the UI. Regarding trust, our findings did not display discernible differences in reported trust scores across UI options. However, during the manipulation phase of our user study, where participants were given the choice to select their preferred UI, they consistently reported higher levels of trust compared to the navigation category. Moreover, there was a drastic change in the percentage of participants that selected the AR UI for completing this manipulation stage after incorporating the cognitive offloading feature. Thus, trust seems to have mediated the use and non-use of the UIs in a dimension different from the ones considered in our study, i.e., delegation and reliance. Therefore, our AR HMD UI for the control of legged manipulators was found to improve human–robot interaction across several relevant dimensions, underscoring the critical role of UI design in the effective and trustworthy utilisation of robotic systems.

Augmented reality (AR) is an emerging technology in microsurgery for free flap reconstruction. AR allows surgeons to superimpose preoperative CT images onto the patient, providing real-time insight into the underlying anatomy and aiding in the decision-making process; however, there is limited empirical evidence supporting the effectiveness of this modality. We conducted a systematic review to investigate whether the use of AR devices in free flap reconstruction correlates with improved outcomes. A comprehensive search of PubMed, Embase, and Scopus was performed on February 19, 2025. Our search included retrospective and prospective articles that demonstrated the use of AR technology intraoperatively and reported surgical outcomes in any type of free flap reconstruction. The initial search yielded 158 articles, with 8 ultimately included in the study. Articles were reviewed for technical accuracy, surgeon ease of use, operative time, complications, and operative costs. AR can guide surgeons, particularly those with less experience. As surgeons gain a better understanding of perforator locations, they operate with increased confidence and efficiency. AR outperformed Doppler ultrasound in perforator identification, demonstrating higher accuracy during flap planning and dissection. The time required to harvest flaps was significantly shorter when AR was used compared to traditional flap dissection techniques. Compared to computer-aided design and computer-aided manufacturing (CAD/CAM) navigation systems, AR surgical guidance exhibited greater accuracy and required significantly less time for flap design. AR devices surpassed cutting guides in various aspects, including cost savings, reduced preparation time, increased flexibility in surgical plan modification, intraoperative tissue drift, and a more favorable effect on hand-eye coordination. No conclusive postoperative complications were reported. The majority of the literature about AR in free flap surgery comes from small clinical studies. Widescale, multi-center studies are necessary to determine the true clinical impact of AR technology in microsurgery.

When delivering presentations to a co-located audience, we typically use slides with text and 2D graphics to complement the spoken narrative. Though presentations have largely been explored on 2D media, augmented reality (AR) allows presentation designers to add data and augmentations to existing physical infrastructure on display. This coupling could provide a more engaging experience to the audience and support comprehension. With HydrogenAR, we present a novel application that leverages the benefits of data-driven storytelling with those of AR to explain the unique challenges of hydrogen dispenser reliability. Utilizing physical props, situated data, and virtual augmentations and animations, HydrogenAR serves as a unique presentation tool, particularly critical for stakeholders, tour groups, and VIPs. HydrogenAR is a product of multiple collaborative design iterations with a local Hydrogen Fuel research team and is evaluated through interviews with team members and a user study with end-users to evaluate the usability and quality of the interactive AR experience. Through this work, we provide design considerations for AR data-driven presentations and discuss how AR could be used for innovative content delivery beyond traditional slide-based presentations.