Frustration Free Pose Computation For Spatial AR Devices in Industrial Scenario

We introduce a general technique of dynamically augmenting physical models. Using a handheld projector attached with an RGBD camera, we map the physical model with corresponding textures, and develop a real-time dynamic spatial augmented reality (SAR) system named iSarProjection. Compared to traditional huge and complex tracking system, this technique, based on KinectFusion, supports precise registration between the physical model point cloud and the reconstructed 3D scene, and provides real-time pose estimation to track the physical display surface effectively. Through projecting medical anatomy visualizations on a white body model, it presents that we ease the development of dynamic SAR system and make it possible to be applied in various research fields.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Reality & Augmented Reality in Industry collects the proceedings of the 2nd Sino-German Workshop on the same topic held in Shanghai on April 16-17, 2009. The papers focus on the latest Virtual Reality (VR) / Augmented Reality (AR) technology and its application in industrial processes and presents readers with innovative methods, typical case studies and the latest information on VR/AR basic research results and industrial applications, such as 3D rendering, innovative human-machine design, VR/AR methodology and new tools for assisting in industry, virtual assembly, virtual factory, training and education, etc. The book is intended for computer scientists, IT engineers as well as researchers in Mechanical Engineering. Dr. Dengzhe Ma and Dr. Xiumin Fan are both professors at Shanghai Jiao Tong University, China; Dr.-Ing. Jrgen Gausemeier is a professor of Computer-Integrated Manufacturing at the Heinz Nixdorf Institute, University of Paderborn, Germany; Dipl.-Ing. Michael Grafe is a senior engineer in the Product Engineering Research Group at the Heinz Nixdorf Institute, University of Paderborn.

Large companies use a lot of resources on workshop operator training and industrial machinery maintenance since the lack of this practice or its poor implementation increases the cost and risks of operating and handling sensitive and/or hazardous machinery. Industrial Augmented Reality (IAR), a major technology in the Industry 4.0 paradigm that may enhance worker performance, minimize hazards and improve manufacturing processes, could be beneficial in this situation. This paper presents an IAR solution that allows for visualizing and interacting with the digital twin of a critical system. Specifically, the augmented digital twin of an industrial cooler was developed. The proposed IAR system provides a dynamic way to perform operator training with a full-size model of the actual equipment and to provide step-by-step guidance so that maintenance processes can be performed more safely and efficiently. The proposed system also allows several users to use devices at the same time, creating a new type of collaborative interaction by viewing the model in the same place and state. Performance tests with many simultaneous users have been conducted, with response latency being measured as the number of connected users grows. Furthermore, the suggested IAR system has been thoroughly tested in a real-world industrial environment.

Augmented Reality is considered one of the most promising technologies for Industry 4.0. Augmented Reality allows to display the needed information at the right time and to locate it in the desired space, superimposing it to the real world. In this way, it could simplify the work of industrial operators in a variety of tasks, from planning and commissioning to assembly and maintenance. Despite the growing research interest and efforts for the development of Augmented Reality applications and technologies for the industrial domain, Augmented Reality is still rarely used in real industrial procedures, often remaining at a conceptual level. In fact, developers fail to completely answer the actual needs of industries and workers because of a lack of knowledge of the context and the absence of guidelines to drive the design and development of Augmented Reality applications. One of the open issues is the choice of the proper visualization methods to display technical information in Augmented Reality, which is not trivial. Both the paper-based documentation, based on text and illustrations, and digital documentation, including CAD models and image or video-based tutorials, present different criticalities when adapted to the Augmented Reality technology. Other visual material has been developed specifically to be used in Augmented Reality, but it is difficult to use because of a lack of standards which are instead present for the former paper-based and digital documentation. In this work, we categorize and compare different Augmented Reality visual assets, illustrating their advantages and disadvantages and providing directions for future research.

In the recent years, many industrial augmented reality (IAR) applications are shifting from video to still images to create a mixed view. This new type of application is called photo-based augmented reality. In order to guarantee the success of these applications, a simple and efficient registration method is required. We present a new method to register an image to a CAD model using a single keyframe. This registration is based on sparse 3D information from the model linked to the keyframe during its offline registration. We demonstrate this method in our in-house IAR software for visual inspection and documentation: VID.

Spatial augmented reality (SAR) is an emerging paradigm that differs from its origin, the traditional augmented reality (AR), in many regards. While traditional AR is a well-studied field of research, the characteristic features of SAR and their implications on the perception of spatial augmented environments have not been analyzed so far. In this paper, we present one of the first studies, which investigates the perceived spatial relationships between a user and their SAR environment. The results indicate that perceived depth of realworld objects can be manipulated by projecting illusions, such as color or blur effects, onto their surfaces. For the purpose of evaluating and comparing the illusions of interest, we developed a prototypic setup for conducting perceptual SAR experiments. Since this testing environment differs significantly from its counterparts in virtual and augmented reality, we also discuss potential challenges, which arise from the nature of SAR experiments.

Concepts for user interfaces are developed in order to enable humans exchanging information and having a better understanding of different processes. With the rise of technology, concepts of enriching our environments with computer-generated graphics appeared in the 1960s. The expansion of visual perception of the surrounding is the core of Augmented Reality (AR). Many AR applications have been appearing in different areas, opening the door for revolutionizing and changing interactivity models and the way people perform tasks. Even though AR systems’ structures are well defined, applications are not yet widespread. One of the main reasons for this is that AR system developers face difficulties with developing their applications and adapting them to different hardware. Each hardware set (i.e. AR device) requires different software development kits and different technical requirements. That leaves AR concepts hard to be scaled and implemented in real life scenarios. One more additional challenge facing the implementing of AR is that AR mostly relies on complex vision-based algorithms. Based on that, this paper explores a lightweight, modular, and cross-platform solution. It presents a novel framework based on redesigning the AR pipeline, introducing a prototype of a solution where AR can operate within the cloud, converting all AR devices into thin clients with high performance capability. This also contributes directly to the Industry 4.0 aim of preparing the way for digitalized and optimized processes in a connected and integrated network.

In recent years, hardware and software advances in digital technology have prompted not only the commercial use of augmented reality (AR) systems, but its implementation in education. Marker-based AR systems allow users to input pre-designed markers, and generate new scenes blending together the real world and computer-generated 3D images. Nevertheless, few AR systems permit simultaneous interactions of multiple markers. This study aims to construct an interactive multimedia learning system. The story script showcases the growth of flowers by applying a series of markers, where interactions with 3D objects can be achieved through AR technology. Learners can use a camera to capture the information of multiple AR markers, and 3D objects and media can be generated on-screen. With variations in AR markers, and time changes of the Arduino Clock, one can learn the growth of plants during day and night through a mixed-reality virtual experience that brings fun to learning. Accordingly, this e-learning system is capable of boosting students' learning interests and improving their learning effectiveness.

With the development of Apple’s ARKit and Google’s ARCore, mobile augmented reality (AR) applications have become much more popular. For Android devices, ARCore provides basic motion tracking and environmental understanding. However, with current software frameworks it can be difficult to create an AR application from the ground up. Our solution is CalAR, which is a lightweight, open-source software environment to develop AR applications for Android devices, while giving the programmer full control over the phone’s resources. With CalAR, the programmer can create marker-less AR applications which run at 60 frames per second on Android smartphones. These applications can include more complex environment understanding, physical simulation, user interaction with virtual objects, and interaction between virtual objects and objects in the physical environment. With CalAR being based on CalVR, which is our multi-platform virtual reality software engine, it is possible to port CalVR applications to an AR environment on Android phones with minimal effort. We demonstrate this with the example of a spatial visualization application.

Rapid development in processing power, graphic cards and mobile computers open up a wide domain for Mixed Reality applications. Thereby the Mixed Reality continuum covers the complete spectrum from Virtual Reality using immersive projection technology to Augmented Reality using mobile systems like Smartphones and UMPCs. At the Fraunhofer IGD the Mixed Reality Framework instantreality (www.instantreality. org) has been developed as a single and consistent interface for AR/VR developers. This framework provides a comprehensive set of features to support classic Virtual Reality (VR) as well as mobile Augmented Reality (AR). The goal is to provide a very simple application interface which includes the latest research results in the fields of high-realistic rendering, 3D user interaction and total-immersive display technology. The system design is based on various industry standards to facilitate application development and deployment.

Introduction: Augmented Reality (AR) systems are systems in which users view and interact with virtual objects overlaying the real world. AR systems are used across a variety of disciplines, i.e., games, medicine, and education to name a few. Optical See-Through (OST) AR displays allow users to perceive the real world directly by combining computer-generated imagery overlaying the real world. While perception of depth and visibility of objects is a widely studied field, we wanted to observe how color, luminance, and movement of an object interacted with each other as well as external luminance in OST AR devices. Little research has been done regarding the issues around the effect of virtual objects’ parameters on depth perception, external lighting, and the effect of an object’s mobility on this depth perception.Methods: We aim to perform an analysis of the effects of motion cues, color, and luminance on depth estimation of AR objects overlaying the real world with OST displays. We perform two experiments, differing in environmental lighting conditions (287 lux and 156 lux), and analyze the effects and differences on depth and speed perceptions.Results: We have found that while stationary objects follow previous research with regards to depth perception, motion and both object and environmental luminance play a factor in this perception.Discussion: These results will be significantly useful for developers to account for depth estimation issues that may arise in AR environments. Awareness of the different effects of speed and environmental illuminance on depth perception can be utilized when performing AR or MR applications where precision matters.

Cyber-physical systems and automation play a major role in Industry 4.0. However, due to its flexibility and high responsiveness to short-term changes in the assembly of complex and varied products, manual work cannot be neglected. Cognitive assistance systems relieve the mental stress of a worker and thus help prevent mistakes during the assembly process. Systems utilizing augmented reality (AR) technologies are suitable for manual operations. In order to motivate the development needs of industrial augmented reality (IAR) applications, a prototype is described below. This paper focuses on the conceptual development and the implementation of a prototype assistance system. By using AR glasses and a data glove, an inexperienced worker is guided through a demonstrative assembly process where each assembly step is documented by the system. Finally, a user test is performed to evaluate the prototype.

A spatial AR system for wide-area axis-aligned metric augmentation of planar scenes

Research on how to take advantage of Augmented Reality and Virtual Reality applications and technologies in the domain of manufacturing has brought forward a great number of concepts, prototypes, and working systems. Although comprehensive surveys have taken account of the state of the art, the design space of industrial augmented and virtual reality keeps diversifying. We propose a visual approach towards assessing this space and present an interactive, community-driven tool which supports interested researchers and practitioners in gaining an overview of the aforementioned design space. Using such a framework we collected and classified relevant publications in terms of application areas and technology platforms. This tool shall facilitate initial research activities as well as the identification of research opportunities. Thus, we lay the groundwork, forthcoming workshops and discussions shall address the refinement.