An improved augmented-reality method of inserting virtual objects into the scene with transparent objects

The perceptual issues in Augmented Reality (AR) systems, is drawing more and more attention these days. Among the perceptual issues, the incorrect depth interpretations degrade user experience significantly. One way to improve the depth interpretations in AR systems is to improve the consistency of the real scene and the virtual objects by using the depth cue method based on blurring effect. In the research of [Okumura 2006], a blur rendering method based on measuring the defocusing on the AR marker from the captured real scene has been proposed. However, the blur effect could only be rendered on the marker position where the blur in the real scene is measured. This limits the practicality of the system since in many applications the virtual objects are out of the marker's range or even moving. In our work, a new method to estimate the degree of blur is proposed. This method makes it possible to calculate the degree of blur that varies spatially with the scene.

Achieving seamless integration between virtual objects and real scenes has always been an important issue in augmented reality (AR) research. To achieve this, it is necessary to provide virtual objects with real-time and accurate lighting conditions from a real scene. Therefore, the purpose of this study is to realize lighting consistency rendering for real-time AR systems in outdoor environments, aiming to enhance the user’s sense of immersion. In this paper, we propose a lighting consistency technique for real-time AR systems in outdoor environments based on multi-source geographical information (MGI). Specifically, we introduce MGI into the study of lighting consistency and construct a comprehensive database to store and manage the acquired MGI data. Based on this, we proposed a sky radiance model driven using the MGI. Finally, we utilized the sky radiance model along with light sensor data to render the virtual objects in outdoor scenes. The experimental results show that the shadow angular error is reduced to 5.2°, and the system frame rate is increased to 94.26. This means that our method achieves a high level of realism in the fusion of virtual objects and real scenes while ensuring a high frame rate in the system. With this technology, users can conveniently and extensively realize the lighting consistency rendering of real-time AR systems in outdoor scenes using mobile devices.

Augmented Reality (AR) is one of the latest developments in human-computer interaction technology. It aims to generate illusions from seamless fusion of virtual objects and real world. Typical AR system requires two basic parts: three-dimensional registration and real-virtual fusion. Occlusion handling is crucial for visual realism. To optimize visual realism, we generated a real-time systematic architecture to operate occlusion handling. The architecture is based on RGBD images, and it consists of three parts: real-time camera tracking system, 3D reconstruction system and AR fusion system. Specifically, we used a two-pass scheme strategy to execute the AR system. The first pass tracks camera poses timely at video rate, which allows the reconstruction results be updated and visualized correspondingly during the scanning. The second pass takes place simultaneously to handle occlusion between virtual objects and real scene according to camera pose. Finally, the render results of virtual objects and the color images are fused to generate AR contents. Our results indicate that this method is stable and precise for occlusion handling, and can effectively improve realism in AR system.

Recupera algunos de los elementos menos estudiados y reconocidos en el ensayo latinoamericano: su profunda vocación de reflexión, diálogo y mediación intelectual; su plasticidad y apertura a la búsqueda del sentido; su vínculo con la sociabilidad, la conversación y la generación de un espacio simbólico de encuentro; su capa cidad de establecer una nueva retroalimentación entre lectura y escritura.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

The immersive augmented reality (AR) system necessitates precise depth registration between virtual objects and the real scene. Prior studies have emphasized the efficacy of surface texture in providing depth cues to enhance depth perception across various media, including the real scene, virtual reality, and AR. However, these studies predominantly focus on black-and-white textures, leaving a gap in understanding the effectiveness of colored textures. To address this gap and further explore texture-related factors in AR, a series of experiments were conducted to investigate the effects of different texture cues on depth perception using the perceptual matching method. Findings indicate that the absolute depth error increases with decreasing contrast under black-and-white texture. Moreover, textures with higher color contrast also contribute to enhanced accuracy of depth judgments in AR. However, no significant effect of texture density on depth perception was observed. The findings serve as a theoretical reference for texture design in AR, aiding in the optimization of virtual-real registration processes.

Introduction: Augmented Reality (AR) systems are systems in which users view and interact with virtual objects overlaying the real world. AR systems are used across a variety of disciplines, i.e., games, medicine, and education to name a few. Optical See-Through (OST) AR displays allow users to perceive the real world directly by combining computer-generated imagery overlaying the real world. While perception of depth and visibility of objects is a widely studied field, we wanted to observe how color, luminance, and movement of an object interacted with each other as well as external luminance in OST AR devices. Little research has been done regarding the issues around the effect of virtual objects’ parameters on depth perception, external lighting, and the effect of an object’s mobility on this depth perception.Methods: We aim to perform an analysis of the effects of motion cues, color, and luminance on depth estimation of AR objects overlaying the real world with OST displays. We perform two experiments, differing in environmental lighting conditions (287 lux and 156 lux), and analyze the effects and differences on depth and speed perceptions.Results: We have found that while stationary objects follow previous research with regards to depth perception, motion and both object and environmental luminance play a factor in this perception.Discussion: These results will be significantly useful for developers to account for depth estimation issues that may arise in AR environments. Awareness of the different effects of speed and environmental illuminance on depth perception can be utilized when performing AR or MR applications where precision matters.

Endoscopic surgery causes less tissue injury compared to open surgical techniques, thus promoting more rapid recuperation and reduced post-operative pain. Endoscopy, however, allows the surgeon to visualise only the anatomical surface of the surgical site, with a relatively narrow field of view. Moreover, the 2D video captured by the conventional endoscope does not provide depth perception of the surgical scene. In this study, these limitations have been addressed with the development of an augmented reality (AR) system with stereoscopic visualisation. A phantom and its 3D CT model were used, respectively, to form the real and virtual parts of the AR. The virtual environment camera pose was tracked using algorithms for image feature detection, feature matching and Perspective-n-Point applied on the endoscopic image and the 3D virtual model-rendered image. The endoscope video frame- and the virtual model-rendered images were superimposed to form the AR composite view. The depth buffer (z-buffer) of the rendering window was further used to make a stereo pair of the AR image. The AR system produced a stereo composite view having well-aligned real and virtual components. The RMS error of the real and virtual image contours registration was 9.6 ± 6.7 mm. Correlation coefficients between the depth map from z-buffer and a depth camera was between 0.60 and 0.96 (p < 0.05). The AR system requires further improvement to be applicable at a higher frame rate of endoscope image acquisition. It also needs to include motion and deformation models when applied to animals or patients.

In an augmented reality system, Mutual occlusion and intensity matching of real and virtual environment enhance the user's feeling that virtual objects truly exist in the real world. Aiming at the attribute of conventional optical see-through HMD which cannot present mutual occlusion and intensity matching correctly, we propose a novel display design with mutual occlusion and intensity matching capabilities. An 8-bit gray level XGA LCD, by which mutual occlusion and intensity matching be accomplished is used as the addressable filter in our design. Furthermore, with our optical system, a viewer can simultaneously observe both outside scene and a pattern on the LCD panel in focus. Experimental results show that our novel design can integrate more seamlessly a virtual object in a real scene.

The measurement of transparent objects has always been regarded as one of the difficult problems in 3D reconstruction technology. For transparent objects conforming to the birefringence model, we propose a method for measuring the parameters of transparent objects based on multi-wavelength incident light. This paper analyzes the transparent objects where light enters the birefringence model. Because the transparent objects refract light of different wavelengths differently, when the lights with different wavelengths emitted from the same point and same direction passes through the transparent objects, the position and direction of light with different wavelength will come out differently. The position and direction of these outgoing rays are related to the refractive index and surface topography of the transparent objects. By obtaining the equations of these outgoing rays, we can derive the equations of the refractive index and surface topography of the transparent objects. In this paper, the incident light obtained by computer simulation is used as the initial data, and the simulated annealing algorithm is used to verify the model. The final results show that within the range of random error values given by the computer, the surface shape, curvature, thickness and refractive index of transparent objects with small error values can be obtained. At the same time, this paper introduces the Ceres Solver C++ nonlinear optimization library into the simulated annealing algorithm, which improves the speed and accuracy of the algorithm and greatly enhances the robustness of the algorithm.

While developers are mainly tackling primary problems in devel-oping augmented reality (AR) systems [1-2], perceptually-correct augmentation rests a critical challenge. In this paper, we focus on how to correctly display and accurately convey the augmented virtual object's size with respect to real-world objects. We conducted a user study to examine how subjects would verify relative size of virtual objects, augmented in a real scene. The results con-firmed that optical illusion occurs in AR applications if comparative size of virtual objects to real-world ones is not considered.

In May 2022, Saudi Arabian Military Industries, a Saudi government agency, acquired an augmented reality training platform for pilots. In September, the Boeing Corporation began the development of an augmented reality pilot simulator. In November, a similar project was launched by BAE Systems, a leading British developer of aeronautical engineering. These facts allow us to confidently speak about the beginning of a new era of aviation simulators – simulators using the augmented reality technology. One of the promising advantages of this technology is the ability to safely simulate dangerous situations in the real world. A necessary condition for using this advantage is to ensure the visual coherence of augmented reality scenes: virtual objects must be indistinguishable from real ones. All the global IT leaders consider augmented reality as the subsequent surge of radical changes in digital electronics, so visual coherence is becoming a key issue for the future of IT, and in aerospace applications, visual coherence has already acquired practical significance. The Russian Federation lags far behind in studying the problems of visual coherence in general and for augmented reality flight simulators in particular: at the time of publication the authors managed to find only two papers on the subject in the Russian research space, while abroad their number is already approximately a thousand. The purpose of this review article is to create conditions for solving the problem. Visual coherence depends on many factors: lighting, color tone, shadows from virtual objects on real ones, mutual reflections, textures of virtual surfaces, optical aberrations, convergence and accommodation, etc. The article reviews the publications devoted to methods for assessing the conditions of illumination and color tone of a real scene and transferring them to virtual objects using various probes and by individual images, as well as by rendering virtual objects in augmented reality scenes, using neural networks.

Real-time markerless Augmented Reality for Remote Handling system in bad viewing conditions

The mixed reality makes the user's realism strong or weak, depending on the fusion effect of the virtual object and the real scene, and the key factor affecting the virtual-real fusion effect is whether the virtual and real object in the synthetic scene has occlusion consistency. In this paper, a method is proposed that utilizes color and depth information provided by an RGB-D camera to improved occlusion. In this method, based on the GPU, extracting the ROI area where the virtual object is located (the area where the virtual-real occlusion occurs),then introducing the real scence color image as the guide image to modified joint bilateral filtering the depth image to repair the wrong edge information and "black hole" of the depth image;pixel-by-pixel point to determine the depth relationship between the virtual object and real object, to achieve virtual-real occlusion rendering blend;for the still existing "sawtooth artifacts" using delay-coloring only fuzzy local boundary algorithm, to achieve accurate and smoothing edges virtual-real occlusion to enhance visual effects.

Augmented Reality (AR) is the process of integrating virtual elements in reality, often by mixing computer graphics into a live video stream of a real scene. It requires registration of the target object with respect to the cameras. To this end, some approaches rely on dedicated hardware, such as magnetic trackers or infra-red cameras, but they are too expensive and cumbersome to reach a large public. Others are based on specifically designed markers which usually look like bar-codes. However, they alter the look of objects to be augmented, thereby hindering their use in application for which visual design matters. Recent advances in Computer Vision have made it possible to track and detect objects by relying on natural features. However, no such method is commonly used in the AR community, because the maturity of available packages is not sufficient yet. As far as deformable surfaces are concerned, the choice is even more limited, mainly because initialization is so difficult. Our main contribution is therefore a new AR framework that can properly augment deforming surfaces in real-time. Its target platform is a standard PC and a single webcam. It does not require any complex calibration procedure, making it perfectly suitable for novice end-users. To satisfy to the most demanding application designers, our framework does not require any scene engineering, renders virtual objects illuminated by real light, and let real elements occlude virtual ones. To meet this challenge, we developed several innovative techniques. Our approach to real-time registration of a deforming surface is based on wide-baseline feature matching. However, traditional outlier elimination techniques such as RANSAC are unable to handle the non-rigid surface's large number of degrees of freedom. We therefore proposed a new robust estimation scheme that allows both 2–D and 3–D non-rigid surface registration. Another issue of critical importance in AR to achieve realism is illumination handling, for which existing techniques often require setup procedures or devices such as reflective spheres. By contrast, our framework includes methods to estimate illumination for rendering purposes without sacrificing ease of use. Finally, several existing approaches to handling occlusions in AR rely on multiple cameras or can only deal with occluding objects modeled beforehand. Our requires only one camera and models occluding objects at runtime. We incorporated these components in a consistent and flexible framework. We used it to augment many different objects such as a deforming T-shirt or a sheet of paper, under challenging conditions, in real-time, and with correct handling of illumination and occlusions. We also used our non-rigid surface registration technique to measure the shape of deformed sails. We validated the ease of deployment of our framework by distributing a software package and letting an artist use it to create two AR applications.