3D Reconstruction of Underwater Natural Scenes and Objects Using Stereo Vision

In this paper, we propose possibility for reconstruction of surface of an underwater object or 3D scene reconstruction of an underwater environment using an economical RGB-D sensor such as Microsoft Kinect. Reconstructing the 3D surface of an underwater object is a challenging task due to degraded quality of underwater images. There are various reasons of quality degradation of underwater images i.e., non-uniform illumination of light on the surface of objects, scattering and absorption effects. Particles and impurities present in underwater produces Gaussian noise on the captured underwater optical images which degrades the quality of images. However, using depth sensors, as a cost effective alternative, we aim to show that underwater 3D scene reconstruction is possible with sight tradeoffs on accuracy but major cost saving. The acquired depth data is proposed to be processed by applying real-time mesh generating techniques from the acquired point cloud. The experimental result aims to show that the proposed method reconstructs 3D surface of underwater objects accurately using captured underwater depth images.

This work deals with the automatic 3D reconstruction of objects from frontal RGB images. This aims at a better understanding of the reconstruction of 3D objects from RGB images and their use in immersive virtual environments. We propose a complete workflow that can be easily adapted to almost any other family of rigid objects. To explain and validate our method, we focus on guitars. First, we detect and segment the guitars present in the image using semantic segmentation methods based on convolutional neural networks. In a second step, we perform the final 3D reconstruction of the guitar by warping the rendered depth maps of a fitted 3D template in 2D image space to match the input silhouette. We validated our method by obtaining guitar reconstructions from real input images and renders of all guitar models available in the ShapeNet database. Numerical results for different object families were obtained by computing standard mesh evaluation metrics such as Intersection over Union, Chamfer Distance, and the F-score. The results of this study show that our method can automatically generate high-quality 3D object reconstructions from frontal images using various segmentation and 3D reconstruction techniques.

With the increasing use of image-based 3D reconstruction in medical procedures, accurate scene reconstruction plays a crucial role in surgical navigation and assisted treatment. However, the monotonous colors, limited image features, and obvious brightness fluctuations of thoracoscopic scenes make the feature point matching process, on which traditional 3D reconstruction methods rely, unstable and unreliable. It brings a great challenge to accurate 3D reconstruction. In this study, a new method for implicit 3D reconstruction of monocular thoracoscopic scenes is proposed. The method combines a pre-trained metric depth estimation model with neural radiation field (NeRF) technique and uses dense SLAM to accurately compute the camera pose. To ensure the accuracy of the depth values and the structural consistency of the reconstructed scene, depth and normal constraints are added to the original color constraints of the NeRF network to achieve high-quality scene reconstruction results. We conducted experiments on the SCARED dataset and the clinical dataset. After comparing with other methods, the depth estimation accuracy and point cloud reconstruction quality of this paper outperform the existing methods. The method in this paper can provide more accurate 3D reconstruction of complex thoracic surgical scenes, which can significantly improve the accuracy and therapeutic efficacy of surgical navigation.

This paper presents preliminary work to utilize a commercial time of flight depth camera for real-time 3D scene reconstruction of underwater objects. Typical RGB stereo camera imaging for 3D capturing suffers from blur and haziness due to turbidity of water in addition to critical dependence on light, either from natural or artificial sources. We propose a method for repurposing the low-cost Microsoft Kinect™ Time of Flight camera for underwater environment enabling dense depth data acquisition that can be processed in real time. Our motivation is the ease of use and low cost of the device for high quality real-time scene reconstruction as compared to multi view stereo cameras, albeit at a smaller range. Preliminary results of depth data acquisition and surface reconstruction in underwater environment are also presented. The novelty of our work is the utilization of the Kinect depth camera for real-time 3D mesh reconstruction and our main objective is to develop an economical and compact solution for underwater 3D mapping.

Effective measurement of the reflective or transparent surface of an object has always been a disadvantage in laser scanning modeling. We propose a fast and complete three-dimensional (3D) reconstruction method for small static objects using laser scanning and the structure from motion (SFM) algorithm. Meanwhile, a complete reconstruction workflow is designed and a multi-angle 3d reconstruction system is set up. To generate the complete point cloud model of the object, the SFM algorithm is used to reconstruct the surface part of the object, the data for which cannot be obtained by the laser measuring instrument. The experimental results show that this method not only improves the speed, accuracy, integrity, and visual effect of 3D reconstruction of small objects, but also extends the scope of 3D reconstruction of laser measurement.

Three-dimensional (3D) reconstruction using line-structured-light-based stereo vision is one key technique in the field of computer vision measurement. However, most existing stereo vision methods are static and have limited measurement range. To overcome the defect in static stereo vision, a 3D reconstruction method for sparsely textured surfaces via freely moving line-structured-light-based binocular stereo vision is proposed in this paper. The light stripes projected on the surface are captured by left and right cameras, and the pixel coordinates of the light strips are extracted by the Steger algorithm. The 3D coordinates of the light stripes at each position of the cameras are calculated based on the principle of binocular intersection measurement. Subsequently, at least six feature points are required to estimate the coarse relative rotation and translation between adjacent positions, and the corresponding precise values are computed based on minimization of pixel reprojection errors. The camera pose at each position is ultimately obtained through step-by-step backward propagation by using the relative rotation and translation between all adjacent positions. Thus, the 3D shape of the surface is obtained quickly once the light strips captured at different positions are unified to the reference position. Moreover, two metallic surfaces with different geometric configurations are used for the 3D reconstruction test, and the 3D shapes of the metallic surfaces are reconstructed effectively, which proves the generalizability of the proposed method to different surface types. To evaluate the accuracy of 3D reconstruction, the 3D point cloud of one metallic plane is fitted into a plane, and the average vertical distance of the 3D point cloud from the fitting plane is regarded as the evaluation criterion. The experimental results reflect that the average vertical distance in each experimental trial and the average value of the standard deviation of vertical distance do not exceed 1.52 and 0.97mm, respectively.

Abstract3D reconstruction of deformable (ornon‐rigid) scenes from a set of monocular 2D image observations is a long‐standing and actively researched area of computer vision and graphics. It is an ill‐posed inverse problem, since—without additional prior assumptions—it permits infinitely many solutions leading to accurate projection to the input 2D images. Non‐rigid reconstruction is a foundational building block for downstream applications like robotics, AR/VR, or visual content creation. The key advantage of using monocular cameras is their omnipresence and availability to the end users as well as their ease of use compared to more sophisticated camera set‐ups such as stereo or multi‐view systems. This survey focuses on state‐of‐the‐art methods for dense non‐rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views. It reviews the fundamentals of 3D reconstruction and deformation modeling from 2D image observations. We then start from general methods—that handle arbitrary scenes and make only a few prior assumptions—and proceed towards techniques making stronger assumptions about the observed objects and types of deformations (e.g. human faces, bodies, hands, and animals). A significant part of this STAR is also devoted to classification and a high‐level comparison of the methods, as well as an overview of the datasets for training and evaluation of the discussed techniques. We conclude by discussing open challenges in the field and the social aspects associated with the usage of the reviewed methods.

With the development of computer vision technologies, 3D reconstruction has become a hotspot. At present, 3D reconstruction relies heavily on expensive equipment and has poor real-time performance. In this paper, we aim at solving the problem of 3D reconstruction of an indoor scene with large vertical span. In this paper, we propose a novel approach for 3D reconstruction of indoor scenes with only a Kinect. Firstly, this method uses a Kinect sensor to get color images and depth images of an indoor scene. Secondly, the combination of scale-invariant feature transform and random sample consensus algorithm is used to determine the transformation matrix of adjacent frames, which can be seen as the initial value of iterative closest point (ICP). Thirdly, we establish the relative coordinate relation between pair-wise frames which are the initial point cloud data by using ICP. Finally, we achieve the 3D visual reconstruction model of indoor scene by the top-down image registration of point cloud data. This approach not only mitigates the sensor perspective restriction and achieves the indoor scene reconstruction of large vertical span, but also develops the fast algorithm of indoor scene reconstruction with large amount of cloud data. The experimental results show that the proposed algorithm has better accuracy, better reconstruction effect, and less running time for point cloud registration. In addition, the proposed method has great potential applied to 3D simultaneous location and mapping.

Structured light 3D vision inspection is widely used in various 3D surface reconstruction techniques. However, there are still two problems to be solved. The first problem is that if the relative position between the camera and laser projector is changed, the system has to be calibrated again. The second problem is that a turntable or a linear positioning system is needed in order to obtain the complete 3D reconstruction data in conventional methods. To overcome these problems, we proposed a novel design using a dual semicircular planar target and a rotating laser projector to perform calibration and 3D reconstruction simultaneously. The system contains two semicircular planar targets hollowed out at center which are installed in up and down position. The middle part of the light stripe can pass through the blank part of semicircular planar target and then cast on the target object, while the rest of the light stripe will cast on the semicircular planar target. The laser projector is rotated 360 degrees along an axis perpendicular to the planar target plane to get the complete 3D surface reconstruction of the target object. Experiments are conducted to validate the flexibility and accuracy of the system.

3D reconstruction of free-formed line-like objects using NURBS representation

Mapping of underwater environments is a critical task for a range of activities from monitoring coral reef habitats to surveying submerged archaeological sites. While recent advances in methods for terrestrial mapping can achieve dense 3D reconstructions of scenes in real-time, there remains the challenge of transferring these methods to the underwater domain due to characteristic effects on propagation of light through the water column that violate the brightness constancy constraint used in terrestrial techniques. Current state-of-the-art methods for underwater 3D reconstruction exploit a physical model of light propagation underwater to account for such range-dependent effects as scattering and attenuation; however, these methods necessitate careful calibration of attenuation coefficients required by the physical model, or rely on rough estimates of these coefficients from prior lab experiments. The main contribution of this paper is to develop a novel method to achieve simultaneous estimation of attenuation coefficients for color correction during structure recovery of an underwater scene by integrating this estimation directly into the bundle adjustment step, which performs non-linear optimization. To validate the proposed method, an artificial scene is submerged in a pure water tank and surveyed with a stereo camera platform to simulate an underwater robotic survey in a controlled environment. The target structure is imaged in air with an RGB-D sensor to provide ground truth structure and color, and a color calibration board is place in the scene for further reference. Results show that the proposed method can automatically estimate a water-column aware model for color correction of underwater images simultaneously to 3D reconstruction of the submerged scene.

In previous work, we presented a structured light field (SLF) method combining light field imaging with structured illumination to perform multi-view depth measurement. However, the previous work just accomplishes depth rather than 3D reconstruction. In this paper, we propose a novel active method involving ray calibration and phase mapping, to achieve SLF 3D reconstruction. We performed the ray calibration for the first time to determine each light field ray with metric spatio-angular parameters, making the SLF realize multi-view 3D reconstruction. Based on the ray parametric equation, we further derived the phase mapping in the SLF that spatial coordinates can be directly mapped from phase. A flexible calibration strategy was correspondently designed to determine mapping coefficients for each light field ray, achieving high-efficiency SLF 3D reconstruction. Experimental results demonstrated that the proposed method was suitable for high-efficiency multi-view 3D reconstruction in the SLF.

High-quality 3D shape recovery from scattering scenario via deep polarization neural networks

Reconstruction of high-fidelity 3D objects or scenes is a fundamental research problem. Recent advances in RGB-D fusion have demonstrated the potential of producing 3D models from consumer-level RGB-D cameras. However, due to the discrete nature and limited resolution of their surface representations (e.g., point or voxel based), existing approaches suffer from the accumulation of errors in camera tracking and distortion in the reconstruction, which leads to an unsatisfactory 3D reconstruction. In this article, we present a method using on-the-fly implicits of Hermite Radial Basis Functions (HRBFs) as a continuous surface representation for camera tracking in an existing RGB-D fusion framework. Furthermore, curvature estimation and confidence evaluation are coherently derived from the inherent surface properties of the on-the-fly HRBF implicits, which are devoted to a data fusion with better quality. We argue that our continuous but on-the-fly surface representation can effectively mitigate the impact of noise with its robustness and constrain the reconstruction with inherent surface smoothness when being compared with discrete representations. Experimental results on various real-world and synthetic datasets demonstrate that our HRBF-fusion outperforms the state-of-the-art approaches in terms of tracking robustness and reconstruction accuracy.

Tele-immersive systems enable 3D interactive experience in a virtual space by bringing together objects born in physical and virtual reality environments that are geographically distributed. The immersive and interactive experiences are achieved by fusing real-time color plus depth video of physical scenes from multiple stereo cameras, displaying 3D reconstructions of physical and virtual objects, and tracking intersections of objects in a virtual space to facilitate interactions between objects. While tele-immersive (TI) systems have been attracting a lot of attention these days as a possible advancement of the current 2D video-based communication systems and of the virtual reality spaces, the challenges lie in limited resources available to satisfy the many requirements on a usable TI system. One of the TI challenges remains the quality of real-time 3D reconstruction of physical scenes. Our work is motivated by the lack of understanding of the importance of various TI system parameters on the quality which is critical for allocating limited resources to make TI systems useful. This paper investigates the impact of different quality parameters of TI systems on accuracy and speed of task executions in TI environments. We have designed a task of moving a virtual ball from an initial center position to a basketball basket close to top in a closed virtual cuboid space. Successful completion of the task involves localization, orientation and motion coordination for a human in TI environment. We report quantitative and qualitative measurements collected during task executions under simulated variations of 3D reconstruction quality parameters. Based on our experimental results with 29 subjects under 13 quality variations, we concluded that the most important parameter is the frame rate followed by the presence of of a human rendering rather than just an avatar representation of hands.