Omnidirectional Camera Calibration and 3D Reconstruction by Contour Matching

Real-time three-dimensional (3D) reconstruction of real-world environments has many significant applications in various fields, including telepresence technology. When depth sensors, such as those from Microsoft's Kinect series, are introduced simultaneously and become widely available, a new generation of telepresence systems can be developed by combining a real-time 3D reconstruction method with these new technologies. This combination enables users to engage with a remote person while remaining in their local area, as well as control remote devices while viewing their 3D virtual representation. There are numerous applications in which having a telepresence experience could be beneficial, including remote collaboration and entertainment, as well as education, advertising, and rehabilitation. The purpose of this systematic literature review is to analyze the recent advances in 3D reconstruction methods for telepresence systems and the significant related work in this field. Next, we determine the input data and the technological device employed to acquire the input data, which will be utilized in the 3D reconstruction process. The methods of 3D reconstruction implemented in the telepresence system as well as the evaluation of the system, have been extracted and assessed from the included studies. Through the analysis and summarization of many dimensions, we discussed the input data used for the 3D reconstruction method, the real-time 3D reconstruction methods implemented in the telepresence system, and how to evaluate the system. We conclude that real-time 3D reconstruction methods for telepresence systems have progressively improved over the years in conjunction with the advancement of machines and devices such as Red Green Blue-Depth (RGB-D) cameras and Graphics Processing Unit (GPU).

In this paper, we propose a novel method for generating 3D line segment based model from an image sequence taken with a RGB-D camera. Constructing 3D geometrical representation by 3D model is essential for model based camera pose estimation that can be performed by corresponding 2D features in images with 3D features of the captured scene. While point features are mostly used as such features for conventional camera pose estimation, we aim to use line segment features for improving the performance of the camera pose estimation. In this method, using RGB images and depth images of two continuous frames, 2D line segments from the current frame and 3D line segments from the previous frame are corresponded. The 2D-3D line segment correspondences provide camera pose of the current frame. All of 2D line segments are finally back-projected to the world coordinate based on the estimated camera pose for generating 3D line segment based model of the target scene. In experiments, we confirmed that the proposed method can successfully generate line segment based models, while 3D models based on the point features often fail to successfully represent the target scene.

As the basis of intelligent breeding management and animal husbandry insurance, the identification of individual cattle is important in animal husbandry management. Given the difficulty of data acquisition caused by the non-rigid and lacking cooperation of cattle, this study proposes a method for cattle face image acquisition and processing that can efficiently adapt to the harsh environment of cattle barns. When processing the non-rigid cow face, the method of approximating the cow face to a rigid body is used to establish the cow face image data set., and the cattle face image data set is established. The Three Dimensional(3D) reconstruction method of cattle face uses a 3D image reconstruction method based on multiple perspectives. First, the scale-invariant feature transform algorithm is used to extract the image feature points. The fast library for approximate nearest neighbors algorithm is used to match feature points. The matching results are selected via random sampling consensus. Second, the structure of the motion method is used for the sparse reconstruction of point clouds, and the dense point cloud is then generated using the three-dimensional multi-view stereo vision algorithm. Finally, the Poisson surface reconstruction method is used for surface reconstruction. The results indicate that this method can effectively realize the three-dimensional reconstruction of cattle faces; the reconstructed images have obvious color, clear texture, and complete shape features.

A New 3D reconstruction method for pressure-sensitive paint measurements under limited optical access

Conventional perspective cameras have been used before to create 3D camera systems. However, these systems are hindered by the field of view of the cameras, which limit the angle of the scene that can be reconstructed. Augmenting these systems to larger reconstructed scene angle requires more cameras or increasing the field of view of the cameras. A 3D camera system is introduced which utilizes omnidirectional cameras. These cameras are capable of capturing the scene in 360 degrees, and only a limited number of cameras are required to reconstruct the entire surrounding. In this article the design, construction and calibration of omnidirectional cameras are discussed. The used camera model is presented which allows us to use an omnidirectional camera as a perspective camera, and the distortion model that is suitable for correcting precision problems of the assembly process.

The automatic reconstruction of photogrammetric mesh models often suffers from issues such as erroneous adhesion of different planes, local distortions, and the loss of edge details. This study aims to improve mesh quality by incorporating three-dimensional (3D) line segments and plane regions as geometric constraints during mesh refinement. Using previously obtained 3D line segments, depth, and normal information, accurate 3D plane regions are extracted to complete missing sections of the dense point cloud, with depth-based outlier removal applied. Correspondences between mesh model vertices and the 3D line segments and plane regions are established, followed by mesh refinement through an energy function that integrates photo-consistency, line and plane constraints, and regularization. Experiments conducted on four open-source datasets demonstrate that the proposed method substantially improves 3D plane region extraction and point cloud completion. The refined mesh model achieves a 24.30% reduction in the average distance related to 3D lines and a 15.75% reduction for 3D planes compared with the original model, confirming the effectiveness of incorporating 3D line and plane constraints in enhancing reconstruction accuracy.

The high quality three-dimensional (3D) presentation of computed tomography (CT) data has become a common technique since fast imaging systems and 3D software became widely available at the beginning of the 1980s. Unfortunately, the quality of the 3D images produced by standard 3D packages depends not only on several basic scanning parameters like voltage, amperage, slice, and imcrernent, but also on the primary reconstruction algorithm, image separation, and 3D reconstruction and shading method. Therefore, no standard 3D scanning method providing a high quality 3D display combined with an acceptable acquisition and reconstruction time has been established. New volume scanning methods such as the spiral CT offer additional methods to shorten the examination time. There are some problems for 3D imaging in the use of spiral CT. The aim of our study was to compare different scanning — and 3D — reconstruction methods in order to develop standard scanning sequences for routine 3D CT imaging.

Target recognition via 3D object reconstruction from image sequence and contour matching

ObjectivesThe primary objective was to determine which of the following angular insertion depth (AID) estimation methods showed the strongest correlation with the postoperative AID (AIDpostop) of a lateral wall (LW) electrode: the Escudé formula, based on the largest distance from the round window (RW) to the LW (distance A); the elliptic-circular approximation (ECA) method, based on both distance A and the perpendicular distance (distance B); and a three-dimensional (3D) reconstruction method. The secondary objective was to assess the impact of using the actual electrode insertion length on the evaluation of AID estimation methods.MethodsThis study included 45 cochleae implanted with the Advanced Bionics SlimJ electrode, with all 16 active electrode contacts positioned intracochlear. Distance A, distance B, and a 3D reconstruction along the LW were evaluated on preoperative magnetic resonance imaging using 3D Slicer. The AIDpostop and insertion length were assessed using cone-beam computed tomography. Two sets of estimated AID values were obtained: assuming a perfect 23-mm insertion length with the reference electrode at the RW (AID23mm); and using the actual insertion length (AIDinsertionlength). Spearman’s rank correlation coefficient was used to assess the association between the estimated AID values and AIDpostop.ResultsA moderately positive correlation was observed between AID23mm and AIDpostop for all three approaches, with no statistically significant differences among them (rs=0.410–0.579). When the analysis was repeated using AIDinsertionlength, the strength of the correlation with AIDpostop increased significantly for all three methods, with the 3D reconstruction method showing a stronger correlation (rs=0.952) than the Escudé formula (rs=0.734) or the ECA method (rs=0.787) (P<0.001).ConclusionThe 3D reconstruction approach has the potential to improve prediction of the AID for LW electrode arrays compared to spiral formulae based on 2D cochlear dimensions. Accurate assessment of the actual insertion length is important for the proper evaluation of AID estimation methods.

In modern medicine, 3D reconstruction plays an important role in improving the accuracy of disease detection and treatment especially in the process of robot-assisted surgery. Accordingly, the choice of 3D reconstruction method is of great importance, which provides real-time visualization. This study analyses and compares two types of methods in 3D medical image reconstruction and visualization: Surface Rendering and Volume Rendering. To be more specific, four algorithms including Contour Filter (CF), Marching Cubes (MC), Composite Volume Rendering (CVR) and Texture Mapping Hardware (TMH) were implemented and compared from the perspective of the rendering effect, speed and interactivity, in which the former two algorithms belong to the concept of Surface Rendering and the latter two algorithms are in the range of Volume Rendering. These algorithms were implemented by Microsoft Visual Studio with Visualization Toolkit (VTK) to reconstruct the 3D images of patients CT image data. The mechanism and realization of these algorithms were described, and the merits and shortcomings of these different rendering methods were compared. Based on the comparison results, the 3D reconstruction and rendering method can be determined according to the specific requirements of robot-assisted surgery.

In view of the existing underwater sensors cannot obtain a relative positional relationship between the target and the underwater vehicle, as well as, target positioning based on binocular vision method which has systematic errors needs more accurate calibration result. An approach for vision of a recognized underwater target is proposed in this paper. The first step is to process the original image with image enhancement and filtering, identify the target image based on HSV threshold. In the next step for target positioning, a three dimensional (3D) re-construction method based on block matching after binocular vision calibration and a 3D re-construction method with learning neural network are studied. Aiming at the problems of slow convergence rate and local optimum of traditional BP networks, several improvements of traditional BP networks are proposed. Finally, the experiment proves that the method can obtain high precision result and good real-time performance under sufficient stylebook data.

This paper proposes a new method for simultaneous 3D reconstruction and semantic segmentation for indoor scenes. Unlike existing methods that require recording a video using a color camera and/or a depth camera, our method only needs a small number of (e.g., 3~5) color images from uncalibrated sparse views, which significantly simplifies data acquisition and broadens applicable scenarios. To achieve promising 3D reconstruction from sparse views with limited overlap, our method first recovers the depth map and semantic information for each view, and then fuses the depth maps into a 3D scene. To this end, we design an iterative deep architecture, named IterNet, to estimate the depth map and semantic segmentation alternately. To obtain accurate alignment between views with limited overlap, we further propose a joint global and local registration method to reconstruct a 3D scene with semantic information. We also make available a new indoor synthetic dataset, containing photorealistic high-resolution RGB images, accurate depth maps and pixel-level semantic labels for thousands of complex layouts. Experimental results on public datasets and our dataset demonstrate that our method achieves more accurate depth estimation, smaller semantic segmentation errors, and better 3D reconstruction results over state-of-the-art methods.

This paper describes a method for omni-directional 3D reconstruction with a hyperbolic mirror and a regular camera. Omni-directional images are becoming increasingly famous in computer vision and other fields. Despite there some related research in omni-directional vision system applications during last two decades, however, omni-directional perception system have not ever been applied in crime scene reconstruction. The main advantages of the catadioptric vision system are robustness because of no needing mechanical moving units and single effective viewpoint constraint which allow the next feature detection. Firstly, we provide the transformation mapping method from the omni-directional images into the cylindrical panoramic images. Secondly, we use the SIFT Method to yield the scenario features. Subsequently, we tackle this matching and disparity estimation problem by proposing Markov random field (MRF) model. Finally with some experiment and we show the 3D clouds actually results.

Many methods have been proposed for the 3-dimensional (3D) reconstruction of coronaries arteries by combing information from two or more X-ray views of the coronary tree, since the 2D representation of coronary lesion using X-ray coronary angiographies is limited. The aim of this study is to present a new semi-automated method for the accurate 3D reconstruction of coronary arterial bifurcations using X-ray coronary angiographic views (CA). X-ray angiography was acquired from seven patients, both pre and post angioplasty procedure, and their data were used for the 3D reconstruction methodology. The proposed approach consists of 3 steps. Initially, the 2D lumen borders and centerlines are detected. Then the 3D bifurcation path is extracted and the 3D lumen borders are reconstructed around the 3D bifurcation path and finally, the main and side segments are intersected in order to produce the finally model of the bifurcated artery. Considering the X-ray angiography as the gold standard, we validated the proposed method based on the 2D versus the 3D bifurcation segment model. More specifically, in the current dataset our results indicate excellent correlation with the 2D angiography: r = 0.98, p < 0.001; r = 0.97, p < 0.001; r = 0.94, p < 0.001 and r = 0.88, p < 0.001 for the Reference Lumen Diameter (RLD), Minimal Lumen Diameter (MLD), Degree of Stenosis (DS) and Lesion Length (LL), respectively. Moreover the mean values of the Hausdorff Distance and the Dice correlation between the 2D annotated borders and the forwardly projected borders are 0.3031 mm and 91% for the first CA while 0.3103 and 92% for the second one.

Three-dimensional (3D) reconstruction technology has found widespread applications across various domains, including intelligent driving and underwater exploration. But the existing imaging systems and methods still have deficiencies in terms of reconstruction accuracy, detection distance and system volume. Herein, this paper presents a three-dimensional detection and reconstruction method based on a compact Risley-prism 3D imaging system that achieves multi-viewpoint imaging by rotating the Risley prism to adjust the camera's optical axis. A refractive camera model that integrates the pinhole camera model with the vector form of Snell's law is established to precisely describe beam trajectory. A forward projection method suitable for refractive interfaces is developed based on Fermat's principle, and the influence of systematic errors on the reconstruction is analyzed in detail through simulation. Furthermore, a new 3D reconstruction method combining error calibration based on the optimization iteration is introduced to avoid the influence of error and improve reconstruction quality. Experimental results demonstrate that the proposed approach markedly enhances 3D reconstruction accuracy, reducing the Normalized Root Mean Square Error (NRMSE) from 0.9076 to 0.0207.