A Fully Automatic 3D Reconstruction Method Based on Images

In this paper, we address a problem of reconstruction of three-dimensional scenes from images taken by cameras, with varying parameters, from different views. This method is based on the projection of 3D points in the image planes. The relationships between the matches and the camera parameters are used to formulate a nonlinear equation system. This system is transformed into an objective function, which is minimized by a genetic algorithm to estimate the intrinsic and extrinsic camera parameters. Finally, the coordinates of 3D points of the scene are obtained by solving a linear equation system. The experiments on synthetic and real data show the quality of this work and the good obtained results.

In this paper, we will study a new approach of reconstruction of three-dimensional scenes from an auto calibration method of camera characterized by variable parameters. Indeed, obtaining the 3D scene is based on the Euclidean reconstruction of the interest points detected and matched between pair of images. The relationship between the matches and camera parameters is used to formulate a nonlinear equation system. This system is transformed into a nonlinear cost function, which will be minimized to determine the intrinsic and extrinsic camera parameters and subsequently estimate the projection matrices. Finally, the coordinates of the 3D points of the scene are obtained by solving a linear equation system. The results of the experiments show the strengths of this contribution in terms of precision and convergence.

Photogrammetry-based methods are traditionally used to estimate the geometrical parameters using optical images. These methods employ specific equipment, computationally sophisticated and expensive algorithms, and utilize projective geometry to reconstruct real-life scenes up to a scale. In this work, we used a computationally less-expensive method for sparse reconstruction to estimate the planar angles using two-view geometry and linear algorithms from non-orthogonal images acquired by a smartphone camera. First, intrinsic camera parameters were determined. Next, scale-invariant feature transform was used to identify the correspondence points from each pair of images. Epipolar constraint was applied on all these points to determine the essential matrix using the eight-point algorithm. Thereafter, extrinsic camera parameters were estimated from the essential matrix and combined with the intrinsic matrix to get the camera projection matrix. Finally, linear triangulation was used to get the sparse point cloud representing the scene. Planar angles were estimated by backprojecting the chosen image points and applying simple vector algebra on the obtained 3D points. The method was successful in estimating the planar angles in less than 10s on non-curved edges with an average error of 3% by using only ten images. Given the simplicity of methods used, this technique can be integrated into a smartphone for on-site measurements as well as large deformations.

In this paper, we present a self-calibration strategy to estimate camera intrinsic and extrinsic parameters using the scale-invariant feature transform (SIFT). The accuracy of the estimated parameters depends on how reliably a set of image correspondences is established. The SIFT employed in the self-calibration algorithms plays an important role in accurate estimation of camera parameters, because of its robustness to changes in viewing conditions. Under the assumption that the camera intrinsic parameters are constant, experimental results show that the SIFT-based approach using two images yields more competitive results than the existing Harris corner detector-based approach using two images.

In this paper we present an easy method for multiple camera calibration with common field of view only from vertical lines. The locations of the vertical lines are known in advance. Compared to other calibration objects, the vertical lines have some good properties, since they can be easily built and can be visible by cameras in any direction simultaneously. Given 5 fixed vertical lines, an image containing them taken by a camera may provide 2 constraints in the intrinsic parameters of the camera, and extrinsic parameters can then be recovered. The calibration procedure consists of three main steps: Firstly, the image is rectified by a homography, which makes the projections of vertical lines parallel to u-axis in the rectified image. Secondly, for any vertical scan line in the rectified image, if we consider the scan line is taken by a virtual 1D camera, then we can calibrate the 1D camera. Finally, the intrinsic parameters of the original camera can be determined from the intrinsic parameters of the virtual 1D camera. By evaluating on both simulated and real data we demonstrate that our method is efficient and robust.

Stereo vision is the key in the visual measurement, robot vision, and autonomous navigation. Before performing the system of stereo vision, it needs to calibrate the intrinsic parameters for each camera and the external parameters of the system. In engineering, the intrinsic parameters remain unchanged after calibrating cameras, and the positional relationship between the cameras could be changed because of vibration, knocks and pressures in the vicinity of the railway or motor workshops. Especially for large baselines, even minute changes in translation or rotation can affect the epipolar geometry and scene triangulation to such a degree that visual system becomes disabled. A technology including both real-time examination and on-line recalibration for the external parameters of stereo system becomes particularly important. This paper presents an on-line method for checking and recalibrating the positional relationship between stereo cameras. In epipolar geometry, the external parameters of cameras can be obtained by factorization of the fundamental matrix. Thus, it offers a method to calculate the external camera parameters without any special targets. If the intrinsic camera parameters are known, the external parameters of system can be calculated via a number of random matched points. The process is: (i) estimating the fundamental matrix via the feature point correspondences; (ii) computing the essential matrix from the fundamental matrix; (iii) obtaining the external parameters by decomposition of the essential matrix. In the step of computing the fundamental matrix, the traditional methods are sensitive to noise and cannot ensure the estimation accuracy. We consider the feature distribution situation in the actual scene images and introduce a regional weighted normalization algorithm to improve accuracy of the fundamental matrix estimation. In contrast to traditional algorithms, experiments on simulated data prove that the method improves estimation robustness and accuracy of the fundamental matrix. Finally, we take an experiment for computing the relationship of a pair of stereo cameras to demonstrate accurate performance of the algorithm.

Depth perception is crucial for many applications including robotics, UAV and autonomous driving. The visual sense, as well as cameras, map the 3D world on a 2D representation, losing the dimension representing depth. A way to recover 3D information from 2D images is to record and join data from multiple viewpoints. In case of a stereo setup, 4D data is gained. Existing methods to recover 3D information are computationally expensive. We propose a new, more intuitive method to recover 3D objects out of event-based stereo data, by using a Self-Organizing Map to solve the correspondence problem and establish a structure similar to a voxel grid. Our approach, as it is also computationally expensive, copes with performance issues by massive parallelization. Furthermore, the relatively small voxel grid makes this a memory friendly solution. This technique is very powerful as it does not need any prior knowledge of extrinsic and intrinsic camera parameters. Instead, those parameters and also the lens distortion are learned implicitly. Not only do we not require a parallel camera setup, as many existing methods, we do not even need any information about the alignment at all. We evaluated our method in a qualitative analysis and finding image correspondences.

A new visual measurement method is proposed to estimate three-dimensional (3D) position of the object on the floor based on a single camera. The camera fixed on a robot is in an inclined position with respect to the floor. A measurement model with the camera's extrinsic parameters such as the height and pitch angle is described. Single image of a chessboard pattern placed on the floor is enough to calibrate the camera's extrinsic parameters after the camera's intrinsic parameters are calibrated. Then the position of object on the floor can be computed with the measurement model. Furthermore, the height of object can be calculated with the paired-points in the vertical line sharing the same position on the floor. Compared to the conventional method used to estimate the positions on the plane, this method can obtain the 3D positions. The indoor experiment testifies the accuracy and validity of the proposed method.

The process of solving camera intrinsic and extrinsic parameters has been presented by using geometric techniques. In contrast with traditional method, which is a pure algebra process, the geometric technique accomplishes the camera parameters more intuitively and easily. If the perspective projection matrix is given, according to the geometric meaning of perspective projection camera model, the coordinates of optical centre can be directly estimated in the world coordinate system. Then rotation matrix and translation vector can be obtained from the matrix. Finally, we can derive camera intrinsic parameters. During the process of derivation, for any given 3 by 4 matrix, the constraint conditions have been discussed in another way when the matrix could be described as a perspective projection transformation. Although it has the same result as pure algebra progress, it's of great significance to understand the geometric meaning among the intrinsic and extrinsic camera parameters

Real-time 3D reconstruction of static scenes can be achieved based on the RGB-D image sequence fusion. It is a popular practice to divide the space into uniform voxels and use a truncated signed distance function to represent surface information. In order to represent a scene of large scale, the voxel hash algorithm which stores voxels compressively can be used, but most of the conventional methods do not consider the complexity and roughness of the object surface in the scene, so the scene is represented with a uniform resolution. It somewhat limits the range of scene representation and the speed of real-time reconstruction. In this paper, a large-scale scene reconstruction algorithm based on voxel hashing storage with LOD representation is proposed. The main contributions include the following two aspects: (1) By preprocessing the depth image with smooth filtering, which ensures the accuracy of the data, it can effectively reduce the distortion caused by the sensor itself and violent motion and provide better support for the stages of voxel hashing, model rendering, and frame-to-model camera position tracking. (2) The 3D reconstruction with LOD representation is realized. We take the view distance and the roughness of the model surface as criteria to control the adaptive division and representation of spatial voxel blocks. Finally, we carried out qualitative and quantitative evaluations of the algorithm, and confirmed that the algorithm can achieve real-time reconstruction with different levels of detail in the commercial graphics processing hardware environment, and achieve a good fusion effect in large-scale scenes.

An improved outdoor camera calibration method based on Extended Kalman Filter (EKF) is proposed in this paper. GPS coordinate system is introduced to establish world coordinate system in the calibration procedure. According to the characteristics of GPS coordinate converted to plane coordinate and the camera calibration model, we establish the mapping relations of reference points between image plane coordinate and the world coordinate. Further, we consider the world coordinate error, set the world coordinates as the measure quantities, use the method of EKF to get the optimal estimation of intrinsic and extrinsic camera parameters, in EKF the camera's intrinsic and extrinsic parameters are set as state variables, the transformation equations of image coordinate to the world coordinate are set as the measurement equations, and we let the observed feature points' image coordinates and corresponding world coordinates as the input of the filter. Experimental results confirmed that the proposed method with high precision. Index Terms - camera calibration, GPS coordinate, EKF, world coordinate error

Accurately estimating the 3D position of underwater objects is of great interest when doing research on marine animals. An inherent problem of 3D reconstruction of underwater positions is the presence of refraction which invalidates the assumption of a single viewpoint. Three ways of performing 3D reconstruction on underwater objects are compared in this work: an approach relying solely on in-air camera calibration, an approach with the camera calibration performed under water and an approach based on ray tracing with Snell's law. As expected, the in-air camera calibration showed to be the most inaccurate as it does not take refraction into account. The precision of the estimated 3D positions based on the underwater camera calibration and the ray tracing based approach were, on the other hand, almost identical. However, the ray tracing based approach is found to be advantageous as it is far more flexible in terms of the calibration procedure due to the decoupling of the intrinsic and extrinsic camera parameters.

3D reconstruction of objects via Shape from Motion (SFM) has made great strides recently. Utilizing images from a variety of poses, objects can be reconstructed in 3D without knowing a priori the camera pose. These feature points can then be bundled together to create large scale scene reconstructions automatically. A shortcoming of current methods of SFM reconstruction is in dealing with specular or flat low feature surfaces. The inability of SFM to handle these places creates holes in a 3D reconstruction. This can cause problems when the 3D reconstruction is used for proximity detection and collision avoidance by a space vehicle working around another space vehicle. As such, we would like the automatic ability to recognize when a hole in a 3D reconstruction is in fact not a hole, but is a place where reconstruction has failed. Once we know about such a location, methods can be used to try to either more vigorously fill in that region or to instruct a space vehicle to proceed with more caution around that area. Detecting such areas in earth orbiting objects is non-trivial since we need to parse out complex vehicle features from complex earth features, particularly when the observing vehicle is overhead the target vehicle. To do this, we have created a Space Object Classifier and Segmenter (SOCS) hole finder. The general principle we use is to classify image features into three categories (earth, man-made, space). Classified regions are then clustered into probabilistic regions which can then be segmented out. Our categorization method uses an augmentation of a state of the art bag of visual words method for object categorization. This method works by first extracting PHOW (dense SIFT like) features which are computed over an image and then quantized via KD Tree. The quantization results are then binned into histograms and results classified by the PEGASOS support vector machine solver. This gives a probability that a patch in the image corresponds to one of three categories: Earth, Man-Made or Space. Here man-made refers to artificial objects in space. To categorize a whole image, a common sliding window protocol is used. Here we utilized 90 high resolution images from space shuttle servicing missions of the international space station. We extracted 9000 128x128 patches from the images, then we hand sorted them into one of three categories. We then trained our categorizer on a subset of 6000 patches. Testing on 3000 testing patches yielded 96.8% accuracy. This is basically good enough because detection returns a probabilistic score (e.g. p of man-made). Detections can then be spatially pooled to smooth out statistical blips. Spatial pooling can be done by creating a three channel (dimension) image where each channel is the probability of each of the three classes at that location in the image. The probability image can then be segmented or co-segmented with the visible image using a classical segmentation method such as Mean Shift. This yields contiguous regions of classified image. Holes can be detected when SFM does not fill in a region segmented as man-made. Results are shown of the SOCS implementation finding and segmenting man-made objects in pictures containing space vehicles very different from the training set such as Skylab, the Hubble space telescope or the Death Star.

With the increasing use of image-based 3D reconstruction in medical procedures, accurate scene reconstruction plays a crucial role in surgical navigation and assisted treatment. However, the monotonous colors, limited image features, and obvious brightness fluctuations of thoracoscopic scenes make the feature point matching process, on which traditional 3D reconstruction methods rely, unstable and unreliable. It brings a great challenge to accurate 3D reconstruction. In this study, a new method for implicit 3D reconstruction of monocular thoracoscopic scenes is proposed. The method combines a pre-trained metric depth estimation model with neural radiation field (NeRF) technique and uses dense SLAM to accurately compute the camera pose. To ensure the accuracy of the depth values and the structural consistency of the reconstructed scene, depth and normal constraints are added to the original color constraints of the NeRF network to achieve high-quality scene reconstruction results. We conducted experiments on the SCARED dataset and the clinical dataset. After comparing with other methods, the depth estimation accuracy and point cloud reconstruction quality of this paper outperform the existing methods. The method in this paper can provide more accurate 3D reconstruction of complex thoracic surgical scenes, which can significantly improve the accuracy and therapeutic efficacy of surgical navigation.

A multi‐camera rig calibration algorithm based on a double sided planar target is proposed. Due to their inherently simple realisation, low cost and accuracy, planar calibration targets came out as one of the most largely adopted calibration tools both for intrinsic and extrinsic camera parameters. However, concerning the estimation of extrinsic parameters, one of the major drawbacks of these targets is their requirement for distinct target visibility from both cameras. This prevents many configurations from being adopted where, e.g. two cameras are facing each other. An inexpensive solution could be based on printing/pasting a planar pattern on both target sides, however, the relative misalignment between the patterns on the two sides and the target thickness could be unknown. The authors propose a solution where double‐sided target displacement error is estimated together with the extrinsic parameters allowing the reuse of all the available planar calibration tools in less constrained configurations. To assess their approach the authors tested the system in two scenarios, one using two professional 4K cameras and one using two smartphones.