Fast Stereo 3D Imaging Based on Random Speckle Projection and Its FPGA Implementation

In this paper, we propose a high-speed 3D shape measurement technique based on the optimized composite fringe patterns and stereo-assisted structured light system. Stereo phase unwrapping, as a new-fashioned method for absolute phase retrieval based on the multi-view geometric constraints, can eliminate the phase ambiguities and obtain a continuous phase map without projecting any additional patterns. However, in order to ensure the stability of phase unwrapping, the period of fringe is generally around 20, which limits the accuracy of 3D measurement. To solve this problem, we develop an optimized method for designing the composite pattern, in which the speckle pattern is embedded into the conventional 4-step phase-shifting fringe patterns without compromising the fringe modulation, and thus the phase measurement accuracy. We also present a simple and effective evaluation criterion for the correlation quality of the designed speckle pattern in order to improve the matching accuracy significantly. When the embedded speckle pattern is demodulated, the periodic ambiguities in the wrapped phase can be eliminated by combining the adaptive window image correlation with geometry constraint. Finally, some mismatched regions are further corrected based on the proposed regional diffusion compensation technique (RDC). These proposed techniques constitute a complete computational framework that allows to effectively recover an accurate, unambiguous, and distortion-free 3D point cloud with only 4 projected patterns. Experimental results verify that our method can achieve high-speed, high-accuracy, robust 3D shape measurement with dense (64-period) fringe patterns at 5000 frames per second.

In this paper, a three-dimensional (3D) shape measurement method based on structured light field imaging is proposed, which contributes to the biomedical imaging. Generally, light field imaging is challenging to accomplish the 3D shape measurement accurately, as the slope estimation method based on radiance consistency is inaccurate. Taking into consideration the special modulation of structured light field, we utilize the phase information to substitute the phase consistency for the radiance consistency in epi-polar image (EPI) at first. Therefore, the 3D coordinates are derived after light field calibration, but the results are coarse due to slope estimation error and need to be corrected. Furthermore, the 3D coordinates refinement is performed based on relationship between the structured light field image and DMD image of the projector, which allows to improve the performance of the 3D shape measurement. The necessary light field camera calibration is described to generalize its application. Subsequently, the effectiveness of the proposed method is demonstrated with a sculpture and compared to the results of a conventional PMP system.

High-speed 3D shape measurement with structured light methods: A review

By utilizing 1-bit binary fringe patterns instead of conventional 8-bit sinusoidal patterns, binary defocusing techniques have been successfully applied for high-speed 3D shape measurement. However, simultaneously achieving high accuracy and high speed remains challenging. To overcome this limitation, we propose a high-efficiency and robust binary fringe optimization method for superfast 3D shape measurement, which consists of 1D optimization and 2D modulation. Specifically, for 1D optimization, the three-level OPWM technique is introduced for high-order harmonics elimination, and an optimization framework is presented for generating the 'best' three-level OPWM pattern especially for large fringe periods. For 2D modulation, a single-pattern three-level OPWM strategy is proposed by utilizing all the dimensions for intensity modulation to decrease the required projection patterns. Thus, the proposed method essentially belongs to the 2D modulation technique, yet iterative optimization is carried out along one dimension, which drastically improves the computational efficiency while ensuring high accuracy. With only one set of optimized patterns, both simulations and experiments demonstrate that high-quality phase maps can be consistently generated for a wide range of fringe periods (e.g., from 18 to 1140 pixels) and different amounts of defocusing, and it can achieve superfast and high-accuracy 3D shape measurement.

Binary coding methods have been widely used for three-dimensional (3D) shape measurement. However, traditional binary coding methods always require a sequence of binary patterns to encode the fringe order. This paper presents a spatial-temporal binary coding (STBC) method for 3D shape measurement with three binary patterns, including a spatial pattern and two temporal patterns. Specifically, the spatial pattern is used to encode a 4-bit spatial code, and the temporal patterns are used to encode the 2-bit temporal code. By combining the spatial code and the temporal code, 6-bit code words can be obtained that distinguish 26=64 fringe periods of sinusoidal phase-shifting patterns. A robust dual-region labeling decoding scheme is also developed to extract the 6-bit code word for fringe order determination, and then absolute-phase unwrapping can be carried out. Experiments have been conducted, and their results confirm that the STBC method can reliably reconstruct the 3D shapes of measured objects with only three binary patterns.

The phase-coding method is useful in 3D shape measurement due to its low sensitivity to surface contrast and camera noise. However, as the number of fringe periods increases, the wrapped phase recovered from the sinusoidal phase-shifting patterns is always misaligned with the fringe order calculated by the stair-coding patterns. To solve this, an improved dual-frequency phase-coding fringe projection method is designed to realize absolute phase retrieval. Specifically, the auxiliary fringe order determined by phase-shifting is used to get the absolute phase, which can eliminate fringe-order errors. Moreover, to increase speed and the number of codewords, this paper presents a scheme that only requires seven patterns, including a uniform gray image, two high-frequency sinusoidal fringe patterns, two low-frequency sinusoidal fringe patterns and two phase-coding fringe patterns. Therefore, this method requires fewer patterns to realize high-resolution measurement than existing methods. The success of the proposed method can be proved by experiments.

Traditional optical 3D shape measurement methods, such as light stripe triangulation, binary coding, and fringe projection, cannot acquire complete and correct 3D measurement results in the presence of interreflections. In this research, a 3D shape measurement method in the presence of interreflections based on light stripe triangulation is presented. The wrong measurement results caused by interreflections are excluded by the geometric constraints introduced by an additional camera. Each 3D point reconstructed by light stripe triangulation is projected onto the image plane of the additional camera to determine whether the 3D point is correct measurement result. Experimental results demonstrate that the proposed method can measure 3D shape in the presence of interreflections.

In this paper, we propose a high-speed 3-D shape measurement technique based on composite structured-light patterns and a multi-view system. Benefiting from the multi-view system, stereo phase unwrapping, as a novel method for the phase unwrapping algorithm, can eliminate the phase ambiguities and obtain absolute phase map without projecting any additional patterns. However, in order to ensure the stability of phase unwrapping, the period of fringe is generally around 20, which leads to the limited accuracy of 3D measurement. To solve the precision-limited problem reasonably, we mathematically developed an optimized design method of the composite pattern. By skillfully embedding speckles without compromising the phase measurement accuracy, we can realize phase unwrapping with high-frequency fringes. In addition, a computational framework will be provided to further achieve the robust and high-performance 3D acquisition. It is demonstrated by several experiments that our method can achieve a high-speed, dense, and accurate 3-D shape measurement with 64-period fringe patterns at 5000 frames per second.

Digital speckle correlation (DSC) solves the problem of searching corresponding points between two images, and it shows great application potential in pattern-projection based fast 3D shape measurement, because only one shot is enough to retrieve the 3D structure. As DSC relies on analyzing the spatial intensity distribution of a subset in image with a given point, it is likely to get false correspondences in low quality DSC area such as the background, because the searching range is hard to locate. So it is still hard to use DSC to realize fast 3D shape measurement. To solve this problem, the gray standard deviation of the subset is designed to recognize and remove the low-quality DSC area, and the principle of epipolar geometry and disparity constraint are utilized to determine the searching range, so the correspondences can be obtained. Moreover, in order to enhance the robustness of this method, a connected region method based on neighboring pixels possessing similar disparity is proposed to remove mismatched points after establishing initial disparity map by correspondences. Once the disparity map is obtained 3D structure can be retrieved based on the triangulation principle. The experiment on reconstructing Gorky plaster statue is performed, verifying that the proposed method can substantially reduce mismatched points and achieve robust single frame 3D shape measurement.

Superfast 3D absolute shape measurement using five binary patterns

Conventional three-dimensional (3D) shape measurement methods are typically generic to all types of objects. Yet, for many measurement conditions, such a level of generality is inessential when having the preknowledge of the object geometry. This paper introduces a novel adaptive algorithm for absolute 3D shape measurement with the assistance of the object computer-aided-design (CAD) model. The proposed algorithm includes the following major steps: (1)export the 3D point cloud data from the CAD model; (2)transform the CAD model into the camera perspective; (3)obtain a wrapped phase map from three phase-shifted fringe images; and (4)retrieve absolute phase and 3D geometry assisted by the CAD model. We demonstrate that if object CAD models are available, such an algorithm is efficient in recovering absolute 3D geometries of both simple and complex objects with only three phase-shifted fringe images.

Fourier transform profilometry (FTP) is one of the frequently adopted three-dimensional (3D) shape measurement methods due to its ability to recover single-shot 3D shapes, yet it is challenging to retrieve the absolute phase map solely from one single grayscale fringe image. This paper presents a computational framework that overcomes this limitation of FTP with digital fringe projection (DFP). By using geometric constraints, an absolute phase map can be retrieved point-by-point from one single grayscale fringe image. Experiments demonstrate the success of our proposed framework with single-shot absolute 3D shape measurement capability.

Fringe projection profilometry (FPP) is a three-dimensional (3D) shape measurement method that involves projecting fringe patterns onto the object. A phase map retrieved from these fringe images is used for reconstructing the 3D surface of the object. Fourier transform and phase-shifting are two of the widely used fringe analysis techniques for performing 3D shape measurement using FPP. Fourier transform profilometry (FTP) has the advantage of performing high-speed measurement due to its single-shot nature. However, the reconstructed 3D surface has artifacts and high noise, specifically on the edges. On the other hand, phase-shifting profilometry (PSP) has the advantage of higher accuracy (relatively less noise level) but compromises on measurement speed. In this research, we propose a deep learning method to enhance the quality of the FTP phase maps using an efficient deep learning model called phase map enhancement net (PMENet). PMENet takes an FTP phase map as input and predicts a high-quality phase map in a supervised manner by using the phase maps obtained from 18-step PSP as ground truth. The training dataset was generated using a virtual FPP system. Validations were conducted on both the simulated data (generated by virtual FPP system) and the real-world data. The experimental results demonstrate that the trained neural network model can successfully improve the quality of the 3D geometric reconstruction with FTP and reduced the mean and root-mean-square error by 66% and 43%, respectively.

본 연구에서는 자동 비파괴검사 구현을 위한 3차원 형상 측정 및 경로 설계 방법에 대하여 연구하였다. 2차원 형상 측정 센서에 회전을 추가하여 측정 대상으로부터 취득한 point cloud 데이터는 MATLAB을 이용하여 메시를 생성하여 가시화를 하였다. 가시화된 데이터는 다시 균일한 크기의 메시로 만들고자 메시 재형성을 거쳐 검사 지점과 검사 방향을 산출하였다. 이렇게 산출한 데이터를 이용한 검사 경로는 지그재그로 흔들리는 형상을 가져 비효율적인 경로이므로, 경로 최적화를 진행하였다. 최적화 방법은 검사 방향을 유지하고 검사 위치를 동일한 높이에 배치함으로써 상하로 흔들리는 문제를 제거하였으며 한 층의 검사가 끝나면 초기 위치로 되돌아가는 것이 아니라 바로 위층으로 이동하여 반대 방향으로 검사를 진행하도록 하여 손실되는 경로를 최소화하였다. 이러한 연구를 통하여 형상 정보를 이용한 자동화 비파괴검사 시스템에 대하여 큰 가능성을 확인하였다.

A widely used method in high-speed 3D shape measurement, color-code fringe projection requires the projection of only one image. In traditional color-code method, the projected fringe is compounded by sinusoidal fringes, grayscale value distribution ranges from 0–255, and projection speed is limited to 120 frames/s. Consequently, measurement speed is restricted, and a nonlinear gamma of the projector exists, as well as high harmonics, which have a great influence on measurement accuracy. Binary color-code fringe defocused projection is proposed to solve the abovementioned problems. With the proposed method, projection speed can switch to tens of K frames/s because it only has two grayscale values (0 and 255). A standard sinusoidal color-code fringe can be generated by properly defocusing the binary color-code fringe, thereby overcoming the influence of nonlinear gamma and ultimately improving measurement accuracy. Experiment results verify the feasibility and superiority of the proposed method.