PMENet: phase map enhancement for Fourier transform profilometry using deep learning

Three-dimensional (3D) shape measurement of fringe projection profilometry (FPP) with high speed and accuracy is required in many applications. The measurement speed of FPP system is mainly limited by the low frame rate of pattern projection. Although defocusing binary patterns can be used to increase the measurement speed greatly, it will decrease the measurement depth. To this end, a novel triangular pattern generation method is proposed in this paper. By designing and projecting binary strip patterns with focused projector, images of triangular patterns can be then acquired with high frame rate. Fast and accurate 3D shape measurement can be finally obtained by using phase-shifting triangular patterns. Compared with the traditional sinusoidal patterns, the proposed method can improve the projection rate more than 4 times. Moreover, the proposed method will not decrease the measurement depth. Experimental results are given to confirm the validity of the proposed method.

Fourier transform profilometry (FTP) is one of the frequently adopted three-dimensional (3D) shape measurement methods due to its ability to recover single-shot 3D shapes, yet it is challenging to retrieve the absolute phase map solely from one single grayscale fringe image. This paper presents a computational framework that overcomes this limitation of FTP with digital fringe projection (DFP). By using geometric constraints, an absolute phase map can be retrieved point-by-point from one single grayscale fringe image. Experiments demonstrate the success of our proposed framework with single-shot absolute 3D shape measurement capability.

In this paper, a three-dimensional (3D) shape measurement method based on structured light field imaging is proposed, which contributes to the biomedical imaging. Generally, light field imaging is challenging to accomplish the 3D shape measurement accurately, as the slope estimation method based on radiance consistency is inaccurate. Taking into consideration the special modulation of structured light field, we utilize the phase information to substitute the phase consistency for the radiance consistency in epi-polar image (EPI) at first. Therefore, the 3D coordinates are derived after light field calibration, but the results are coarse due to slope estimation error and need to be corrected. Furthermore, the 3D coordinates refinement is performed based on relationship between the structured light field image and DMD image of the projector, which allows to improve the performance of the 3D shape measurement. The necessary light field camera calibration is described to generalize its application. Subsequently, the effectiveness of the proposed method is demonstrated with a sculpture and compared to the results of a conventional PMP system.

Conventional three-dimensional (3D) shape measurement methods are typically generic to all types of objects. Yet, for many measurement conditions, such a level of generality is inessential when having the preknowledge of the object geometry. This paper introduces a novel adaptive algorithm for absolute 3D shape measurement with the assistance of the object computer-aided-design (CAD) model. The proposed algorithm includes the following major steps: (1)export the 3D point cloud data from the CAD model; (2)transform the CAD model into the camera perspective; (3)obtain a wrapped phase map from three phase-shifted fringe images; and (4)retrieve absolute phase and 3D geometry assisted by the CAD model. We demonstrate that if object CAD models are available, such an algorithm is efficient in recovering absolute 3D geometries of both simple and complex objects with only three phase-shifted fringe images.

Blade is the key component of the aero-engine. In generally, it requires precise size and accurate shape, so the three-dimensional shape measurement of the blade is very important. Fringe projection profilometry is generally used to measure the 3D shape of an object, because of the advantages of non-contact operation, full-field acquisition, high resolution, and fast data processing. In this paper, 3D shape measurement method based on surface structured light that combination of gray-code and phase-shift projection is proposed. The wrapped phase of the blade is got by the four-step phase-shift method. In the projection process of the gray-code, due to the ambient light, different surface reflectance and surface physical discontinuous, the edge of gray-code becomes smooth, leading to decode error, so that the gray-code pattern must be binaried before decode. Through the accurate binarization of the blade gray-code pattern, the decoding cycle of the blade is achieved, and the unwrapped phase is achieved by phase unwrapping. Then the unwrapped phase difference between the blade and reference plane is got, the height of the blade can be obtained by the relationship between the phase difference and the height. The experimental and simulation results show that the proposed method can achieve a high precision, high speed and low cost 3D shape measurement of the blade. The measurement accuracy reaches 0.03 mm. The proposed method extends the reliability and practicality of the fringe projection profilometry.

Binary coding methods have been widely used for three-dimensional (3D) shape measurement. However, traditional binary coding methods always require a sequence of binary patterns to encode the fringe order. This paper presents a spatial-temporal binary coding (STBC) method for 3D shape measurement with three binary patterns, including a spatial pattern and two temporal patterns. Specifically, the spatial pattern is used to encode a 4-bit spatial code, and the temporal patterns are used to encode the 2-bit temporal code. By combining the spatial code and the temporal code, 6-bit code words can be obtained that distinguish 26=64 fringe periods of sinusoidal phase-shifting patterns. A robust dual-region labeling decoding scheme is also developed to extract the 6-bit code word for fringe order determination, and then absolute-phase unwrapping can be carried out. Experiments have been conducted, and their results confirm that the STBC method can reliably reconstruct the 3D shapes of measured objects with only three binary patterns.

Color-coded fringe projection profilometry (FPP) is one of the most important single-frame three-dimensional (3D) shape measurement techniques. Transverse chromatic aberration (TCA) in a color-coded FPP system yields imaging deviations, which will result in 3D shape measurement error. In this work, a TCA compensation method for color-coded FPP is proposed. The proposed method establishes the relationship among phase, pixel deviation, and imaging position for decreasing the 3D shape measurement error introduced by TCA. Compared to existing methods that use the relationship among depth, pixel deviation, and imaging position or polynomial fitting for each pixel, the proposed method offers the merits of precise and simple implementation, since it voids the cumulative error in-depth calculation and removes the time-consuming process of polynomial fitting for each pixel. For validation, a color-coded FPP system is established and calibrated. Two verification experiments, including a translation test and 3D shape measurement of a regular spherical object, are conducted with the calibrated system. Experimental results demonstrate that the proposed TCA compensation method is capable of effectively reducing the 3D shape measurement error caused by TCA.

The phase-coding method has been widely used for 3D shape measurement, which uses sinusoidal phase-shifting patterns to recover the wrapped phase and the stair phase-coding patterns to determine the fringe order. However, due to random noises and image blurring, the fringe order is always misaligned with the wrapped phase, which will lead to fringe order errors. This paper presents an enhanced phase-coding method to address this misalignment problem by using half-period codewords, in which each codeword is aligned to the half-period of the sinusoidal patterns. Then, two complementary fringe orders with half-period dislocation can be calculated, which can effectively eliminate the fringe order errors. To extend the coding range of stair phase, this paper further develops a computational scheme based on the geometric constraint method. Simulations and experiments have been carried out, and their results confirm that the enhanced method can reliably recover the 3D shape of the measured objects.

To solve the challenging issue of uneven reflectance of high-fidelity surface, this paper proposes a three-dimensional (3D) shape measurement method based on surface adaptation complementary colorimetry-encoded fringe projection profilometry. A complete complementary hue extraction method simulates the retina stimulation to establish the complementary conversion relationship between the projected and collected colors of the measurement system. On this basis, a surface reflectance adaptation fringe pattern encoding strategy is proposed to measure the high-fidelity surfaces. The experiments demonstrate that the proposed method is capable of efficiently balancing and adjusting the reflected light intensity of each fidelity surface, thereby suppressing the ambiguity reconstruction texture. The established complementary colorimetry relationship also improves the light wavelength discrimination, saturation avoidance, measurement accuracy and efficiency.

In-situ 3-D shape measurements with submicron shape uncertainty of fast rotating objects in a cutting lathe are expected, which can be achieved by simultaneous distance and velocity measurements. Conventional tactile methods, coordinate measurement machines, only support ex-situ measurements. Optical measurement techniques such as triangulation and conoscopic holography offer only the distance, so that the absolute diameter cannot be retrieved directly. In comparison, laser Doppler distance sensors (P-LDD sensor) enable simultaneous and in-situ distance and velocity measurements for monitoring the cutting process in a lathe. In order to achieve shape measurement uncertainties below 1 μm, a P-LDD sensor with a dual camera based scattered light detection has been investigated. Coherent fiber bundles (CFB) are employed to forward the scattered light towards cameras. This enables a compact and passive sensor head in the future. Compared with a photo detector based sensor, the dual camera based sensor allows to decrease the measurement uncertainty by the order of one magnitude. As a result, the total shape uncertainty of absolute 3-D shape measurements can be reduced to about 100 nm.

In this paper, a whole-field and high-speed optical three-dimensional (3D) shape measurement for the vibration pattern of a drumhead based on Fourier Transform Profilometry (FTP) is presented and verified by experiment. A projected sinusoidal fringe pattern on the surface of measured drumhead is dynamically deformed with the vibration of the membrane and grabbed by a high-speed camera. The 3D shape deformation of the drumhead at each sampling instant can be recovered from demodulating the phase of this obtained sequence of fringe patterns. The vibration of the membrane of a Chinese drum has been measured with a high-speed sampling rate (1,000 fps) and a standard deviation (0.075 mm). This method should be helpful in studying the acoustic characteristic and the manufacture technique of the percussion instrument.

By utilizing 1-bit binary fringe patterns instead of conventional 8-bit sinusoidal patterns, binary defocusing techniques have been successfully applied for high-speed 3D shape measurement. However, simultaneously achieving high accuracy and high speed remains challenging. To overcome this limitation, we propose a high-efficiency and robust binary fringe optimization method for superfast 3D shape measurement, which consists of 1D optimization and 2D modulation. Specifically, for 1D optimization, the three-level OPWM technique is introduced for high-order harmonics elimination, and an optimization framework is presented for generating the 'best' three-level OPWM pattern especially for large fringe periods. For 2D modulation, a single-pattern three-level OPWM strategy is proposed by utilizing all the dimensions for intensity modulation to decrease the required projection patterns. Thus, the proposed method essentially belongs to the 2D modulation technique, yet iterative optimization is carried out along one dimension, which drastically improves the computational efficiency while ensuring high accuracy. With only one set of optimized patterns, both simulations and experiments demonstrate that high-quality phase maps can be consistently generated for a wide range of fringe periods (e.g., from 18 to 1140 pixels) and different amounts of defocusing, and it can achieve superfast and high-accuracy 3D shape measurement.

The phase-coding method is useful in 3D shape measurement due to its low sensitivity to surface contrast and camera noise. However, as the number of fringe periods increases, the wrapped phase recovered from the sinusoidal phase-shifting patterns is always misaligned with the fringe order calculated by the stair-coding patterns. To solve this, an improved dual-frequency phase-coding fringe projection method is designed to realize absolute phase retrieval. Specifically, the auxiliary fringe order determined by phase-shifting is used to get the absolute phase, which can eliminate fringe-order errors. Moreover, to increase speed and the number of codewords, this paper presents a scheme that only requires seven patterns, including a uniform gray image, two high-frequency sinusoidal fringe patterns, two low-frequency sinusoidal fringe patterns and two phase-coding fringe patterns. Therefore, this method requires fewer patterns to realize high-resolution measurement than existing methods. The success of the proposed method can be proved by experiments.

In this paper, we propose a fast stereo 3D imaging technique based on random speckle projection and its FPGA implementation. Stereo vision, as a classic passive method for 3D shape measurement based on the multi-view geometric constraints, can realize the 3D reconstruction of the tested scene using a pair of images captured through the binocular cameras. In addition, some complicated matching techniques, such as graph cut and block matching, are used to obtain a global disparity map but it leads to massive computing overhead. To solve this problem, we developed a fast stereo vision system based on FPGA. Benefiting from the full parallel architecture of FPGA, the complete computational framework is based on a full pipeline design, that is, the storage and calculation of data are performed under the system clock to implement different works of stereo vision (including stereo rectify and stereo matching) at the same time, promoting calculation speed and measurement efficiency. In order to further improve the accuracy of 3D measurement, by introducing structured light illumination into the existing system, a projection system based on random speckle is designed where fast speckle projection and synchronous acquisition are realized on the FPGA hardware. Experimental results verify that our method can achieve high-speed and robust 3D shape measurement.

This paper presents a novel area-modulated phase-shifting binary method for three-dimensional (3D) shape measurement. First, the defocused triangular binary pattern is introduced to obtain the high-quality sinusoidal fringe without gamma correction. Then, the modified three steps phase-shifting algorithm which includes an average intensity image is adopted to get the wrapped phase and the object mask. After that, a multi-frequency method is used to obtain the unwrapped phase. Finally, a two-step matching method is applied to get the corresponding matching points. The experiments show that the proposed method is more precise and efficient compared with the previous methods.