A spatial algorithm to reduce phase wraps from two dimensional signals in fringe projection profilometry

A phase unwrapping method suitable for high frequency fringe based on edge feature

In recent years, fringe projection profilometry (FPP), as a kind of three-dimensional shape measurement technology, has achieved the great breakthrough, due to the rapid development of the high-speed camera and high-speed projection equipment. The number-theoretical approach, as a classical method for the temporal phase unwrapping algorithm, is suitable for the binary defocusing FPP since it can avoid the acquiring of low frequency fringes. However, in order to ensure the stability of phase unwrapping, the period of fringe is generally around 20, which leads to the limited accuracy of 3D measurement. In this paper, we propose a bi-frequency number-theoretical phase unwrapping method with depth constraint. Using the principle of depth constraint, we will eliminate the period ambiguities of each pixel within a pixel-variant local period range so that the method only requires the coprime of two fringe frequencies within the local period range instead of the conventional global range. In this way, the requirement of stability of the traditional number-theoretical phase unwrapping can be adjusted from global range to local range. The stability is higher in the local period range due to containing less period ambiguities. As a result, we can realize phase unwrapping of higher frequency fringes with the same stability. Several experiments on various scenes are performed, verifying that our method can achieve high-speed and high-precision 3D measurement.

Phase unwrapping is one of the key problems in fringe projection profilometry (FPP). These years, CNN-based and Transformer-based models are widely used in FPP phase unwrapping. Unfortunately, the limitation of CNNs in long-range modeling capabilities prevent them from effectively extracting fine-grained features in wrapping phase images, while Transformer-based models struggle with efficiently handling long-range dependencies due to their local focus or computational demands. Recent studies demonstrate that Mamba, a novel selective state space model (SSM), achieves efficient modeling of long-range dependencies by dynamically adapting its parameters based on input context, particularly in tasks such as long-sequence modeling. Inspired by this, we propose a hybrid deep learning model integrating Res-UNet with Mamba (CNN-Mamba network, CMNet) for phase unwrapping of single-frame fringe patterns. First, a new dual-branch skip connection module based on Mamba is proposed named convolution weighted feature fusion SSM (Conv_WFF-SSM) is proposed, integrating Mamba’s selective state space mechanisms with multi-scale convolutional feature weighting to simultaneously address long-range interaction modeling and hierarchical feature preservation during phase expansion. Second, a parameter-free attention module (PFAM) is introduced into the encoder and decoder to reduce information loss caused by downsampling and upsampling operations without increasing network parameters. Finally, the iterative attentional feature fusion (IAFF) module integrated into residual block to instead of the ordinary sum operation for the first time. Experiments demonstrate the validity and robustness of the proposed technique.

Fringe projection profilometry (FPP) is a three-dimensional (3D) shape measurement method that involves projecting fringe patterns onto the object. A phase map retrieved from these fringe images is used for reconstructing the 3D surface of the object. Fourier transform and phase-shifting are two of the widely used fringe analysis techniques for performing 3D shape measurement using FPP. Fourier transform profilometry (FTP) has the advantage of performing high-speed measurement due to its single-shot nature. However, the reconstructed 3D surface has artifacts and high noise, specifically on the edges. On the other hand, phase-shifting profilometry (PSP) has the advantage of higher accuracy (relatively less noise level) but compromises on measurement speed. In this research, we propose a deep learning method to enhance the quality of the FTP phase maps using an efficient deep learning model called phase map enhancement net (PMENet). PMENet takes an FTP phase map as input and predicts a high-quality phase map in a supervised manner by using the phase maps obtained from 18-step PSP as ground truth. The training dataset was generated using a virtual FPP system. Validations were conducted on both the simulated data (generated by virtual FPP system) and the real-world data. The experimental results demonstrate that the trained neural network model can successfully improve the quality of the 3D geometric reconstruction with FTP and reduced the mean and root-mean-square error by 66% and 43%, respectively.

We introduce our recent work in deep-learning-based fringe projection profilometry (FPP), including phase retrieval, phase unwrapping, geometric constraint, color fringe demultiplexing, and high-dynamic-range imaging.

Fringe projection profilometry based on structured light has been widely used in 3-D vision due to its advantages of simple structure, good robustness, and high speed. The principle of this technique is to project multiple orders of stripes on the object, and the camera captures the deformed stripe map. Phase unwrapping and depth map calculation are important steps. Still, in actual situations, phase ambiguity is prone to occur at the edges of the object. In this paper, an adaptive phase segmentation and correction (APSC) method after phase unwrapping is proposed. In order to effectively distinguish the stable area and unstable area of the phase, a boundary identification method is proposed to obtain the structural mask of the phase. A phase compensation method is proposed to improve the phase accuracy. Finally, we obtain the 3-D reconstruction result based on the corrected phase. Specific experimental results verify the feasibility and effectiveness of this method.

To reveal the fundamental aspects hidden behind a variety of transient events in mechanics, physics, and biology, the highly desired ability to acquire three-dimensional (3D) images with ultrafast temporal resolution has been long sought. As one of the most commonly employed 3D sensing techniques, fringe projection profilometry (FPP) reconstructs the depth of a scene from stereo images taken with sequentially structured illuminations. However, the imaging speed of current FPP methods is generally capped at several kHz, which is limited by the projector-camera hardware and the number of fringe patterns required for phase retrieval and unwrapping. Here we report a novel learning-based ultrafast 3D imaging technique, termed single-shot super-resolved FPP (SSSR-FPP), which enables ultrafast 3D imaging at 100,000 Hz. SSSR-FPP uses only one pair of low signal-to-noise ratio (SNR), low-resolution, and pixelated fringe patterns as input, while the high-resolution unwrapped phase and fringe orders can be deciphered with a specific trained deep neural network. Our approach exploits the significant speed gain achieved by reducing the imaging window of conventional high-speed cameras, while “regenerating” the lost spatial resolution through deep learning. To demonstrate the high spatio-temporal resolution of SSSR-FPP, we present 3D videography of several transient scenes, including rotating turbofan blades, exploding building blocks, and the reciprocating motion of a steam engine, etc., which were previously challenging or even impossible to capture with conventional methods. Experimental results establish SSSR-FPP as a significant step forward in the field of 3D optical sensing, offering new insights into a broad spectrum of dynamic processes across various scientific disciplines.

Blade is the key component of the aero-engine. In generally, it requires precise size and accurate shape, so the three-dimensional shape measurement of the blade is very important. Fringe projection profilometry is generally used to measure the 3D shape of an object, because of the advantages of non-contact operation, full-field acquisition, high resolution, and fast data processing. In this paper, 3D shape measurement method based on surface structured light that combination of gray-code and phase-shift projection is proposed. The wrapped phase of the blade is got by the four-step phase-shift method. In the projection process of the gray-code, due to the ambient light, different surface reflectance and surface physical discontinuous, the edge of gray-code becomes smooth, leading to decode error, so that the gray-code pattern must be binaried before decode. Through the accurate binarization of the blade gray-code pattern, the decoding cycle of the blade is achieved, and the unwrapped phase is achieved by phase unwrapping. Then the unwrapped phase difference between the blade and reference plane is got, the height of the blade can be obtained by the relationship between the phase difference and the height. The experimental and simulation results show that the proposed method can achieve a high precision, high speed and low cost 3D shape measurement of the blade. The measurement accuracy reaches 0.03 mm. The proposed method extends the reliability and practicality of the fringe projection profilometry.

In fringe projection profilometry (FPP), efficiently recovering the absolute phase has always been a great chal-lenge. The stereo phase unwrapping (SPU) technologies based on geometric constraints can eliminate phase ambiguity without projecting any additional patterns, which maximizes the efficiency of the retrieval of abso-lute phase. Inspired by recent successes of deep learning for phase analysis, we demonstrate that deep learning can be an effective tool that organically unifies phase retrieval, geometric constraints, and phase unwrapping into a comprehensive framework. Driven by extensive training dataset, the properly trained neural network can achieve high-quality phase retrieval and robust phase ambiguity removal from only single-frame projection. Experimental results demonstrate that compared with conventional SPU, our deep-learning-based approach can more efficiently and robustly unwrap the phase of dense fringe images in a larger measurement volume with fewer camera views.

Fringe projection profilometry is a widely used three-dimensional (3D) optical measurement technique. However, it is quite challenging to quickly obtain the 3D information of objects. Minimizing the number of patterns for phase unwrapping is an efficient way to address the above problem, and many methods have been proposed. This paper reports a period-wise phase unwrapping method with only two gray level coding patterns. Due to a hybrid coding strategy implemented, the number of patterns used in the proposed method reduces by one in comparison with conventional gray level method when generating the same number of codewords. A robust decoding scheme is also developed to determine the fringe order, and its accuracy is guaranteed by using the self-correcting method to remove the phase errors. Some experiments have been conducted, and their results confirm that the success of the proposed method.

Phase unwrapping is the core of phase shift profilometry. Conventional phase unwrapping usually requires the projection of additional patterns or other constrains to unwrap the high-frequency wrapped phase into absolute phase, which increases time consumption and complicates measurement procedures. To solve this problem, we propose a self-coding phase shift (SCPS) method for retrieving the absolute phase. Two sets of codes are encoded into the projection patterns, namely, the De-Bruijn sequence code and the phase gradient code. The fringe order can then be determined by combining these two different encodings. The proposed method does not require any prior information and only demands four phase shift fringes. Experimental results demonstrate that the SCPS method is highly robust and accurate in measuring both complex and separated objects.

A predistortion-based 3-D phase mapping method with linear epipolar constraints for fringe projection system calibration and measurement is reported. Hybrid system calibration methods have been developed recently to provide accurate measurement, high speed, and flexibility for fringe projection profilometry (FPP). However, these methods still have significant limitations. With projector distortion, for example, rigorous post-undistortion often results in low efficiency in computation. Conversely, simple, approximated undistortion is only effective for low levels of distortion. As a result, projector undistortion is usually not prioritized in practice. Aiming to achieve high-accuracy 3-D reconstruction with FPP, an easily implementable projector undistortion method is proposed in this article. Theoretical analyses and simulations confirm that the proposed method is rigorous. Experiments validate that the pixelwise root-mean-square reconstruction error is 0.042 mm, a 24% reduction from that of a simple post-undistortion method. The proposed method is also efficient computationally. Finally, a 3-D phase mapping lookup table is built. The results show that the proposed method is effective for achieving high accuracy and speed in 3-D surface measurement.

Phase unwrapping is one of the key technologies in electronic speckle pattern interferometry. A new phase unwrapping algorithm, based on windowed Fourier transform is proposed. The high noisy phase map is denoised by the window Fourier transform approach and then is uwrapped by the discrete cosine transform. The method is tested in this paper using a circumferentially fixed circular plate with a point load at the centre and compared with not denoised wrapped phase map and unwrapped phase map. The result shows the new proposed phase unwrapping method in denoising and improving image quality has obvious superiority.

In the task of 3-D topography measurement by fringe projection profilometry (FPP), it is crucial to establish mapping from the phase map to the 3-D coordinates, known as 3-D calibration. The traditional methods are prone to select some specific functions to fit the phase-to-coordinates’ relationship, which needs to make a compromise between measurement accuracy and efficiency. This article proposes a novel calibration method based on the Gaussian process (GP) regression to solve this problem. In this work, according to the geometric and other systemic constraints, a pixel-dependent semiparameterized calibration model is derived to guarantee the efficiency of computation and data storage. Based on the spatial correlations of the calibration data, the GP regression method is applied to enhance the fitting ability and flexibility of the calibration model without any specific functions and parameters. The GP regression method is also applied to remove random noise from the phase map, which further improves the accuracy of 3-D coordinates. The experimental results of measuring a whiteboard and a double ball bar demonstrate the superiority of the proposed GP-based calibration model in terms of accuracy and robustness when compared with the traditional models.

Single-shot fringe projection profilometry (FPP) is essential for retrieving the absolute depth information of the objects in high-speed dynamic scenes. High-precision 3D reconstruction using only one single pattern has become the ultimate goal in FPP. The frequency-multiplexing (FM) method is a promising strategy for realizing single-shot absolute 3D measurement by compounding multi-frequency fringe information for phase unwrapping. In order to solve the problem of serious spectrum aliasing caused by multiplexing schemes that cannot be removed by traditional spectrum analysis algorithms, we apply deep learning to frequency multiplexing composite fringe projection and propose a composite fringe projection deep learning profilometry (CDLP). By combining physical model and data-driven approaches, we demonstrate that the model generated by training an improved deep convolutional neural network can directly perform high-precision and unambiguous phase retrieval on a single-shot spatial frequency multiplexing composite fringe image. Experiments on both static and dynamic scenes demonstrate that our method can retrieve robust and unambiguous phases information while avoiding spectrum aliasing and reconstruct high-quality absolute 3D surfaces of objects only by projecting a single composite fringe image.