Composite fringe projection deep learning profilometry for single-shot absolute 3D shape measurement.

In fringe projection profilometry (FPP), efficiently recovering the absolute phase has always been a great chal-lenge. The stereo phase unwrapping (SPU) technologies based on geometric constraints can eliminate phase ambiguity without projecting any additional patterns, which maximizes the efficiency of the retrieval of abso-lute phase. Inspired by recent successes of deep learning for phase analysis, we demonstrate that deep learning can be an effective tool that organically unifies phase retrieval, geometric constraints, and phase unwrapping into a comprehensive framework. Driven by extensive training dataset, the properly trained neural network can achieve high-quality phase retrieval and robust phase ambiguity removal from only single-frame projection. Experimental results demonstrate that compared with conventional SPU, our deep-learning-based approach can more efficiently and robustly unwrap the phase of dense fringe images in a larger measurement volume with fewer camera views.

Fourier transform profilometry (FTP) is one of the frequently adopted three-dimensional (3D) shape measurement methods due to its ability to recover single-shot 3D shapes, yet it is challenging to retrieve the absolute phase map solely from one single grayscale fringe image. This paper presents a computational framework that overcomes this limitation of FTP with digital fringe projection (DFP). By using geometric constraints, an absolute phase map can be retrieved point-by-point from one single grayscale fringe image. Experiments demonstrate the success of our proposed framework with single-shot absolute 3D shape measurement capability.

Conventional three-dimensional (3D) shape measurement methods are typically generic to all types of objects. Yet, for many measurement conditions, such a level of generality is inessential when having the preknowledge of the object geometry. This paper introduces a novel adaptive algorithm for absolute 3D shape measurement with the assistance of the object computer-aided-design (CAD) model. The proposed algorithm includes the following major steps: (1)export the 3D point cloud data from the CAD model; (2)transform the CAD model into the camera perspective; (3)obtain a wrapped phase map from three phase-shifted fringe images; and (4)retrieve absolute phase and 3D geometry assisted by the CAD model. We demonstrate that if object CAD models are available, such an algorithm is efficient in recovering absolute 3D geometries of both simple and complex objects with only three phase-shifted fringe images.

In-situ areal inspection of powder bed for electron beam fusion system based on fringe projection profilometry

Single-shot fringe projection profilometry (FPP) offers significant advantages for dynamic three-dimensional (3D) measurement. However, existing deep learning-based single-shot methods still struggle to simultaneously and accurately extract the wrapped phase and fringe order. Because the former is a continuous physical quantity while the latter is a discrete integer label, their joint estimation from a single periodic pattern is inherently ambiguous, especially for isolated objects lacking spatial continuity. To address this problem, a deep learning-enabled single-shot FPP method is proposed by integrating inner shifting-phase encoding with a physically constrained dual-network architecture. The fringe order is embedded into the phase-shifting component of a single high-frequency fringe pattern, and two lightweight convolutional neural networks are then employed: one regresses the numerator and denominator of the wrapped phase, while the other classifies the inner shifting-phase value to recover the fringe order. This physically consistent decoupled design avoids the optimization conflicts of end-to-end regression and enables unambiguous absolute phase recovery. Experiments show that the proposed method could achieve high-accuracy absolute phase recovery comparable to traditional multi-frame techniques for measurements of complex and isolated objects, while maintaining low computational complexity. The dynamic experimental results demonstrate its application potential for dynamic 3D measurement.

In fringe projection profilometry, by taking the projector as a camera, depth measurement has the similar principle with binocular vision. The matching of feature points carrying depth information plays an important role in measurement. Due to the existence of difficultly-corrected projector errors, the measurement results of using a fixed point in camera corresponding to the points in the projector or of conversely using a fixed point in projector corresponding to the points in the camera is not exactly same. According to fringe projection system, a function representing the relationship between the depths and the corresponding fringe phases is introduced for measurement. This traditional method usually requires three or more known reference planes to calibrate parameters determined by the measurement system. However, another way is also feasible. Based on the measurement system, a generic function representing the relationship between the depths and the corresponding pixel shifts are also presented in this paper. Using these two functions allows us to recover the depth information of the object. Theoretical analyses are carried out to investigate the performances of these two methods. Comparative experiments are implemented, and their results demonstrate that the depth measurement method from pixel shifts is immune to the influence of high order harmonics as a little expense of efficiency, being better than the method from fringe phases having high measurement efficiency regardless of the low accuracy.

Fringe projection profilometry (FPP) has been one of the most popular non-contact methods for 3D surface measurement in recent years. In FPP, the quality of the fringe pattern determines the measurement accuracy and measurement range to a great extent. In this paper, we proposed a high-quality fringe projection method using a biaxial MEMS scanning mirror and a laser diode (LD). The fringe pattern is produced by a very low NA (numerical aperture) scanning laser beam. Compared with pixel array based fringe pattern generation method, such as DLP and LCOS, the generation method can produce higher performance fringe pattern, which is high contrast, narrow pitch and long depth. In this paper, we also did a contrast between different fringe pattern generation methods.

Recovering high-precision 3D information of dynamic scenes from single-frame fringe pattern is a major challenge in the field of fringe projection profilometry (FPP). Inspired by the successful application of deep learning in the field of FPP, we achieve single-frame, high-precision 3D measurement through the combination of data driven and physical model-based approaches. More specifically, we utilize deep learning with powerful feature extraction ability to reduce the number of fringe images required for phase demodulation to the physical limit. Then stereo phase unwrapping (SPU) approach based on geometric constraint is used to unwrap the high frequency wrapped phases obtained from deep learning, which maximizes the efficiency of FPP without projecting additional auxiliary patterns. Experimental results demonstrate that our method can realize high-precision 3D measurement only by a single projection, overcoming the motion sensitivity problem compared to traditional methods in dynamic scenes.

Profilometry by fringe projection is an optical metrology technique implemented projecting a sinusoidal grating onto a three-dimensional surface, where the resulting deformed grating image represents the object's height dis- tribution modulated in phase. Fringe projection can be implemented as a phase shifting process or as a one-frame method. In this work, both approaches are compared using a correlation and a Fourier transform based algorithm for phase extraction. As a first step of the surface recovering process, a wrapped phase image is obtained. Therefore, a correction is applied by means of a phase un- wrapping algorithm, obtaining a continuous phase distribution. The results show that the phase shifting approach returns a phase distribution with higher resolution and less noise than the one-frame approach.

Real-time motion-induced error reduction for phase-shifting profilometry with projection points tracking method

In high dynamic scenes, fringe projection profilometry (FPP) may encounter fringe saturation, and the phase calculated will also be affected to produce errors. This paper proposes a saturated fringe restoration method to solve this problem, taking the four-step phase shift as an example. Firstly, according to the saturation of the fringe group, the concepts of reliable area, shallow saturated area, and deep saturated area are proposed. Then, the parameter A related to the reflectivity of the object in the reliable area is calculated to interpolate A in the shallow and deep saturated areas. The theoretically shallow and deep saturated areas are not known in actual experiments. However, morphological operations can be used to dilate and erode reliable areas to produce cubic spline interpolation areas (CSI) and biharmonic spline interpolation (BSI) areas, which roughly correspond to shallow and deep saturated areas. After A is restored, it can be used as a known quantity to restore the saturated fringe using the unsaturated fringe in the same position, the remaining unrecoverable part of the fringe can be completed using CSI, and then the same part of the symmetrical fringe can be further restored. To further reduce the influence of nonlinear error, the Hilbert transform is also used in the phase calculation process of the actual experiment. The simulation and experimental results validate that the proposed method can still obtain correct results without adding additional equipment or increasing projection number, which proves the feasibility and robustness of the method.

In fringe projection profilometry, temporal phase unwrapping is an essential procedure to recover an unambiguous absolute phase even in the presence of large discontinuities or spatially isolated surfaces. In this work, a dual-sensitivity profilometry technique is presented, which is based on defocused projection of binary and sinusoidal fringe patterns with a high and low-frequency spatial carrier, respectively. The binary defocusing techniques (based on PWM or square-wave profile patterns, whose pitch is relatively narrow) have demonstrated successful for high-quality three-dimensional shape measurement when the projector presents a nonlinear response. But they suffer if pitch fringe is wide. On the other hand, using dual sensitivity profilometry, the quality of the unwrapped phase is determined by the high-frequency carrier. Thus, working with only one binary pattern, one can use a single defocusing level (low) in order to reduce the data acquisition time and maintain the quality of the unwrapped phase. Experimental results are presented to verify the success of the proposed method.

Fringe projection profilometry (FPP) is a widely employed technique owing to its rapid speed and high accuracy. However, when FPP is utilized to measure shiny surfaces, the fringes tend to be saturated or too dark, which significantly compromises the accuracy of the 3D measurement. To overcome this challenge, this paper proposes an efficient method for the 3D measurement of shiny surfaces based on FPP. Firstly, polarizers are employed to alleviate fringe saturation by leveraging the polarization property of specular reflection. Although polarizers reduce fringe intensity, a deep learning method is utilized to enhance the quality of fringes, especially in low-contrast regions, thereby improving measurement accuracy. Furthermore, to accelerate measurement efficiency, a dual-frequency complementary decoding method is introduced, requiring only two auxiliary fringes for accurate fringe order determination, thereby achieving high-efficiency and high-dynamic-range 3D measurement. The effectiveness and feasibility of the proposed method are validated through a series of experimental results.

This paper presents a non-contact continuous respiration monitoring system based on Fringe Projection Profilometry (FPP). The aim is to answer the question whether the FPP system is suitable for continuous monitoring of respiration in terms of accuracy and robustness. A FPP 3D sensor suitable for measuring respiratory motion of the chest wall and abdomen was designed and implemented. The sensor prototype was evaluated based on its temporal and spatial repeatability and resolution in the context of respiratory variations. To extract a respiration rate (RR) and a continuous respiration signal, the resulting 3D images from the sensor were further analyzed. The FPP sensor showed a high signal-to-noise ratio (SNR) of 37 dB when evaluated with an ideal sinusoidal respiration signal. Furthermore, a high mean correlation between the measured respiration signal and the reference signal of 0.95 was achieved across different scenarios.

Single-frame fringe pattern analysis with synchronous phase-shifting based on polarization interferometry phase measuring deflectometry (PIPMD)