Practical three-dimensional computer vision techniques for full-field surface measurement

Fringe projection profilometry (FPP) and phase measuring deflectometry (PMD) are the currently popular technologies for measuring diffuse and specular reflective surfaces respectively. Both methodologies demonstrate superior performance for single-type reflective surfaces. However, with the rapid development of advanced manufacturing and intelligent manufacturing technologies, products have seen significant improvements in complexity, diversity, and personalization, often exhibiting composite reflective characteristics on their surfaces. The aforementioned single technology cannot meet the measurement requirements for composite reflective surfaces. To address this challenge, this paper presents a composite reflective surfaces measurement method combining FPP with PMD techniques, and an optimized system calibration method based on binocular stereo vision is proposed to enhance the system's measurement accuracy. To improve the measurement efficiency, a strategy in which the LCD screen and projector project different colorful fringe patterns simultaneously is adopted. To mitigate stray light interference from the projector on specular surface profilometry, a partition projection method is presented. Instead of establishing the pixel mapping relationship between the projector and diffused surface aeras, the coordinate mapping relationship between the specular reflective region and the projector pixel is established according to the phase-height relationship to realize the accurate projection of the projector to the diffused surface. Experiments have been performed to validate the proposed method, and the results demonstrate that it can reconstruct the composite surfaces with higher accuracy. In particular, the reconstruction accuracy of specular surface can reach several tens of nanometres.

Conditions of the digital recording of the fringe pattern determine the phase reconstruction procedure, which in turn directly shapes the final accuracy and throughput of the full-field (non-scanning) optical measurement technique and defines the system capabilities. In this way, the fringe pattern analysis plays a crucial role in the ubiquitous optical measurements and thus is under constant development focused on high temporal/spatial resolution. It is especially valuable in the quantitative phase imaging technology, which emerged in the high-contrast label-free biomedical microscopy. In this paper, I apply recently blossomed two-frame phase-shifting techniques to the QPI and merge them with advanced adaptive interferogram pre-filtering algorithms. Next, I comprehensively test such frameworks against classical and adaptive single-shot methods applied for phase reconstruction in dynamic QPI enabling highest phase time-space-bandwidth product. The presented study systematically tackles important question: what is the gain, if any, in QPI realized by recording two phase-shifted interferograms? Counterintuitively, the results show that single-shot demodulation exhibited higher phase reconstruction accuracy than two-frame phase-shifting methods in low and medium interferogram signal-to-noise ratio regimes. Thus, the single-shot approach is promoted due to not only high temporal resolution but also larger phase-information throughput. Additionally, in the majority of scenarios, the best option is to shift the paradigm and employ two-frame pre-filtering rather than two-frame phase retrieval. Experimental fringe analysis in QPI of LSEC/RWPE cell lines successfully corroborated all novel numerical findings. Hence, the presented numerical-experimental research advances the important field of fringe analysis solutions for optical full-field measurement methods with widespread bio-engineering applications.

Contour measurement by fibre optic fringe projection and Fourier transform analysis

Optical full-field measurement techniques have been widely studied in academia and applied to many actual fields of automated inspection, reverse engineering, cosmetic surgery, and so on. With the advent of color CCD cameras and DMD (Digital Micromirror Device) based color DLP (Digital Light Processing) projectors, their major red, green, and blue channels have been used as a carrier to code fringe patterns. Since three fringe patterns can be simultaneously projected and captured at one shot, the acquisition time reduces to 1/3 of the value by the gray fringe pattern projection. This chapter will introduce two kinds of applications of red, green, and blue as a carrier: 1) modulation and demodulation method of coding sinusoidal fringe patterns into RGB channels of a composite color image; and 2) modulation and demodulation method of coding sinusoidal and binary fringe patterns into RGB channels of multiple composite color images. Experiments on testing the two kinds of applications were carried out by measuring the shape of objects’ surface. The results confirm that red, green, and blue channels can be used as a carrier to reduce the acquisition time.

A new sequential phase demodulator based on a regularized quadrature and phase tracker system (RQPT) is applied to demodulate two-dimensional fringe patterns. This RQPT system tracks the fringe pattern's quadrature and phase in a sequential way by following the path of the fringes. To make the RQPT system more robust to noise, the modulating phase around a small neighborhood is modeled as a plane and the quadrature of the signal is estimated simultaneously with the fringe's modulating phase. By sequentially calculating the quadrature of the fringe pattern, one obtains a more robust sequential demodulator than was previously possible. This system may be applied to the demodulation of a single interferogram having closed fringes.

This paper presents an absolute phase calculation method from one composite RGB fringe pattern image by using Wavelet transform algorithm and the optimum fringe number selection. Three fringe patterns having optimum fringe numbers are projected simultaneously onto an object surface via the red, green and blue channels of a DLP (Digital Light Processing) projector. From a different viewpoint, a CCD camera captures the deformed fringe patterns with respect to the object shape to get a composite RGB image. After compensating for the crosstalk and chromatic aberration between color channels, three fringe patterns are extracted from the composite color image. Wavelet Transform algorithm is studied to calculate wrapped phase from one fringe pattern. Therefore, three wrapped maps are obtained from the three extracted fringe patterns. An absolute phase map is calculated pixel by pixel after applying the optimum three-fringe numbers selection method to the three obtained wrapped phase maps. Simulated and experimental data demonstrate the algorithm's validity of calculating the absolute phase and shape information. The proposed method can measure 3D shape information of moving objects since the system needs only one RGB fringe pattern image.

In the dynamic 3D shape measurement phase distribution has 3D character, in which phase changes along x and y directions in space and also along t direction in time. 3D phase unwrapping plays a very important role in the dynamic 3D shape measurement. In the dynamic 3D shape measurement methods based on the structured illumination, Fourier transformation profilometry (FTP) is particularly fit for dynamic 3D measurement, because of only one fringe pattern needed and full field analysis. In this paper some 3D phase unwrapping techniques for dynamic 3D shape measurement mainly in our Lab. are presented and reviewed. The basic methods and algorithm design are introduced. The basic methods include direct 3D phase unwrapping, 3D diamond phase unwrapping, 3D phase unwrapping based on reliability ordering, 3D phase unwrapping based on marked fringe tracing. The advantage of the phase unwrapping based on reliability ordering is that the path of phase unwrapping is always along the direction from the pixel with higher reliability parameter value to the pixel with low reliability parameter value. Therefore, in the worse case the error is limited, if there is any, to local minimum areas.

Many optical measurement techniques provide fringe patterns as their results. The decodification processes that employ one or several fringe patterns to automatically retrieve the phase are generally designated as phase-evaluation methods. In this work, an overview of these methods will be schematically presented. Their particular performances will be compared, stressing their main advantages and drawbacks. An important group of these methods employs the well-known phase-shifting algorithms as a tool for calculating the phase. In the general form of these algorithms, the principal value of the optical phase is computed by an inverse trigonometric function whose argument is a combination of phase-shifted intensity values, provided by the modulation of one or several fringe patterns. These algorithms will be also studied in the general context of the phase-evaluation methods.

Digital image correlation method is widely used in macro- and mesoscopic mechanical tests because of the advantages of non-contact, high precision, full field measurement, and simple experimental equipment. But it still has many shortcomings in theory and application. For example, the traditional DIC method has difficulty obtaining the desired result when the surface of the test object is rotated substantially or the deformation involves large rotation (i.e. measuring helicopter blade deformation dynamically). Some scholars argue that a rotation angle greater than 7° will not work with the DIC algorithm, and this phenomenon is called decorrelation [1-3]. The application of various nonlinear optimization algorithms greatly reduces the computation time of the DIC iteration process.

Projector nonlinearity is a common problem for digital structured light-based three-dimensional (3-D) shape measurement techniques. A temporal-spatial binary encoding method is presented for the purpose of eluding it. We build a 3-D shape measurement scheme by combining our proposed method with phase measurement profiling. A standard sinusoidal fringe pattern is divided into more than two binary fringe patterns using specially designed temporal and spatial binary encoding rule based on intensity hierarchic quantification, and then are in-focus projected onto the measured object at a time sequence to reconstruct a frame phase-shifting fringe image. On account of the projected binary fringe pattern strictly consisting of zeros and ones, the influence of the projector nonlinearity on the measurement result can be effectively ruled out and simultaneously enables high-quality sinusoidality. In-depth investigations by theoretical analysis and experiments are conducted to demonstrate the performance of this method.

For the surface shape measurement of a semiconductor with a highly reflective index, it is im portant to effectively suppress the harmonic signal s from multi ple reflection s. In application, the phase extraction algorithm should have a maximum value when there is no phase -shift miscalibration. In this presentation, a new 4N - 3 phase extraction algorithm , which has the ability to suppress harmonic signal s and exhibits a fringe contrast maximum value when there is no phase -shift error , was derived. This n ew 4N - 3 algorithm consists of a new polynomial window function and a discrete Fourier transform term and has the ability to compensate for 2 nd -order nonlineari ty in the phase shift. The suppression ability of the new polynomial window function is compared with other conventional window functions. The sampling functions of the new 4N - 3 algorithm have much smaller amplitudes in the vicinity of the detection freq uency than does synchronous detection or other phase extraction algorithm s with conventional window functions . Keywords: H ighly reflective index, p hase measurement, nonlinearity, window function

Conjugated wavefront fidelity has been evaluated using an interferometer with a phase conjugate mirror and fringe pattern computer processing. The quantitative interpretation of the fringe patterns is performed using direct spatial phase reconstruction methods (in the space and Fourier domain). A set of algorithms for fast and accurate phase evaluation, by fringe pattern processing, were derived. The result of optical interferograms evaluation appears in terms of different phase representations. These methods have been verified by studies of the optical phase conjugation fidelity in photorefractive crystals.

Holographic interference fringes, in general, contain information about the whole -field surface deformation of the test object over the entire surface. In order to analyse the fringe patterns, it is necessary that the centre of fringes should be well-defined so that accurate results could be extracted from the fringe pattern. Hence, it is necessary to process a poor quality fringe pattern by removing the noise before it is used for further analysis. The image degradation occurs due to (i) Non-Uniform light intensity used for illumination, (ii) Object surface and background characteristics and (iii) Presence of speckle noise. Hence, we need a special technique to improve the fringe quality i.e., contrast and visibility. This will enable extraction of central line of the fringes and the fringe order more accurately. In this paper, various processes used to achieve the improvement of fringe pattern stored in digital forn are presented. Different types of filters are evaluated to identify their performance. An interactive image processing software package has been described. In this package, the user has the option to choose different combinations of processes till an acceptable quality of fringe patterns are obtained. Results of sample images of different qualities are shown to highlight the improvement of fringe quality.

The quality control of highly reflective surfaces requires a measurement method which is able to resolve the surface shape in the nanometer range. Different methods have been developed in the past, e.g. based on interferometry or by tactile coordinate measurement machines. However, most of them do not match the industrial need for a fast method which is insensitive to environmental disturbance. The newly developed method using the reflection of fringe pattern by the surface under test, and therefore called "Fringe Reflection Technique (FRT)", overcomes the difficulties of known measurement methods. In this method a pattern of straight fringes is generated by a monitor. The mirrored pattern is observed by a camera via the object surface under test. Any deviation of the surface against the ideal, i.e. the mathematically accurate surface will yield a distortion of the pattern. This distortion is analyzed by an image processing system, called the Fringe Processor. The surface topology is delivered by local surface gradients which can be integrated to object shape or differentiated to local curvature. The resolution of the system can be adapted to the measurement requirements in a wide range from micrometer down to subnanometer. Anyhow, the system is stable against environmental disturbances. It works without vibration isolation in rooms without any climate control. It is possible to measure freeform surfaces with no constraints on object geometry. The measurement of a silicon mirror surface produced by diamond turning in a high precision tool machine serves as one example. The surface shape could be determined with a resolution below one nanometer. The measurements match the results of an interferometer and are better in certain areas.

Optical interferometry is a popular technique for high precision measurement. It produces fringe patterns for analysis. The output fringe patterns usually have noises, and thus a filtering process for fringe patterns is necessary. A few filtering algorithms, such as windowed Fourier filtering (WFF), have been proposed specially for fringe pattern denoising. Meanwhile, filtering techniques have been intensively studied for a long time in the general image processing area. It is curious that how the filtering techniques in general image processing area perform on fringe pattern denoising. In this paper, a state-of-the-art filtering algorithm, block-matching 3D filtering (BM3D), is selected and compared with the WFF.