Light source optimization for automatic visual inspection of piston surface defects

Erasing-inpainting-based data augmentation using denoising diffusion probabilistic models with limited samples for generalized surface defect inspection

This paper presents an automatic visual inspection of exterior surface defects of oil tanks using unmanned aerial vehicles (UAVs) and image processing with two cascading fuzzy logic algorithms. Corrosion is one of the defects that has a serious effect on the safety of the surface of oil and gas tanks. At present, human inspection, and climbing robots inspection are the dominant approach for rust detection in oil and gas tanks. However, there are many shortcomings to this approach, such as taking longer, high cost, and covering less surface area inspection of the tank. The purpose of this research is to detect the rust in oil tanks by localizing visual inspection technology using UAVs, as well as to develop algorithms to distinguish between defects and noise. The study focuses on two basic aspects of oil tank inspection through the images captured by the UAV, namely, the detection of defects and the distinction between defects and noise. For the former, an image processing algorithm was developed to improve or remove noise, adjust the brightness of the captured image, and extract features to identify defects in oil tanks. Meanwhile, for the latter aspect, a cascading fuzzy logic algorithm and threshold algorithm were developed to distinguish between defects and noise levels and reduce their impact through three stages of processing: The first stage of fuzzy logic aims to distinguish between defects and low noise generated by the appearance of objects on the surface of the tank, such as trees or stairs, and reduce their impact. The second stage aims to distinguish between defects and medium noise generated by shadows or the presence of small objects on the surface of the tank and reduce their impact. The third stage of the thresholding algorithm aims to distinguish between defects and high noise generated by sedimentation on the surface of the tank and reduce its impact. The samples were classified based on the output of the third stage of the threshold process into defective or non-defective samples. The proposed algorithms were tested on 180 samples and the results show its superiority in the inspection and detection of defects with an accuracy of 83%.

Surface defect inspection based on deep learning has demonstrated outstanding performance in improving detection accuracy and model generalization. However, the small scale of defect datasets always limits the application of deep models in industry. Generative models can obtain realistic samples in a very cheap way, which can effectively solve this problem and thus has received widespread attention in recent years. This paper provides a comprehensive analysis and summary of the current studies of surface defect inspection methods proposed between 2022 and 2024. First, according to the use of generative models, these methods are classified into four categories: Variational Auto-Encoders (VAEs), Generative Adversarial Networks (GANs), Diffusion Models (DMs), and multi-models. Second, the research status of surface defect inspection based on generative models in recent years is discussed from four aspects: sample generation, detection objective, inspection task, and learning model. Then, the public datasets and evaluation metrics that are commonly used for surface defect inspection are discussed, and a comparative evaluation of defect inspection methods based on generative models is provided. Finally, this study discusses the existing challenges for the defect inspection methods based on generative models, providing insights for future research.

For photometric stereo in capsule endoscopy, calibration of light source is crucial for improving the precision of surface normal estimation. Therefore, this paper presents an improved planar-mirror-based light source position calibration method: from captured images of light source and detected poses of planar mirror, light paths are retraced from camera to light source, and position of light source is triangulated with least square method. The contribution of this paper is that a refraction model of the planar mirror is employed in the retracement of light paths, thus the bias of light paths caused by refraction can be compensated and the position of light source can be estimated more precisely. The results of simulation and experiment show that the proposed method provides higher calibration accuracy than the current planar mirror-based calibration method and can improve the precision of subsequent photometric stereo-based 3D reconstruction.

Several techniques have been developed for recovering reflectance properties of real surfaces under unknown illumination conditions. However, in most cases, those techniques assume that the light sources are located at infinity, which cannot be applied to, for example, photometric modelling of indoor environments. We propose two methods to estimate the surface reflectance property of an object, as well as the position of a light source from a single image without the distant illumination assumption. Given a color image of an object with specular reflection as an input, the first method estimates the light source position by fitting to the Lambertian diffuse component, while separating the specular and diffuse components by using an iterative relaxation scheme. Moreover, we extend the above method by using a single specular image as an input, thus removing its constraints on the diffuse reflectance property and the number of light sources. This method simultaneously recovers the reflectance properties and the light source positions by optimizing the linearity of a log-transformed Torrance-Sparrow model. By estimating the object's reflectance property and the light source position, we can freely generate synthetic images of the target object under arbitrary source directions and source-surface distances.

An understanding of light and its interaction with matter is essential to produce images. As the modeling of light sources, light transport and light reflection improves, it becomes possible to render images with increasing realism. The central motivation behind this thesis is to improve realism in computer graphics images by more accurate local shading models and to assist the user to obtain the desired lighting effects with these more complex models. The first part of the thesis addresses the problem of rendering surfaces illuminated by extended (linear and area) light sources. To compute the light reflected by a surface element in a given direction, one needs to determine the unoccluded regions (shadowing) of each light source and then to compute the light reflection (shading) from each of these regions. Traditionally, point light sources are distributed on the lights to approximate both the shadowing and the shading. Instead, an efficient analytical solution is developed for the shading. Shadowing from extended light sources is a fairly expensive process. To give some insights on the complexity of computing shadows, some properties of shadows and algorithms are presented. To reduce the cost of computing shadows from linear light sources, two acceleration schemes, extended from ray tracing, are introduced and evaluated. The second part of this thesis addresses the problem of achieving a desired shading effect by building up systems of constraints. This approach is called inverse shading. It allows a user to obtain a desired lighting effect by interacting directly with the scene model. An interactive modeling system has been built to study the introduction of rendering issues into the modeling process itself. Two rendering aspects are considered: shading and shadows. By specifying a highlight position and size, the unique direction of a directional light source as well as the surface glossiness are determined. Interactively moving this highlight on the surface redefines the light direction while changing its size redefines the surface glossiness. By manipulating shadow volumes (the volume within which a particular light cannot be completely seen), the light source definition and position can be modified. By assigning colours to various points in a 3D scene, information about the colour of a surface as well as other surface characteristics can be deduced. This relatively new concept of defining the causes by manipulating the effects is expected to have a major impact on the design of future computer graphics modeling systems. It begins a new family of tools for the design of more intuitive user interfaces.

In this paper, we consider the problem of estimating the 3D position and intensity of multiple light sources without using any light probe or user interaction. The proposed approach is twofold and relies on RGB-D data acquired with a low cost 3D sensor. First, we separate albedo/texture and illumination using lightness ratios between pairs of points with the same reflectance property but subject to different lighting conditions. Our selection algorithm is robust in presence of challenging textured surfaces. Then, estimated illumination ratios are integrated, at each frame, within an iterative process to recover position and intensity of light sources responsible of cast shadows. Estimated lighting characteristics are finally used to achieve realistic Augmented Reality (AR).

The inelastic contribution to high resolution images of defects

In order to be able to match correctly the track elements produced by a muon in the Tracker and the Muon System of the CMS experiment [1] the mutual alignment precision between the Tracker and the Barrel Muon System must be no worse than 100-400 micrometers depending on the radial distance of the muon chambers from the Tracker. To fulfill this requirement an alignment system had to be designed. This system contains subsystems for determining the positions of the barrel and endcap chambers while a third one connects these two to the Tracker. Since the Barrel muon chambers are embedded into the magnet yoke of the experiment a nonconventional alignment method had to be developed. In this paper we restrict ourselves to the Barrel Alignment System and the calibration methods of its components.

Advanced optical proximity correction (OPC) designs have resulted in many challenges for both the manufacturers of photomasks and automated defect inspection equipment. The successful manufacture of advanced OPC photomasks includes the ability to resolve the OPC features, complete an automated defect inspection that captures all of the defects of concern, and accurately recognize and disposition these defects. New defect types associated with OPC features are important and will print on a wafer or affect critical feature dimensions. Advances in defect detection on designs with OPC required breakthroughs by both the photomask and inspection system manufacturer. Process improvements focused on better resolving OPC features, along with advancements in automated defect inspection systems, have resulted in overall improvements in automated defect infection capability for OPC designs. The ability of OPC inspection algorithms to detect critical defects was confirmed with the DuPont VerithoroTM 890, the OPC3 programed defect test reticle, and defect capture occurrences on actual designs with advanced OPC features. Sensitivity improvements of up to 0.17 micrometer were demonstrated for both OPC and non-OPC type defects.

Light source position (LSP) estimation is a difficult yet an important problem in computer vision. A common approach for estimating the LSP assumes Lambert's law. However, in real-world scenes, Lambert's law does not hold for all different types of surfaces. Instead of assuming all that surfaces follow Lambert's law, our approach classifies image surface segments based on their photometric and geometric surface attributes (i.e. glossy, matte, curved, and so on) and assigns weights to image surface segments based on their suitability for LSP estimation. In addition, we propose the use of the estimated camera pose to globally constrain LSP for RGB-D video sequences. Experiments on Boom and a newly collected RGB-D video data sets show that the state-of-the-art methods are outperformed by the proposed method. The results demonstrate that weighting image surface segments based on their attributes outperform the state-of-the-art methods in which the image surface segments are considered to equally contribute. In particular, by using the proposed surface weighting, the angular error for LSP estimation is reduced from 12.6° to 8.2° and 24.6° to 4.8° for Boom and RGB-D video data sets, respectively. Moreover, using the camera pose to globally constrain LSP provides higher accuracy (4.8°) compared with using single frames (8.5°).

Automatic inspection of gas pipeline welding defects using an expert vision system

We present a light position calibration technique based on a general arrangement of at least two reflective spheres in a single image. Contrary to other techniques we do not directly intersect rays for triangulation but instead solve for the optimal light position by evaluating the image-space error of the light highlights reflected from the spheres. This approach has been very successful in the field of Structure-from-Motion estimation. It has not been applied to light source calibration because determining the reflection point on the sphere to project the highlight back in the image is a challenging problem. We show a solution and define a novel, non-linear error function to recover the position of a point light source. We also introduce a light position estimation that is based on observing the light source directly in multiple images which does not use any reflections. Finally, we evaluate both proposed techniques and the classical ray intersection method in several scenarios with real data.

Guided wave techniques are expected to become an effective means for rapid, long-range inspection of pipes. Such techniques still have many practical difficulties in application, however, due to the complex characteristics of guided waves such as dispersion and their multimodal nature. A defect imaging technique is developed in this study to overcome the complexities of guided wave inspection. Received signals are separated into single-mode waveforms with a mode extraction technique and then spatial waveforms on the pipe surface at an arbitrary time are reconstructed. The predicted waveforms can provide a defect image at the moment when an incident wave arrives at a defect region, which is based on a time-reversal technique. This defect imaging technique is experimentally verified using eight signals detected at eight different circumferential positions. Images of artificial defects are obtained with one-hole and two-hole test pipes, and increasing the frequency of incident waves increases the resolution of the images. Holes and pipe ends are recognizable in the images, but the reconstructed images contain some errors in the area behind the defects where guided waves do not propagate or do not reflect back to the receiving transducers.

High-speed non-contact defect imaging for a plate-like structure