Light source position calibration method for photometric stereo in capsule endoscopy

We present a light position calibration technique based on a general arrangement of at least two reflective spheres in a single image. Contrary to other techniques we do not directly intersect rays for triangulation but instead solve for the optimal light position by evaluating the image-space error of the light highlights reflected from the spheres. This approach has been very successful in the field of Structure-from-Motion estimation. It has not been applied to light source calibration because determining the reflection point on the sphere to project the highlight back in the image is a challenging problem. We show a solution and define a novel, non-linear error function to recover the position of a point light source. We also introduce a light position estimation that is based on observing the light source directly in multiple images which does not use any reflections. Finally, we evaluate both proposed techniques and the classical ray intersection method in several scenarios with real data.

Several techniques have been developed for recovering reflectance properties of real surfaces under unknown illumination conditions. However, in most cases, those techniques assume that the light sources are located at infinity, which cannot be applied to, for example, photometric modelling of indoor environments. We propose two methods to estimate the surface reflectance property of an object, as well as the position of a light source from a single image without the distant illumination assumption. Given a color image of an object with specular reflection as an input, the first method estimates the light source position by fitting to the Lambertian diffuse component, while separating the specular and diffuse components by using an iterative relaxation scheme. Moreover, we extend the above method by using a single specular image as an input, thus removing its constraints on the diffuse reflectance property and the number of light sources. This method simultaneously recovers the reflectance properties and the light source positions by optimizing the linearity of a log-transformed Torrance-Sparrow model. By estimating the object's reflectance property and the light source position, we can freely generate synthetic images of the target object under arbitrary source directions and source-surface distances.

In order to be able to match correctly the track elements produced by a muon in the Tracker and the Muon System of the CMS experiment [1] the mutual alignment precision between the Tracker and the Barrel Muon System must be no worse than 100-400 micrometers depending on the radial distance of the muon chambers from the Tracker. To fulfill this requirement an alignment system had to be designed. This system contains subsystems for determining the positions of the barrel and endcap chambers while a third one connects these two to the Tracker. Since the Barrel muon chambers are embedded into the magnet yoke of the experiment a nonconventional alignment method had to be developed. In this paper we restrict ourselves to the Barrel Alignment System and the calibration methods of its components.

In this paper, we consider the problem of estimating the 3D position and intensity of multiple light sources without using any light probe or user interaction. The proposed approach is twofold and relies on RGB-D data acquired with a low cost 3D sensor. First, we separate albedo/texture and illumination using lightness ratios between pairs of points with the same reflectance property but subject to different lighting conditions. Our selection algorithm is robust in presence of challenging textured surfaces. Then, estimated illumination ratios are integrated, at each frame, within an iterative process to recover position and intensity of light sources responsible of cast shadows. Estimated lighting characteristics are finally used to achieve realistic Augmented Reality (AR).

This paper focused on the light source optimization for visual inspection of piston surface defects. A novel structured lighting system was developed to overcome the difficulties in the well-qualified imaging of tiny and random defects on the cylindrical smooth surface of components such as pistons. A polarized light-filtering based method was proposed to enhance the defect image contrast, enlarge the detection range, and suppress the noise of flawless area. It was both experimentally proven and theoretically proven effective by mathematical modeling of light intensity loss during light propagation. To analyze the influence of the pose and position of light sources on defect imaging quality, the system model for camera, light source, and workpiece was established, and the structure parameters of the lighting system were optimized to improve the brightness uniformity over the entire inspection region and lighting efficiency. Moreover, a soft lighting method was introduced as a subsidiary means of removing the surface noise at two ends of a piston. This method was found to be more effective compared with conventional software filtering algorithm. Finally, a case was examined. The defects on the automotive brake master piston surface were captured using the optimized light source system. A random sampling experiment with 600 specimens revealed that the false-positive rate was about 2.3%, and with no leak detection. The findings have been applied in the production, and it is expected to be referred in inspecting defects on other smooth cylindrical surfaces.

An understanding of light and its interaction with matter is essential to produce images. As the modeling of light sources, light transport and light reflection improves, it becomes possible to render images with increasing realism. The central motivation behind this thesis is to improve realism in computer graphics images by more accurate local shading models and to assist the user to obtain the desired lighting effects with these more complex models. The first part of the thesis addresses the problem of rendering surfaces illuminated by extended (linear and area) light sources. To compute the light reflected by a surface element in a given direction, one needs to determine the unoccluded regions (shadowing) of each light source and then to compute the light reflection (shading) from each of these regions. Traditionally, point light sources are distributed on the lights to approximate both the shadowing and the shading. Instead, an efficient analytical solution is developed for the shading. Shadowing from extended light sources is a fairly expensive process. To give some insights on the complexity of computing shadows, some properties of shadows and algorithms are presented. To reduce the cost of computing shadows from linear light sources, two acceleration schemes, extended from ray tracing, are introduced and evaluated. The second part of this thesis addresses the problem of achieving a desired shading effect by building up systems of constraints. This approach is called inverse shading. It allows a user to obtain a desired lighting effect by interacting directly with the scene model. An interactive modeling system has been built to study the introduction of rendering issues into the modeling process itself. Two rendering aspects are considered: shading and shadows. By specifying a highlight position and size, the unique direction of a directional light source as well as the surface glossiness are determined. Interactively moving this highlight on the surface redefines the light direction while changing its size redefines the surface glossiness. By manipulating shadow volumes (the volume within which a particular light cannot be completely seen), the light source definition and position can be modified. By assigning colours to various points in a 3D scene, information about the colour of a surface as well as other surface characteristics can be deduced. This relatively new concept of defining the causes by manipulating the effects is expected to have a major impact on the design of future computer graphics modeling systems. It begins a new family of tools for the design of more intuitive user interfaces.

Light source position (LSP) estimation is a difficult yet an important problem in computer vision. A common approach for estimating the LSP assumes Lambert's law. However, in real-world scenes, Lambert's law does not hold for all different types of surfaces. Instead of assuming all that surfaces follow Lambert's law, our approach classifies image surface segments based on their photometric and geometric surface attributes (i.e. glossy, matte, curved, and so on) and assigns weights to image surface segments based on their suitability for LSP estimation. In addition, we propose the use of the estimated camera pose to globally constrain LSP for RGB-D video sequences. Experiments on Boom and a newly collected RGB-D video data sets show that the state-of-the-art methods are outperformed by the proposed method. The results demonstrate that weighting image surface segments based on their attributes outperform the state-of-the-art methods in which the image surface segments are considered to equally contribute. In particular, by using the proposed surface weighting, the angular error for LSP estimation is reduced from 12.6° to 8.2° and 24.6° to 4.8° for Boom and RGB-D video data sets, respectively. Moreover, using the camera pose to globally constrain LSP provides higher accuracy (4.8°) compared with using single frames (8.5°).

Inverse rendering, a technique for reconstructingthree-dimensional (3D) geometric structures and surface materials from two-dimensional (2D) images, is of great significance in fields such as robotic manipulation, virtual reality, and augmented reality. However, existing inverse rendering methods based on neural implicit representations are primarily suitable for scenarios with fixed light sources and struggle with dynamic point lighting. This paper introduces a novel 3D inverse rendering method that estimates the positions of point light sources, facilitating the 3D reconstruction of geometrical structures and the determination of light source locations from a collection of images captured from various perspectives, even under dynamic illumination conditions from an unknown point light source.

Using shadows in finding surface orientations

Differentiable rendering has recently opened the door to a number of challenging inverse problems involving photorealistic images, such as computational material design and scattering-aware reconstruction of geometry and materials from photographs. Differentiable rendering algorithms strive to estimate partial derivatives of pixels in a rendered image with respect to scene parameters, which is difficult because visibility changes are inherently non-differentiable. We propose a new technique for differentiating path-traced images with respect to scene parameters that affect visibility, including the position of cameras, light sources, and vertices in triangle meshes. Our algorithm computes the gradients of illumination integrals by applying changes of variables that remove or strongly reduce the dependence of the position of discontinuities on differentiable scene parameters. The underlying parameterization is created on the fly for each integral and enables accurate gradient estimates using standard Monte Carlo sampling in conjunction with automatic differentiation. Importantly, our approach does not rely on sampling silhouette edges, which has been a bottleneck in previous work and tends to produce high-variance gradients when important edges are found with insufficient probability in scenes with complex visibility and high-resolution geometry. We show that our method only requires a few samples to produce gradients with low bias and variance for challenging cases such as glossy reflections and shadows. Finally, we use our differentiable path tracer to reconstruct the 3D geometry and materials of several real-world objects from a set of reference photographs.

Shape from Shading (SFS) is a fundamental task in computer vision. By given information about the reflectance of an object’s surface and the position of the light source, the SFS problem is to reconstruct the 3D depth of the object from a single grayscale 2D input image. A modern class of SFS models relies on the property that the camera performs a perspective projection. The corresponding perspective SFS methods have been the subject of many investigations within the last years. The goal of this chapter is to give an overview of these developments. In our discussion, we focus on important model aspects, and we investigate some prominent algorithms appearing in the literature in more detail than it was done in previous works.

Augmented Reality (AR) scenarios aim to provide realistic blending between real world and virtual objects. A key factor for realistic AR is thus a correct illumination simulation. This consists in estimating the characteristics of real light sources and use them to model virtual lighting. In this paper, we briefly introduce a novel method for recovering both 3D position and intensity of multiple light sources using detected cast shadows. Our algorithm has been successfully tested on a set of real scenes where virtual objects have visually coherent shadows.

The production of high-quality low-defect single-domain flexible polymer opals which possess fundamental photonic bandgaps tuneable across the visible and near-infrared regions is demonstrated in an industrially-scalable process. Incorporating sub-50nm nanoparticles into the interstices of the fcc lattice dramatically changes the perceived color without affecting the lattice quality. Contrary to iridescence based on Bragg diffraction, color generation arises through spectrally-resonant scattering inside the 3D photonic crystal. Viewing angles widen beyond 40 masculine removing the strong dependence of the perceived color on the position of light sources, greatly enhancing the color appearance. This opens up a range of decorative, sensing, security and photonic applications, and suggests an origin for structural colors in Nature.

Modified dual reciprocity boundary element (MDBM) solution to the diffusion approximation of the radiative transport equation is employed to calculate the light distribution in normal and cancerous tissue, when irradiated with a collimated light beam via a cylindrical diffusion optical fiber. Three different active widths of the light source were investigated separately. The results of the light distribution obtained with this method were compared to results obtained using an in-house Monte-Carlo (MC) simulation program, and showed very close agreement with each other. Both methods showed that the optical penetration depth of the therapeutic light is lower in cancerous than in normal prostate tissue. The MDBM is over 90% faster than the MC technique, which could eventually be used to develop a real-time multidimensional program to (1) qualitatively estimate the therapeutic light distribution in heterogeneous tissue and (2) predict the best position of light sources in the targeted tissue to optimi e photodynamic therapy treatment planning.

An iterative scheme for computing the three-dimensional position and the surface orientation of an opaque object from a single shaded image is proposed. This method demonstrates that calculating the depth (distance) between the camera and the object from one shaded video image is possible. Most previous research work on this Shape from Shading problem involved the determination of surface orientation only. To measure the depth of an object, a point light source is used. We have estabilished an expression for the image intensity which depends on the light source position, surface orientation of the object, depth of the object, and the reflectance properties of the surface. Assuming that the object surface is uniform Lambertian the measured intensity level at a given pixel becomes the function of the surface orientation and the depth. To solve this non-linear equation, the surface also has to satisfy smoothness conditions. The equation is solved iteratively using standard methods of calculus of variation. The theoretical result is tested by experiments. Three objects (plane, cylinder, and sphere) are used. These initial results are very encouraging since they match the theoretical calculations within 10% error.