Computational methods for the shape from shading problem

Shape-from-shading (SfS) is a fundamental problem in computer vision; it is based upon the image irradiance equation. Recently, the authors proposed to solve the image irradiance equation under the assumption of perspective projection rather than the common orthographic one. The solution was a modification of the fast marching method of Kimmel and Sethian. This paper presents an application of this novel perspective algorithm to reconstruction of medical images. We focus on gastrointestinal endoscopy and compare the two versions of the fast marching method (orthographic vs. perspective). The examples and comparison show that, unlike orthographic SfS, perspective SfS is robust and can be utilized for real-life applications.

This paper presents a new method for recovering the shape of hybrid surfaces that have both diffuse reflection and specular reflection using shape from shading (SFS). The image irradiance equation has been derived as an explicit partial differential equation (PDE) under the assumptions of orthographic camera projection and distant point light source. The reflectance model of Ward has been used to express the hybrid reflection as a linear combination from the diffuse and specular components. The resulting PDE is solved using the Lax-Friedrichs sweeping method. The proposed algorithm is evaluated by using both synthetic data and real images and the experimental results show the efficiency of the approach.

Poster from:<br> V. B. S. Prasath, H. Kawanaka. Near-light perspective shape from shading for 3D visualizations in endoscopy systems. IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Kansas, MO, USA, Nov 13-16, 2017.

Shape-from-Shading (SfS) is a fundamental problem in Computer Vision. A very common assumption in this field is that image projection is orthographic. This paper re-examines the basis of SfS, the image irradiance equation, under a perspective projection assumption. The resultant equation does not depend on the depth function directly, but rather, on its natural logarithm. As such, it is invariant to scale changes of the depth function. A reconstruction method based on the perspective formula is then suggested; it is a modification of the Fast Marching method of Kimmel and Sethian. Following that, a comparison of the orthographic Fast Marching, perspective Fast Marching and the perspective algorithm of Prados and Faugeras on synthetic images is presented. The two perspective methods show better reconstruction results than the orthographic. The algorithm of Prados and Faugeras equates with the perspective Fast Marching. Following that, a comparison of the orthographic and perspective versions of the Fast Marching method on endoscopic images is introduced. The perspective algorithm outperformed the orthographic one. These findings suggest that the more realistic set of assumptions of perspective SfS improves reconstruction significantly with respect to orthographic SfS. The findings also provide evidence that perspective SfS can be used for real-life applications in fields such as endoscopy.

In the areas of computer vision and computer graphics, analyzing information about three-dimensional objects from two-dimensional images is essential. Shape from shading is a very fundamental problem in computer vision, which uses features and patterns in images to infer the shape of the object in our viewing direction. The influence of solving this problem is significant. For example, a typical application of shape from shading is in astronomy, in which we can obtain landscape features of a planet by analyzing the information from the image we get.

&lt;p&gt;This paper describes a shape from shading technique for the reconstruction of specular surfaces based on refraction and color illumination or multiple pulsed light sources. The technique is used to retrieve the small-scale shape of wave-undulated water surfaces in the laboratory and the field.&lt;/p&gt;

Although photogrammetry has long been used as the primary method to produce digital elevation models (DEMs), there are still some locations that are not covered by multiangle images because of various limitations. Moreover, traditional interpolation methods for producing denser DEMs often cause oversmoothing, particularly over rough terrain. To formulate a robust procedure that is capable of reconstructing the terrain surface using sparse ground control points in heterogeneous areas, this letter describes an improved spatial enhancement method combining perspective shape from shading (SFS) using a single optical image. This method is called SFS-based densification using multispectral information (SDMI). First, the image irradiance equation based on the perspective Lambertian model is formulated as a static Hamilton–Jacobi equation and is solved using a fast sweeping strategy. We then reconstruct the relative surface shape and apply an edge-preserved iterative interpolation method to generate a higher resolution DEM grid. Multispectral information is used to reveal the actual surface reflection properties, and the land surface is classified into several types to better estimate surface reflectance. Experiments indicate that SDMI is effective for the interpolation of a sparse grid DEM over heterogeneous terrain.

Wavelet-Based Shape from Shading

Shape-from-shading (SFS) is an important method to reconstruct three-dimensional (3D) shape of a surface in photometry and computer vision. Lambertian surface reflectance and orthographic camera projection are two fundamental assumptions which generally result in undesirable reconstructed results since inaccurate imaging model is adopted. In this paper, we propose a new fast 3D shape reconstruction approach via the SFS method relaxing the two assumptions. To this end, the Oren-Nayar reflectance and perspective projection models are used to establish an image irradiance equation which depicts the relationship between the 3D shape of non-Lambertian surfaces and its corresponding two-dimensional (2D) shading image. Considering the light attenuation of the near point source, the image irradiance equation is transformed into a static Hamilton-Jacobi partial differential equation (PDE) by solving a quadratic equation. The viscosity solution of the resultant Hamilton-Jacobi PDE is approximated by using optimal control theory and iterative fast marching method starting from a viscosity supersolution. The performance of the proposed approach is evaluated on both synthetic and real-world images and the experimental results demonstrate that the proposed approach is accurate and fast.

In this chapter, the algorithms to reconstruct the shape of the visible 3D surface using only the brightness variation information (shading, that is, the level variations of gray or colored) observed in the image are described. The inverse problem of reconstructing the shape of the visible surface from the changes in brightness in the image is known as the Shape from Shading (SfS) problem. The reconstruction of the visible surface should not be strictly understood as a 3D reconstruction of the surface. In fact, from a single point of the observation of a scene, a monocular vision system cannot estimate a distance measure between the observer and a visible object, so with the SfS algorithms there is a nonmetric but qualitative reconstruction of the 3D surface. The theory of the SfS is described, which is based on the knowledge of the light source (direction and distribution), the model of reflectance of the scene, the observation point, and the geometry of the visible surface, which together contributes to the image formation process. The relationships between the light intensity values of the image and the geometry of the visible surface are derived (in terms of the orientation of the surface, point by point) under some lighting conditions and the reflectance model. Other 3D surface reconstruction algorithms based on the Shape from XXX paradigm are also described, where XXX can be texture, structured light projected onto the surface to be reconstructed, or 2D images of the focused or defocused surface.

Abstract. Topographic mapping, e.g. the generation of Digital Elevation Models (DEM), is of general interest to the remote sensing community and scientific research. Commonly, photogrammetric methods, e.g. stereo image analysis methods (SIAM) or bundle adjustment methods (BAM), are applied to derive 3D information based on multiple images of an area. These methods require the detection of control points, i.e. common points within multiple images, which relies on a similarity measure and usually yields a sparse map of 3D points. The full spatial DEM is then obtained by interpolation techniques or imposed restrictions, e.g. smoothness constraints. Since BAM utilizes all images of the area, it is assumed to provide a more accurate DEM than SIAM which utilizes only pairs of images. Intensity-based shape recovery, e.g. shape from shading (SfS), utilizes the reflectance behavior of the object surface and thus provides a dense map of relative height changes, which provide the possibility to refine the photogrammetric DEMs. Based on Rosetta NavCam images of 67P/Churyumov-Gerasimenko we compare intensity-based DEM refinement methods which use DEMs obtained based on SIAM and BAM as a reference. We show that both the SIAM based DEM refinement and the BAM based DEM refinement are of similar quality. It is thus possible to derive DEMs of high lateral resolution by applying the intensity-based refinement to the less complex SIAM.

The Effects of Continuously Changing Lighting and Perspective on Shape from Shading in Pigeons

No AccessHandbuch Optische MesstechnikSep 2017Shape from ShadingMichael Schuth, Wassili BuerakovMichael Schuth, Wassili Buerakovhttps://doi.org/10.3139/9783446436619.024SectionsAboutPDF ToolsAdd to FavoritesDownload CitationTrack CitationsCopy LTI LinkPDF key 'share (en)' returned an object instead of string.FacebookTwitterEmailLinkedIn previous chapternext chapter FiguresReferencesRelatedDetails 2017Pages: 199-201Print ISBN: 978-3-446-43634-3eISBN: 978-3-446-43661-9 Copyright & Permissions© 2017 Carl Hanser Verlag GmbH & Co. KGPDF downloadLoading ...

On shape from shading

Abstract. Lunar topographic information, e.g., lunar DEM (Digital Elevation Model), is very important for lunar exploration missions and scientific research. Lunar DEMs are typically generated from photogrammetric image processing or laser altimetry, of which photogrammetric methods require multiple stereo images of an area. DEMs generated from these methods are usually achieved by various interpolation techniques, leading to interpolation artifacts in the resulting DEM. On the other hand, photometric shape reconstruction, e.g., SfS (Shape from Shading), extensively studied in the field of Computer Vision has been introduced to pixel-level resolution DEM refinement. SfS methods have the ability to reconstruct pixel-wise terrain details that explain a given image of the terrain. If the terrain and its corresponding pixel-wise albedo were to be estimated simultaneously, this is a SAfS (Shape and Albedo from Shading) problem and it will be under-determined without additional information. Previous works show strong statistical regularities in albedo of natural objects, and this is even more logically valid in the case of lunar surface due to its lower surface albedo complexity than the Earth. In this paper we suggest a method that refines a lower-resolution DEM to pixel-level resolution given a monocular image of the coverage with known light source, at the same time we also estimate the corresponding pixel-wise albedo map. We regulate the behaviour of albedo and shape such that the optimized terrain and albedo are the likely solutions that explain the corresponding image. The parameters in the approach are optimized through a kernel-based relaxation framework to gain computational advantages. In this research we experimentally employ the Lunar-Lambertian model for reflectance modelling; the framework of the algorithm is expected to be independent of a specific reflectance model. Experiments are carried out using the monocular images from Lunar Reconnaissance Orbiter (LRO) Narrow Angle Camera (NAC) (0.5 m spatial resolution), constrained by the SELENE and LRO Elevation Model (SLDEM 2015) of 60 m spatial resolution. The results indicate that local details are largely recovered by the algorithm while low frequency topographic consistency is affected by the low-resolution DEM.