• Home
  • Search
  • Point spread function engineering for scene recovery
  • https://doi.org/10.7916/d8pr836sCopy DOI Icon

Point spread function engineering for scene recovery

Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

A computational camera uses a combination of optics and processing to produce images that cannot be captured with traditional cameras. Over the last decade, a range of computational cameras have been proposed, which use various optics designs to encode and use computation to decode useful visual information. What is often missing, however, is the quantitative analysis of the relation between camera design and the captured visual information, and little systematic work has been done to evaluate and optimize these computational camera designs. While the optics of computational cameras may be quite complicated, many of them can be effectively characterized by their point spread functions (PSFs): the intensity distribution on an image sensor as a response to a point light source in a scene. This thesis explores the techniques to characterize, evaluate and optimize computational cameras via PSF engineering for computer vision tasks, including image recovery, 3D reconstruction, and image refocusing. I first explore PSF engineering techniques to recover image details from blurry images. Image blur is a problem commonly seen in photography due to a number of reasons, including defocus, lens aberration, diffraction, atmospheric turbulence, object motion, and etc. Image blur is often formulated as a convolution of the latent blur-free image and a PSF, and deconvolution (or deblurring) techniques have to be used to recover image details from a blurry region. Here, I propose a comprehensive framework of PSF evaluation for the purpose of image deblurring, in which the effects of image noise, deblurring algorithm, and the structure of natural images are all taken into account. In defocus blur, the shape of defocus PSF is largely determined by the aperture pattern of the camera lens. By using an evaluation criterion derived from the comprehensive framework, I optimize the aperture pattern to preserve many more image details when defocus occurs. Both through simulations and experiments, I demonstrate the significant improvement gained by using optimized aperture patterns. While defocus blur causes a loss in image detail, it also encodes depth information of scenes in images. A typical depth from defocus (DFD) technique computes depth from two or more images that are captured with circular aperture lenses of different focus settings. Circular apertures produce circular defocus PSFs. In this thesis, I show that the use of a circular aperture severely restricts the accuracy of DFD, and propose a comprehensive framework of PSF evaluation for depth recovery. With this framework, we can derive a criterion for evaluating a pair of apertures with respect to the precision of depth recovery. This criterion is optimized using a genetic algorithm and gradient descent search to arrive at a pair of high resolution apertures. The two coded apertures are found to complement each other in the scene frequencies they preserve. With this property it becomes possible to not only recover depth with greater fidelity but also to obtain a high quality all-focused image from the two defocused images. While depth recovery can significantly benefit from optimized aperture patterns, its overall performance is rigidly limited by the lens aperture's physical size. To transcend this limitation, I propose a novel depth recovery technique using an optical diffuser - referred to as depth from diffusion (DFDiff). I show that DFDiff is analogous to conventional DFD, in which the scatter angle of the diffuser determines the system's effective aperture. High precision depth estimation can be achieved by choosing a proper diffuser and no longer requires the large lenses that DFD requires. Even a consumer camera with a low-end small lens can be used to do high-precision depth estimation when coupled with an optical diffuser. In my detailed analysis of the image formation properties of a DFDiff system, I show a number of examples demonstrating greater precision in depth estimation when using DFDiff. While the finite depth of field (DOF) of a lens camera leads to defocus blur, it also produces artistic visual experience. Many of today's displays are interactive in nature, which opens up a possibility for new kind of visual representations. Users could, for example, interactively refocus images to different depths, so that they can experience the artistic narrow DOF images while simultaneously making available the image detail for the entire image. To enable image refocusing, one typical approach is to capture the entire light field. But this method has the drawback of a significant sacrifice in spatial resolution due to the dimensionality gap: the captured information (light field) is 4D, while the required information (focal stack) is only 3D.In this thesis, I present an imaging system that directly captures focal stacks by a sweeping focal plane. First, I describe how to synchronize focus sweeping with image capturing so that the summed DOF of a focal stack efficiently covers the entire depth range. Then, I take a customized algorithm to enable a seamless refocusing experience, even in textureless regions or with moving objects. Prototype cameras are presented to capture real space-time focal stacks. There is also an interactive refocusing viewer available online at www.focalsweep.com.

Similar Papers
  • Research Article
  • Citations10

A pair of noisy/blurry patches-based PSF estimation and channel-dependent deblurring

  • Nov 01, 2011
  • IEEE Transactions on Consumer Electronics
  • Chang-Hwan Son +1
  • Book Chapter
  • Citations3

IDFD: A Dataset Annotated for Depth and Defocus

  • Jan 01, 2023
  • Saqib Nazir +4
  • Research Article
  • Citations2

Dynamic masking of truncated image segments for blind recovery of blurred images based on simulated annealing

  • May 01, 2004
  • Optical Engineering
  • Shinichi Komatsu
  • Conference Article
  • Citations4

Focal spot deconvolution using convolutional neural networks

  • Mar 01, 2019
  • Medical Imaging 2019: Physics of Medical Imaging
  • Jan Kuntz +3
  • Research Article
  • Citations5

Learning scene and blur model for active chromatic depth from defocus.

  • Oct 29, 2021
  • Applied Optics
  • Benjamin Buat +3
  • Conference Article
  • Citations3

<title>Myopic scheme for parametric blur estimation using the generalized cross-validation criterion</title>

  • Sep 22, 1998
  • Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE
  • Sophie Chardon +2
  • Conference Article

Deep Learning-Driven Depth from Defocus via Active Multispectral Quasi-Random Projections with Complex Subpatterns

  • May 01, 2018
  • Avery Ma +2
  • Research Article
  • Citations51

Passive depth estimation using chromatic aberration and a depth from defocus approach

  • Oct 09, 2013
  • Applied Optics
  • Pauline Trouvé +5
  • Conference Article
  • Citations2

Point spread function (PSF) measurement for cell phone camera with a high resolution PSF of the imaging lens and a sub-pixel digital algorithm

  • Aug 28, 2008
  • Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE
  • Changmeng Liu +1
  • Conference Article
  • Citations9

Optimized Aperture for Estimating Depth from Projector's Defocus

  • Jun 01, 2013
  • Hiroshi Kawasaki +5
  • Research Article
  • Citations35

Phase transfer and point-spread function of the human eye determined by a new asymmetric double-pass method.

  • Nov 01, 1995
  • Journal of the Optical Society of America A
  • Rafael Navarro +1
  • Research Article
  • Citations15

Increasing circular synthetic aperture sonar resolution via adapted wave atoms deconvolution.

  • Apr 01, 2017
  • The Journal of the Acoustical Society of America
  • Yan Pailhas +2
  • Conference Article
  • Citations3

Image Restoration Method for Non-uniform Blurred Images

  • Oct 01, 2018
  • Tomio Goto +4
  • Conference Article
  • Citations1

Fast blind restoration of blurred images based on local patches

  • Jan 01, 2016
  • Tomio Goto +3
  • Conference Article
  • Citations74

Identification of parameters and restoration of motion blurred images

  • Apr 23, 2006
  • R Lokhande +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.