Directed acyclic graph representation of deformable models

Intrinsic image decomposition is an important topic in computer vision and computer graphics applications. However, this is a challenging problem by adopting the information of a single image. Therefore, additional priors or supplementary information such as multiply images or user interactions are necessary to address this problem. In this paper, we propose a novel scheme to use multiple images for intrinsic image decomposition, based on a similar strategy in robust principal component analysis (RPCA). RPCA utilizes the fact that the reflectance layer is common in multiple lights of a scene, and attempts to decompose the data matrix constructed from input images into a low-rank matrix and a sparse matrix. This is possible if the sparse matrix is sufficiently sparse, which is often not the case in computer vision applications. Moreover, the weighting parameter between the low-rank and sparse matrices greatly affects the accuracy of the results, and tuning this parameter can be tricky. This paper proposes a rank-constrained PCA algorithm (RCPCA) for solving background recovering problems. Fixing the rank of the low-rank matrix to be 1 allows RCPCA to better recover the low-rank matrix from the data matrix. Comprehensive tests show that RCPCA produces more stable and accurate results than RP-CA.

Deformable models are a powerful tool in both computer graphics and computer vision. The description and implementation of the deformations have to be simultaneously flexible and powerful, otherwise the technique may not satisfy the requirements of all the distinct applications. In this paper we introduce a new method for deformable model specification: deformable fields. Deformable fields are conceptually simple, lead to an easy implementation, and are suitable for adaptive models. We apply our new technique to describe an adaptive deformable face, and compare three different adaptation strategies. Additionally, we show how our technique is suitable to describe different individuals.

Over the last few years energy minimization has emerged as an indispensable tool in computer vision. The primary reason for this rising popularity has been the successes of efficient graph cut based minimization algorithms in solving many low level vision problems such as image segmentation, object reconstruction, image restoration and disparity estimation. The scale and form of computer vision problems introduce many challenges in energy minimization. In this chapter we address the problem of efficient and exact minimization of groups of similar functions which are known to be solvable in polynomial time. We will present a novel dynamic algorithm for minimizing such functions. This algorithm reuses computation from previous problem instances to solve new instances resulting in a substantial improvement in the running time. We will present the results of using this approach on the problems of interactive image segmentation, image segmentation in video, human pose estimation and segmentation, and measuring uncertainty of solutions obtained by minimizing energy functions.

We consider the flatland or 2D properties of the light field generated when a homogeneous convex curved surface reflects a distant illumination field. Besides being of considerable theoretical interest, this problem has applications in computer vision and graphics - for instance, in determining lighting and bidirectional reflectance distribution functions (BRDFs), in rendering environment maps, and in image-based rendering. We demonstrate that the integral for the reflected light transforms to a simple product of coefficients in Fourier space. Thus, the operation of rendering can be viewed in simple signal processing terms as a filtering operation that convolves the incident illumination with the BRDF. This analysis leads to a number of interesting observations for computer graphics, computer vision, and visual perception.

Physically-based modeling: past, present, and future

In this chapter, we introduce basic 2D digital geometry. The main topic in 2D geometry is curves. A 2D digital curve is a simple path in \(\Sigma_{2}\). A simple closed digital curve is usually the boundary of a connected component. We first discuss how we precisely define a curve in a graph and Euclidean space, then we discuss how we represent digital curves. Digital curves have two important applications in computer graphics and computer vision: (a) Construction of a digital line when two end points are given, and (b) Determination of a closed digital curve to identify a connected region in computer vision. At the end of this chapter, we present two classic theorems related to 2D digital geometry: Pick’s theorem and Minkowski’s theorem. In addition, we discuss the basic concept of image segmentation, one of the major applications of 2D digital planes.

In the arts and sciences, as well as in our daily lives, symmetry has made a profound and lasting impact. Likewise, a computational treatment of symmetry and group theory (the ultimate mathematical formalization of symmetry) has the potential to play an important role in computational sciences. Though the term computational symmetry was formally defined a decade ago by the first author, referring to algorithmic treatment of symmetries, seeking symmetry from digital data has been attempted for over four decades. Computational symmetry on real world data turns out to be challenging enough that, after decades of effort, a fully automated symmetry–savvy system remains elusive for real world applications. The recent resurging interests in computational symmetry for computer vision and computer graphics applications have shown promising results. Recognizing the fundamental relevance and potential power that computational symmetry affords, we offer this survey to the computer vision and computer graphics communities. This survey provides a succinct summary of the relevant mathematical theory, a historic perspective of some important symmetry-related ideas, a partial yet timely report on the state of the arts symmetry detection algorithms along with its first quantitative benchmark, a diverse set of real world applications, suggestions for future directions and a comprehensive reference list.

Image mosaicing is the act of combining two or more images and is used in many applications in computer vision, image processing, and computer graphics. It aims to combine images such that no obstructive boundaries exist around overlapped regions and to create a mosaic image that exhibits as little distortion as possible from the original images. Most of the existing algorithms are the computationally complex and don't show good results always in obtaining of the stitched images, which are different: scale, light, various free points of view and others. In this paper we consider an algorithm which allows increasing the speed of processing in the case of stitching high-resolution images. We reduced the computational complexity used an edge image analysis and saliency map on high-detailisation areas. On detected areas are determined angles of rotation, scaling factors, the coefficients of the color correction and transformation matrix. We define key points using SURF detector and ignore false correspondences based on correlation analysis. The proposed algorithm allows to combine images from free points of view with the different color balances, time shutter and scale. We perform a comparative study and show that statistically, the new algorithm deliver good quality results compared to existing algorithms.

Surface reconstruction from point clouds has been a well-studied research topic with applications in computer vision and computer graphics. Recently, several learningbased methods were proposed for 3D shape representation through implicit functions which among others can be used for point cloud-based reconstruction. Although delivering compelling results for synthetic object datasets of overseeable size, they fail to represent larger scenes accurately, presumably due to the use of only one global latent code for encoding an entire scene or object. We propose to encode only parts of objects with features attached to unstructured point clouds. To this end we use a hierarchical feature map in 3D space, extracted from the input point clouds, with which local latent shape encodings can be queried at arbitrary positions. We use a permutohedral lattice to process the hierarchical feature maps sparsely and efficiently. This enables accurate and detailed point cloud-based reconstructions for large amounts of points in a time-efficient manner, showing good generalization capabilities across different datasets. Experiments on synthetic and real world datasets demonstrate the reconstruction capability of our method and compare favorably to state-of-the-art methods.

This chapter proposes a novel approach for the calibration of a panoramic camera using geometric information available in real scenes. Panoramic cameras are of increasing importance for various applications in computer vision, computer graphics or robotics. Previously developed camera calibration methods (for 'standard' camera architectures following the pinhole camera model) are not applicable due to the non-linearity of the panoramic camera, defined by the existence of multiple (nonlinear) optical centers and a cylindrical image manifold. This article addresses the calibration subject of a more general yet flexible panoramic camera model for the first time. The chapter focuses on the calibration of two dominant parameters that characterize the camera model and provide flexibility in selecting different types of panoramas such as single-center (e.g. as assumed for QTVR), symmetric stereo, concentric or polycentric panoramas. We elaborate selected geometric constraints (for increasing numerical stability) with the corresponding solutions; summarize the experimental results with captured image data, and discuss the performance of different geometric constraints via error-sensitivity simulation and analysis.

The problem of measuring geometrical distortions is not trivial for volumetric domains. There exist intrinsic restrictions and constrains on higher dimensional mappings. Moreover, according to Liouville theorem, most existing techniques for 2D data can not be directly applied to volumetric objects. In this work we approximate continuous deformations by piecewise affine functions defined on tetrahedral meshes. Our aim is to study a few types of geometrical distortions that can be expressed as functions of singular values of a Jacobian. We employ the proposed methods of estimating conformal and isometric distortions to analyze volumetric data. In particular, we examine parametrization of tetrahedral models to a ball. Distortions produced by the resulting spatial mappings depict intrinsic structure of domains, and therefore can be employed in such tasks as detection of abnormalities and comparison (i.e. similarity assessment) between 3D objects. This geometric approach and results are highly relevant to various applications in Computer Vision, Computer Graphics, 3D Printing and Medical Imaging.

Triangular mesh is one of the most popular 3D modalities which usage spans a wide variety of application in computer vision, computer graphics and multimedia. Raw mesh surfaces generated by 3D scanning devices often suffer from mesh irregularity and also lacks the implicit ordered structure that characterizes 2D images. Therefore they are not suitable to be processed as such, especially for applications requiring fine surface analysis tasks. In this paper, we propose CSIOR, a novel mesh regularization technique exhibiting novel capabilities including a quasi-equilateral triangle tessellation, and structured and ordered tessellation that can accommodate non-convex shaped surfaces, implicit generation of iso-geodesic contours and preservation of the geometric texture on the mesh surface. We evidence the superiority of our technique over current methods through a series of experiments performed on a variety of geometric textured surfaces.

Local image histograms contain a great deal of information useful for applications in computer graphics, computer vision and computational photography. Making use of that information has been challenging because of the expense of computing histogram properties over large neighborhoods. Efficient algorithms exist for some specific computations like the bilateral filter, but not others. Here, we present an efficient and practical method for computing accurate derivatives and integrals of locally-weighted histograms over large neighborhoods. The method allows us to compute the location, height, width and integral of all local histogram modes at interactive rates. Among other things, it enables the first constant-time isotropic median filter, robust isotropic image morphology operators, an efficient "dominant mode" filter and a non-iterative alternative to the mean shift. In addition, we present a method to combat the over-sharpening that is typical of histogram-based edge-preserving smoothing. This post-processing step should make histogram-based filters not only fast and efficient, but also suitable for a variety of new applications.

Modern operating rooms are highly complex, safety-critical environments in which surgeons must maintain sustained concentration and an uninterrupted workflow to ensure optimal patient outcomes. Despite significant technological advances, several fundamental challenges remain insufficiently addressed. One such challenge is achieving optimal and consistent illumination of the surgical field. In current operating rooms, surgical lights must often be manually repositioned to compensate for shadows cast by medical personnel, leading to frequent workflow interruptions and increased cognitive load. Another persistent challenge is providing surgeons with immediate and non-distracting access to critical information. As the amount of information required during surgery continues to grow, presenting this information on remote monitors can disrupt workflow and divert attention away from the surgical field. Both interruptions can compromise safety and efficiency. In this dissertation, I address both challenges by developing algorithms based on multi-view depth camera technology that enable shadow-free illumination of the surgical field and shadow-free projection of information adjacent to it. For shadow-free illumination, I propose algorithms for an autonomous surgical lighting system that detect occlusions caused by medical personnel and dynamically control ceiling-mounted light modules to ensure optimal illumination. For shadow-free projection, I develop algorithms for a multi-projector system that displays information directly next to the surgical site. While multiple projectors are employed to facilitate shadow-free projection, my algorithms specifically enable precise geometric distortion correction on dynamic, uneven surfaces. In addition, I present a blur-mitigation strategy that significantly reduces ghosting artifacts caused by overlapping projections and unavoidable minor errors in dynamic geometry reconstruction. Beyond the medical application itself, I developed novel, general-purpose multi-view point cloud algorithms that enable these systems: (a) a fast, accurate, and robust multi-depth-sensor registration method that allows rapid registration between surgeries while complying with data privacy constraints; (b) BlendPCR, a real-time multi-camera point cloud rendering technique that preserves the native sensor resolution while enabling seamless visualization and precise geometric correction for projection onto dynamic, deformable surfaces; and (c) TemPCC, an experimental approach for temporal point cloud completion. I demonstrate that the autonomous surgical lighting system significantly improves illumination quality and shadow reduction in a simulated surgical environment, compared to conventional surgical lights, based on validation with practicing surgeons (n = 11). Furthermore, the multi-projector system enables shadow-free projection onto dynamic, deformable surfaces with a visual accuracy of 1–2 mm using point cloud–based rendering, while the proposed blur mitigation pipeline significantly reduces blurring and ghosting (n = 23). Beyond the surgical domain, my proposed multi-depth-sensor registration method attains an accuracy of 1.6 mm at a distance of 2 m without requiring RGB or infrared data, and could therefore be adapted to LiDAR-based setups. My multi-point cloud rendering technique, BlendPCR, enables low-latency, real-time surface reconstruction and rendering. It achieves frame rates exceeding 200 fps at ultra-high-definition resolution while processing data from seven RGB-D sensors simultaneously, making it suitable for live virtual reality and telepresence scenarios. In summary, my dissertation establishes a technical and algorithmic foundation for reducing distractions in the operating room while also contributing general-purpose methods for multi-view point cloud registration, rendering, and completion, providing a basis for future research and applications in computer graphics and computer vision.

Real-time three-dimensional acquisition of real-world scenes has many important applications in computer graphics, computer vision and human-computer interaction. Inexpensive depth sensors such as the Microsoft Kinect allow to leverage the development of such applications. However, this technology is still relatively recent, and no detailed studies on its scalability to dense and view-independent acquisition have been reported. This paper addresses the question of what can be done with a larger number of Kinects used simultaneously. We describe an interference-reducing physical setup, a calibration procedure and an extension to the KinectFusion algorithm, which allows to produce high quality volumetric reconstructions from multiple Kinects whilst overcoming systematic errors in the depth measurements. We also report on enhancing image based visual hull rendering by depth measurements, and compare the results to KinectFusion. Our system provides practical insight into achievable spatial and radial range and into bandwidth requirements for depth data acquisition. Finally, we present a number of practical applications of our system.