OmniKinect

Deformable models are a useful tool in computer vision and computer graphics. A deformable model is a curve (in two dimensions) or a surface (in three dimensions), whose shape, position, and orientation are controlled through a set of parameters. Deformable models can represent manufactured objects, human faces and skeletons, and even bodies of fluid. In computer graphics, we use deformable models for animations and simulations, whereas in computer vision applications, such as tracking and fitting, deformable models help to restrict the family of possible solutions. We introduce the use of a directed acyclic graph (DAG) to describe the position and Jacobian of each point on the surface of deformable models. This data structure, combined with a topological description of the points, is simple, powerful, and extremely useful for both computer vision and computer graphics applications. We show a computer vision application, 3D deformable face tracking, and a computer graphics application, cyberglove data visualization and calibration.

In this chapter, we introduce basic 2D digital geometry. The main topic in 2D geometry is curves. A 2D digital curve is a simple path in \(\Sigma_{2}\). A simple closed digital curve is usually the boundary of a connected component. We first discuss how we precisely define a curve in a graph and Euclidean space, then we discuss how we represent digital curves. Digital curves have two important applications in computer graphics and computer vision: (a) Construction of a digital line when two end points are given, and (b) Determination of a closed digital curve to identify a connected region in computer vision. At the end of this chapter, we present two classic theorems related to 2D digital geometry: Pick’s theorem and Minkowski’s theorem. In addition, we discuss the basic concept of image segmentation, one of the major applications of 2D digital planes.

There has been much research on computer vision in last three decades. Computer vision methods have been developed for different situations. One example is a detection of human face. For computers face detection is hard. Faces look different from different viewing directions. Facial expressions affect to the look of the face. Each individual person has a unique face. The lightning conditions can vary and so on.However, face detection is currently possible in limited conditions. In addition, there are some methods that can be used for gender recognition [3], face recognition [5] and facial expression recognition [2].Nonetheless, there has been very little research on how to combine these methods. There has also been quite little research on how to apply these methods in human-computer interaction (HCI).Finding sets of techniques that complement each other in a useful way is one research challenge. There are some applications that take advantage of one or two computer vision techniques. For example, Christian and Avery [1] have developed an information kiosk that uses computer vision to detect potential users from a distance. A similar kiosk has been developed by us in the University of Tampere [4]. There are also some games that use simple computer vision techniques for the interaction. However, there are very few applications that use several computer vision techniques together such as face detection, facial expression recognition and gender recognition. Overall, there has been very little effort in combining different techniques.In my research I develop computer vision methods and combine them, so that the combined method can detect face, recognize gender and facial expressions. After successfully combining the methods, it is easier to develop HCI applications that take advantage of computer vision. Applications that can be used by small group of people are my specific interest. These applications allow me to build adaptive user interfaces and analyze the use of computer vision techniques in improving human-computer interaction.

Intrinsic image decomposition is an important topic in computer vision and computer graphics applications. However, this is a challenging problem by adopting the information of a single image. Therefore, additional priors or supplementary information such as multiply images or user interactions are necessary to address this problem. In this paper, we propose a novel scheme to use multiple images for intrinsic image decomposition, based on a similar strategy in robust principal component analysis (RPCA). RPCA utilizes the fact that the reflectance layer is common in multiple lights of a scene, and attempts to decompose the data matrix constructed from input images into a low-rank matrix and a sparse matrix. This is possible if the sparse matrix is sufficiently sparse, which is often not the case in computer vision applications. Moreover, the weighting parameter between the low-rank and sparse matrices greatly affects the accuracy of the results, and tuning this parameter can be tricky. This paper proposes a rank-constrained PCA algorithm (RCPCA) for solving background recovering problems. Fixing the rank of the low-rank matrix to be 1 allows RCPCA to better recover the low-rank matrix from the data matrix. Comprehensive tests show that RCPCA produces more stable and accurate results than RP-CA.

In the arts and sciences, as well as in our daily lives, symmetry has made a profound and lasting impact. Likewise, a computational treatment of symmetry and group theory (the ultimate mathematical formalization of symmetry) has the potential to play an important role in computational sciences. Though the term computational symmetry was formally defined a decade ago by the first author, referring to algorithmic treatment of symmetries, seeking symmetry from digital data has been attempted for over four decades. Computational symmetry on real world data turns out to be challenging enough that, after decades of effort, a fully automated symmetry–savvy system remains elusive for real world applications. The recent resurging interests in computational symmetry for computer vision and computer graphics applications have shown promising results. Recognizing the fundamental relevance and potential power that computational symmetry affords, we offer this survey to the computer vision and computer graphics communities. This survey provides a succinct summary of the relevant mathematical theory, a historic perspective of some important symmetry-related ideas, a partial yet timely report on the state of the arts symmetry detection algorithms along with its first quantitative benchmark, a diverse set of real world applications, suggestions for future directions and a comprehensive reference list.

We consider the flatland or 2D properties of the light field generated when a homogeneous convex curved surface reflects a distant illumination field. Besides being of considerable theoretical interest, this problem has applications in computer vision and graphics - for instance, in determining lighting and bidirectional reflectance distribution functions (BRDFs), in rendering environment maps, and in image-based rendering. We demonstrate that the integral for the reflected light transforms to a simple product of coefficients in Fourier space. Thus, the operation of rendering can be viewed in simple signal processing terms as a filtering operation that convolves the incident illumination with the BRDF. This analysis leads to a number of interesting observations for computer graphics, computer vision, and visual perception.

Local image histograms contain a great deal of information useful for applications in computer graphics, computer vision and computational photography. Making use of that information has been challenging because of the expense of computing histogram properties over large neighborhoods. Efficient algorithms exist for some specific computations like the bilateral filter, but not others. Here, we present an efficient and practical method for computing accurate derivatives and integrals of locally-weighted histograms over large neighborhoods. The method allows us to compute the location, height, width and integral of all local histogram modes at interactive rates. Among other things, it enables the first constant-time isotropic median filter, robust isotropic image morphology operators, an efficient "dominant mode" filter and a non-iterative alternative to the mean shift. In addition, we present a method to combat the over-sharpening that is typical of histogram-based edge-preserving smoothing. This post-processing step should make histogram-based filters not only fast and efficient, but also suitable for a variety of new applications.

Modern operating rooms are highly complex, safety-critical environments in which surgeons must maintain sustained concentration and an uninterrupted workflow to ensure optimal patient outcomes. Despite significant technological advances, several fundamental challenges remain insufficiently addressed. One such challenge is achieving optimal and consistent illumination of the surgical field. In current operating rooms, surgical lights must often be manually repositioned to compensate for shadows cast by medical personnel, leading to frequent workflow interruptions and increased cognitive load. Another persistent challenge is providing surgeons with immediate and non-distracting access to critical information. As the amount of information required during surgery continues to grow, presenting this information on remote monitors can disrupt workflow and divert attention away from the surgical field. Both interruptions can compromise safety and efficiency. In this dissertation, I address both challenges by developing algorithms based on multi-view depth camera technology that enable shadow-free illumination of the surgical field and shadow-free projection of information adjacent to it. For shadow-free illumination, I propose algorithms for an autonomous surgical lighting system that detect occlusions caused by medical personnel and dynamically control ceiling-mounted light modules to ensure optimal illumination. For shadow-free projection, I develop algorithms for a multi-projector system that displays information directly next to the surgical site. While multiple projectors are employed to facilitate shadow-free projection, my algorithms specifically enable precise geometric distortion correction on dynamic, uneven surfaces. In addition, I present a blur-mitigation strategy that significantly reduces ghosting artifacts caused by overlapping projections and unavoidable minor errors in dynamic geometry reconstruction. Beyond the medical application itself, I developed novel, general-purpose multi-view point cloud algorithms that enable these systems: (a) a fast, accurate, and robust multi-depth-sensor registration method that allows rapid registration between surgeries while complying with data privacy constraints; (b) BlendPCR, a real-time multi-camera point cloud rendering technique that preserves the native sensor resolution while enabling seamless visualization and precise geometric correction for projection onto dynamic, deformable surfaces; and (c) TemPCC, an experimental approach for temporal point cloud completion. I demonstrate that the autonomous surgical lighting system significantly improves illumination quality and shadow reduction in a simulated surgical environment, compared to conventional surgical lights, based on validation with practicing surgeons (n = 11). Furthermore, the multi-projector system enables shadow-free projection onto dynamic, deformable surfaces with a visual accuracy of 1–2 mm using point cloud–based rendering, while the proposed blur mitigation pipeline significantly reduces blurring and ghosting (n = 23). Beyond the surgical domain, my proposed multi-depth-sensor registration method attains an accuracy of 1.6 mm at a distance of 2 m without requiring RGB or infrared data, and could therefore be adapted to LiDAR-based setups. My multi-point cloud rendering technique, BlendPCR, enables low-latency, real-time surface reconstruction and rendering. It achieves frame rates exceeding 200 fps at ultra-high-definition resolution while processing data from seven RGB-D sensors simultaneously, making it suitable for live virtual reality and telepresence scenarios. In summary, my dissertation establishes a technical and algorithmic foundation for reducing distractions in the operating room while also contributing general-purpose methods for multi-view point cloud registration, rendering, and completion, providing a basis for future research and applications in computer graphics and computer vision.

Image mosaicing is the act of combining two or more images and is used in many applications in computer vision, image processing, and computer graphics. It aims to combine images such that no obstructive boundaries exist around overlapped regions and to create a mosaic image that exhibits as little distortion as possible from the original images. Most of the existing algorithms are the computationally complex and don't show good results always in obtaining of the stitched images, which are different: scale, light, various free points of view and others. In this paper we consider an algorithm which allows increasing the speed of processing in the case of stitching high-resolution images. We reduced the computational complexity used an edge image analysis and saliency map on high-detailisation areas. On detected areas are determined angles of rotation, scaling factors, the coefficients of the color correction and transformation matrix. We define key points using SURF detector and ignore false correspondences based on correlation analysis. The proposed algorithm allows to combine images from free points of view with the different color balances, time shutter and scale. We perform a comparative study and show that statistically, the new algorithm deliver good quality results compared to existing algorithms.

Surface reconstruction from point clouds has been a well-studied research topic with applications in computer vision and computer graphics. Recently, several learningbased methods were proposed for 3D shape representation through implicit functions which among others can be used for point cloud-based reconstruction. Although delivering compelling results for synthetic object datasets of overseeable size, they fail to represent larger scenes accurately, presumably due to the use of only one global latent code for encoding an entire scene or object. We propose to encode only parts of objects with features attached to unstructured point clouds. To this end we use a hierarchical feature map in 3D space, extracted from the input point clouds, with which local latent shape encodings can be queried at arbitrary positions. We use a permutohedral lattice to process the hierarchical feature maps sparsely and efficiently. This enables accurate and detailed point cloud-based reconstructions for large amounts of points in a time-efficient manner, showing good generalization capabilities across different datasets. Experiments on synthetic and real world datasets demonstrate the reconstruction capability of our method and compare favorably to state-of-the-art methods.

This chapter proposes a novel approach for the calibration of a panoramic camera using geometric information available in real scenes. Panoramic cameras are of increasing importance for various applications in computer vision, computer graphics or robotics. Previously developed camera calibration methods (for 'standard' camera architectures following the pinhole camera model) are not applicable due to the non-linearity of the panoramic camera, defined by the existence of multiple (nonlinear) optical centers and a cylindrical image manifold. This article addresses the calibration subject of a more general yet flexible panoramic camera model for the first time. The chapter focuses on the calibration of two dominant parameters that characterize the camera model and provide flexibility in selecting different types of panoramas such as single-center (e.g. as assumed for QTVR), symmetric stereo, concentric or polycentric panoramas. We elaborate selected geometric constraints (for increasing numerical stability) with the corresponding solutions; summarize the experimental results with captured image data, and discuss the performance of different geometric constraints via error-sensitivity simulation and analysis.

The problem of measuring geometrical distortions is not trivial for volumetric domains. There exist intrinsic restrictions and constrains on higher dimensional mappings. Moreover, according to Liouville theorem, most existing techniques for 2D data can not be directly applied to volumetric objects. In this work we approximate continuous deformations by piecewise affine functions defined on tetrahedral meshes. Our aim is to study a few types of geometrical distortions that can be expressed as functions of singular values of a Jacobian. We employ the proposed methods of estimating conformal and isometric distortions to analyze volumetric data. In particular, we examine parametrization of tetrahedral models to a ball. Distortions produced by the resulting spatial mappings depict intrinsic structure of domains, and therefore can be employed in such tasks as detection of abnormalities and comparison (i.e. similarity assessment) between 3D objects. This geometric approach and results are highly relevant to various applications in Computer Vision, Computer Graphics, 3D Printing and Medical Imaging.

Triangular mesh is one of the most popular 3D modalities which usage spans a wide variety of application in computer vision, computer graphics and multimedia. Raw mesh surfaces generated by 3D scanning devices often suffer from mesh irregularity and also lacks the implicit ordered structure that characterizes 2D images. Therefore they are not suitable to be processed as such, especially for applications requiring fine surface analysis tasks. In this paper, we propose CSIOR, a novel mesh regularization technique exhibiting novel capabilities including a quasi-equilateral triangle tessellation, and structured and ordered tessellation that can accommodate non-convex shaped surfaces, implicit generation of iso-geodesic contours and preservation of the geometric texture on the mesh surface. We evidence the superiority of our technique over current methods through a series of experiments performed on a variety of geometric textured surfaces.

Spectral reflectance is defined as the "fingerprint" of an object and is illumination invariant. It has many applications in color reproduction, imaging, computer vision, and computer graphics. In previous reflectance reconstruction methods, spectral reflectance has been treated equally over the whole wavelength. However, human eyes or sensors in an imaging device usually have different weights over different wavelengths. We propose a novel method to reconstruct reflectance, considering a wavelength-sensitive function (WSF) that is constructed from sensor-sensitive functions (or color matching functions). Our main idea is to achieve more accurate reconstruction at wavelengths where sensors have high sensitivities. This more accurate reconstruction can achieve better imaging or color reproduction performance. In our method, we generate a matrix through the Hadamard product of the reflectance matrix and the WSF matrix. We then obtain reconstructed reflectance by applying the singular value decomposition on the generated matrix. The experimental results show that our method can reduce 47% mean-square error and 55% Lab error compared with the classical principal component analysis method.

Generative models for 3D geometric data arise in many important applications in 3D computer vision and graphics. In this paper, we focus on 3D deformable shapes that share a common topological structure, such as human faces and bodies. Morphable Models and their variants, despite their linear formulation, have been widely used for shape representation, while most of the recently proposed nonlinear approaches resort to intermediate representations, such as 3D voxel grids or 2D views. In this work, we introduce a novel graph convolutional operator, acting directly on the 3D mesh, that explicitly models the inductive bias of the fixed underlying graph. This is achieved by enforcing consistent local orderings of the vertices of the graph, through the spiral operator, thus breaking the permutation invariance property that is adopted by all the prior work on Graph Neural Networks. Our operator comes by construction with desirable properties (anisotropic, topology-aware, lightweight, easy-to-optimise), and by using it as a building block for traditional deep generative architectures, we demonstrate state-of-the-art results on a variety of 3D shape datasets compared to the linear Morphable Model and other graph convolutional operators.