New Perspectives in Shape Analysis (Dagstuhl Seminar 14072)

In computer vision, geometric processing and image analysis, the notation of a shape of a 3-D object has been studied either by an embedded manifold for the continuous stetting or as a collection of a discrete set of marker positions on the manifold. Within the last years, there have been many rapid developments in the field of shape representation, shape correspondence and shape manipulation with technical applications ranging from laser-range scanners to 3-D printing. Classic shape analysis tools from differential geometry have a fresh influence in the field, often powered by modern methods from optimization and numerical computing. At the same time, discrete geometric methods and related techniques such as from mathematical morphology have evolved significantly. Moreover, techniques like deep learning gained a significant influence in the development of corresponding methods and tools. New developments from tropical geometry have a high potential for use in shape analysis. The topics in our seminar addressed the sketched challenges and developments that will be useful for shape analysis. Especially we aimed to discuss the possibilities of combining fields like tropical geometry with more classical techniques as for instance from mathematical morphology. We discussed possibilities of applying machine learning methods in this context and considered recent advances from more classical fields like differential geometry and partial differential equations that can be useful for setting up and understanding shape analysis methods in all of these approaches. This seminar brought together 26 researchers from North America and Europe engaged in recent and upcoming developments in shape analysis who view these challenges from different perspectives and who together discussed the pressing open problems and novel solutions to them.

We present O-CNN , an Octree-based Convolutional Neural Network (CNN) for 3D shape analysis. Built upon the octree representation of 3D shapes, our method takes the average normal vectors of a 3D model sampled in the finest leaf octants as input and performs 3D CNN operations on the octants occupied by the 3D shape surface. We design a novel octree data structure to efficiently store the octant information and CNN features into the graphics memory and execute the entire O-CNN training and evaluation on the GPU. O-CNN supports various CNN structures and works for 3D shapes in different representations. By restraining the computations on the octants occupied by 3D surfaces, the memory and computational costs of the O-CNN grow quadratically as the depth of the octree increases, which makes the 3D CNN feasible for high-resolution 3D models. We compare the performance of the O-CNN with other existing 3D CNN solutions and demonstrate the efficiency and efficacy of O-CNN in three shape analysis tasks, including object classification, shape retrieval, and shape segmentation.

This work presents the use of Dictionary Learning methods for a sparse representation of 4D seismic data in history matching. We consider a trade-off between the number of coefficients retained in the sparse data representation, the computational cost, and how well we can capture the main features of the 4D seismic signal. K-SVD is an iterative algorithm used in Dictionary Learning problems that alternates between the calculation of the sparse representation vector and dictionary update. For the definition of the sparse representation vector, one can constrain the problem into two distinct approaches: (1) sparsity-constrained; and (2) error-constrained. We evaluated the two methods and the influence of critical parameters of the algorithm (dictionary size, number of iterations, patch size, and training dataset size) using a synthetic reservoir model. Results showed that regardless of which of the constrained approaches we used, the dictionary learning method can capture the main features of the 4D seismic signal with a sparse representation. However, the number of nonzero coefficients is highly dependent on the approach. Besides, dictionary size, number of iterations, patch size, and training dataset size, also have a significant impact on the number of coefficients and the computational cost. Consequently, the selection of these variables is crucial as it might affect the history-matching process. In a permanent-reservoir-monitoring (big data) scenario, the use of sparse representation allows us to retain the main features of the reservoir in a reasonably sized data representation that can be directly used in a history matching method, leading to improved reservoir characterization and a better understanding of the reservoir properties. This paper gives a practical implementation of a technique already used in other areas (image denoising, processing of raw seismic data, and facies representation) applied to 4D seismic data.

In this thesis, we measure 3D shapes with the use of 3D laser technology, a recent technology that combines physics, mathematics, and computer science to acquire the surface geometry of 3D shapes in the computer. We use this surface geometry to fully reconstruct real world shapes as computer models, and to analyze the characteristics and proportions of human body models for biometric purposes. Tasks that would require a tremendous effort if not done automatically by a computer.

We investigate the use of sparse principal components for representing hyperspectral imagery when performing feature selection. For conventional multispectral data with low dimensionality, dimension reduction can be achieved by using traditional feature selection techniques for producing a subset of features that provide the highest class separability, or by feature extraction techniques via linear transformation. When dealing with hyperspectral data, feature selection is a time consuming task, often requiring exhaustive search of all the feature subset combinations. Instead, feature extraction technique such as PCA is commonly used. Unfortunately, PCA usually involves non-zero linear combinations or 'loadings' of all of the data. Sparse principal components are the sets of sparse vectors spanning a low-dimensional space that explain most of the variance present in the data. Our experiments show that sparse principal components having low-dimensionality still characterize the variance in the data. Sparse data representations are generally desirable for hyperspectral images because sparse representations help in human understanding and in classification.

We used functional magnetic resonance imaging to investigate the human cortical areas involved in processing 3-dimensional (3D) shape from texture (SfT) and shading. The stimuli included monocular images of randomly shaped 3D surfaces and a wide variety of 2-dimensional (2D) controls. The results of both passive and active experiments reveal that the extraction of 3D SfT involves the bilateral caudal inferior temporal gyrus (caudal ITG), lateral occipital sulcus (LOS) and several bilateral sites along the intraparietal sulcus. These areas are largely consistent with those involved in the processing of 3D shape from motion and stereo. The experiments also demonstrate, however, that the analysis of 3D shape from shading is primarily restricted to the caudal ITG areas. Additional results from psychophysical experiments reveal that this difference in neuronal substrate cannot be explained by a difference in strength between the 2 cues. These results underscore the importance of the posterior part of the lateral occipital complex for the extraction of visual 3D shape information from all depth cues, and they suggest strongly that the importance of shading is diminished relative to other cues for the analysis of 3D shape in parietal regions.

Shape-from-shading (SFS) is an important method to reconstruct three-dimensional (3D) shape of a surface in photometry and computer vision. Lambertian surface reflectance and orthographic camera projection are two fundamental assumptions which generally result in undesirable reconstructed results since inaccurate imaging model is adopted. In this paper, we propose a new fast 3D shape reconstruction approach via the SFS method relaxing the two assumptions. To this end, the Oren-Nayar reflectance and perspective projection models are used to establish an image irradiance equation which depicts the relationship between the 3D shape of non-Lambertian surfaces and its corresponding two-dimensional (2D) shading image. Considering the light attenuation of the near point source, the image irradiance equation is transformed into a static Hamilton-Jacobi partial differential equation (PDE) by solving a quadratic equation. The viscosity solution of the resultant Hamilton-Jacobi PDE is approximated by using optimal control theory and iterative fast marching method starting from a viscosity supersolution. The performance of the proposed approach is evaluated on both synthetic and real-world images and the experimental results demonstrate that the proposed approach is accurate and fast.

With the advent of commodity 3D capturing devices and better 3D modeling tools, 3D shape content is becoming increasingly prevalent. Therefore, the need for shape retrieval algorithms to handle large-scale shape repositories is more and more important. This track provides a benchmark to evaluate large-scale 3D shape retrieval based on the ShapeNet dataset. It is a continuation of the SHREC 2016 large-scale shape retrieval challenge with a goal of measuring progress with recent developments in deep learning methods for shape retrieval. We use ShapeNet Core55, which provides more than 50 thousands models over 55 common categories in total for training and evaluating several algorithms. Eight participating teams have submitted a variety of retrieval methods which were evaluated on several standard information retrieval performance metrics. The approaches vary in terms of the 3D representation, using multi-view projections, point sets, volumetric grids, or traditional 3D shape descriptors. Overall performance on the shape retrieval task has improved significantly compared to the iteration of this competition in SHREC 2016. We release all data, results, and evaluation code for the benefit of the community and to catalyze future research into large-scale 3D shape retrieval (website: https://www.shapenet.org/shrec17).

Interpolation methods that rely on partial differential equations can reconstruct images with high quality from a few prescribed pixels. A whole class of compression codecs exploits this concept to store images in terms of a sparse grey value representation. Recently, Brinkmann et al. (2015) have suggested an alternative approach: They propose to store gradient data instead of grey values. However, this idea has not been evaluated and its potential remains unknown. In our paper, we compare gradient and grey value data for homogeneous diffusion inpainting w.r.t. two different aspects: First, we evaluate the reconstruction quality, given a comparable amount of data of both kinds. Second, we assess how well these sparse representations can be stored in compression applications. To this end, we establish a framework for optimising and encoding the known data. It allows a fair comparison of both the grey value and the gradient approach. Our evaluation shows that gradient-based reconstructions avoid visually distracting singularities involved in the reconstructions from grey values, thus improving the visual fidelity. Surprisingly, this advantage does not carry over to compression due to the high sensitivity to quantisation.

The recent advances of low-cost and mobile depth sensors dramatically extend the potential of 3D shape retrieval and analysis. While the traditional research of 3D retrieval mainly focused on searching by a rough 2D sketch or with a high-quality CAD model, we tackle a novel and challenging problem of cross-domain 3D shape retrieval, in which users can use 3D scans from low-cost depth sensors like Kinect as queries to search CAD models in the database. To cope with the imperfection of user-captured models such as model noise and occlusion, we propose a cross-domain shape retrieval framework, which minimizes the potential function of a Conditional Random Field to efficiently generate the retrieval scores. In particular, the potential function consists of two critical components: one unary potential term provides robust cross-domain partial matching and the other pairwise potential term embeds spatial structures to alleviate the instability from model noise. Both potential components are efficiently estimated using random forests with 3D local features, forming a Regression Tree Field framework. We conduct extensive experiments on two recently released user-captured 3D shape datasets and compare with several state-of-the-art approaches on the cross-domain shape retrieval task. The experimental results demonstrate that our proposed method outperforms the competing methods with a significant performance gain.

In this paper single image superresolution problem using sparse data representation is described. Image super-resolution is ill - posed inverse problem. Several methods have been proposed in the literature starting from simple interpolation techniques to learning based approach and under various regularization frame work. Recently many researchers have shown interest to super-resolve the image using sparse image representation. We slightly modified the procedure described by a similar work proposed recently. The modification suggested in the proposed approach is the method of dictionary training, feature extraction from the trained data base images and regularization. We have used singular values as prior for regularizing the ill-posed nature of the single image superresolution problem. Method of Optimal Directions algorithm (MOD) has been used in the proposed algorithm for obtaining high resolution and low resolution dictionaries from training image patches. Using the two dictionaries the given low resolution input image is superresolved. The results of the proposed algorithm showed improvements in visual, PSNR, RMSE and SSIM metrics over other similar methods.

In this paper, we propose the estimating method of reflectance properties and the recovery of shape by using Normal Sampler and Indirect Diffuse Illumination Method(IDIM). Photometric Stereo Method(PSM) is generally based on the direct illumination. PSM in this paper is modified with indirect diffuse illumination(IDI) and then applied to hybrid reflectance model which consists of two main components; Lambertian and specular reflectance. Under the indirect diffuse illumination, the reflectance properties of natural objects can be determined by using Normal Sampler that has almost all the normal components in the observed direction. The estimated reflectance properties are used to construct reference table. Also, 3-D shape of an object can be recovered from intensity distribution of a pixel and a reference table. In this paper, the reference table is used to recover the 3-D shape of an object and IDI simplifies the limited conditions of reflectance analysis for prior studies without any loss in performance. The proposed method can be applied to various types of surfaces which can be defined by hybrid reflectance.

This paper presents a shape analysis on orthogonal feature curves of 3D bald head scans with the intention of predicting scalp shape under the hair. While there are currently a number of large scale 3D head data collections available around the world, they unfortunately all suffer from hair obstruction preventing an accurate description of true scalp shape. This study is aimed at exploring the relationship between a small set of head anthropometric measurements and the feature curves of the scalp shape based on a small set of 3D bald head scans. The feature curves include scalp profile along the sagittal plane, coronal plane and a cross-sectional curve at the level of glabella.Introduction: The ever increasing availability of 3D scans of the human head has been a valuable source of information for improving head shape related equipment design and engineering with the goal of improving overall fit, sizing and comfort. However the existence of the hair prevents the actual recording and analysis of true scalp shape, and thus the analysis of scalp shape remains an elusive work. For this reason, much of the data analysis based on 3D head scans is mostly limited on facial shape and some facial landmarks. To overcome this limitation, it is desirable to be able to reconstruct the cranium shape from a few easy to obtain anthropometric measurements. In order to achieve this goal we obtained three orthogonal feature curves from 83 bald head scans and build regression equation between principal component scores of these curves and a number of head measurements such as head length, head breadth and tragion-to-top of head distance. The prediction error of these equations are evaluated. The approach: A total of 83 bald head scans taken in the 2012 anthropometric survey of US Army personnel (ANSUR II) from male soldiers are available for the analysis. From these bald head scans we identified three feature curves that principally define the scalp shape. These curves are a profile along the sagittal plane, a profile curve along the coronal plane and a cross-sectional curve at glabella level. These curves are then sampled in an equal angular space and grouped by their respective plane. The collection of each feature curves forms a shape space. We applied Principal Component Analysis (PCA) to those shape curves and decomposed the shape variation of each curve group into its respective principal axes. After conducting the PCA each curve group has a reduced dimension of 4~5 principal components that account for 95% of total variance. It shows major shape variations within each feature curve group. Then we apply multiple linear regression to the above head measurements and PCA scores of each curve group (Principal component regression or PCR). The regression equations for each curve and each PC were evaluated for their predicting power. From these regression equations the constituent feature curves can be selected from respective shape space based on head anthropometric measurements. Then PCR was also applied to the shape space of combined three curves as compared to prediction power in individual curve spaces.The results and discussion: PCR results from all orthogonal curve spaces and combined three curve space have similar predicting power as their R-squared value falling between 0.6 ~ 0.7. With a set of feature curves it is possible to further derive a 3D shape of the scalp. Although the three head measurements used in this study are effectively correlated to the first and second principal component in each group, other principal components contribute to subtle shape variations. This is an area need further analysis. We will also investigate methods to predicting 3D scalp shape from three feature curves and accuracy of the reconstruction of 3D scalp from those curves.

Over the past few years, Generative Adversarial Networks (GANs) have garnered increased interest among researchers in Computer Vision, with applications including, but not limited to, image generation, translation, imputation, and super-resolution. Nevertheless, no GAN-based method has been proposed in the literature that can successfully represent, generate or translate 3D facial shapes (meshes). This can be primarily attributed to two facts, namely that (a) publicly available 3D face databases are scarce as well as limited in terms of sample size and variability (e.g., few subjects, little diversity in race and gender), and (b) mesh convolutions for deep networks present several challenges that are not entirely tackled in the literature, leading to operator approximations and model instability, often failing to preserve high-frequency components of the distribution. As a result, linear methods such as Principal Component Analysis (PCA) have been mainly utilized towards 3D shape analysis, despite being unable to capture non-linearities and high frequency details of the 3D face—such as eyelid and lip variations. In this work, we present 3DFaceGAN, the first GAN tailored towards modeling the distribution of 3D facial surfaces, while retaining the high frequency details of 3D face shapes. We conduct an extensive series of both qualitative and quantitative experiments, where the merits of 3DFaceGAN are clearly demonstrated against other, state-of-the-art methods in tasks such as 3D shape representation, generation, and translation.

A new method for reconstructing the 3D shape of single cells in fruit