Harmonic Beltrami Signature: A Novel 2D Shape Representation for Object Classification

Shape analysis is a central problem in the field of computer vision. In 2D shape analysis, classification and recognition of objects from their observed silhouettes are extremely crucial but difficult. It usually involves an efficient representation of 2D shape space with a metric, so that its mathematical structure can be used for further analysis. Although the study of 2D simply-connected shapes has been subject to a corpus of literatures, the analysis of multiply-connected shapes is comparatively less studied. In this work, we propose a representation for general 2D multiply-connected domains with arbitrary topologies using conformal welding. A metric can be defined on the proposed representation space, which gives a metric to measure dissimilarities between objects. The main idea is to map the exterior and interior of the domain conformally to unit disks and circle domains (unit disk with several inner disks removed), using holomorphic 1-forms. A set of diffeomorphisms of the unit circle S(1) can be obtained, which together with the conformal modules are used to define the shape signature. A shape distance between shape signatures can be defined to measure dissimilarities between shapes. We prove theoretically that the proposed shape signature uniquely determines the multiply-connected objects under suitable normalization. We also introduce a reconstruction algorithm to obtain shapes from their signatures. This completes our framework and allows us to move back and forth between shapes and signatures. With that, a morphing algorithm between shapes can be developed through the interpolation of the Beltrami coefficients associated with the signatures. Experiments have been carried out on shapes extracted from real images. Results demonstrate the efficacy of our proposed algorithm as a stable shape representation scheme.

This paper presents a shape analysis on orthogonal feature curves of 3D bald head scans with the intention of predicting scalp shape under the hair. While there are currently a number of large scale 3D head data collections available around the world, they unfortunately all suffer from hair obstruction preventing an accurate description of true scalp shape. This study is aimed at exploring the relationship between a small set of head anthropometric measurements and the feature curves of the scalp shape based on a small set of 3D bald head scans. The feature curves include scalp profile along the sagittal plane, coronal plane and a cross-sectional curve at the level of glabella.Introduction: The ever increasing availability of 3D scans of the human head has been a valuable source of information for improving head shape related equipment design and engineering with the goal of improving overall fit, sizing and comfort. However the existence of the hair prevents the actual recording and analysis of true scalp shape, and thus the analysis of scalp shape remains an elusive work. For this reason, much of the data analysis based on 3D head scans is mostly limited on facial shape and some facial landmarks. To overcome this limitation, it is desirable to be able to reconstruct the cranium shape from a few easy to obtain anthropometric measurements. In order to achieve this goal we obtained three orthogonal feature curves from 83 bald head scans and build regression equation between principal component scores of these curves and a number of head measurements such as head length, head breadth and tragion-to-top of head distance. The prediction error of these equations are evaluated. The approach: A total of 83 bald head scans taken in the 2012 anthropometric survey of US Army personnel (ANSUR II) from male soldiers are available for the analysis. From these bald head scans we identified three feature curves that principally define the scalp shape. These curves are a profile along the sagittal plane, a profile curve along the coronal plane and a cross-sectional curve at glabella level. These curves are then sampled in an equal angular space and grouped by their respective plane. The collection of each feature curves forms a shape space. We applied Principal Component Analysis (PCA) to those shape curves and decomposed the shape variation of each curve group into its respective principal axes. After conducting the PCA each curve group has a reduced dimension of 4~5 principal components that account for 95% of total variance. It shows major shape variations within each feature curve group. Then we apply multiple linear regression to the above head measurements and PCA scores of each curve group (Principal component regression or PCR). The regression equations for each curve and each PC were evaluated for their predicting power. From these regression equations the constituent feature curves can be selected from respective shape space based on head anthropometric measurements. Then PCR was also applied to the shape space of combined three curves as compared to prediction power in individual curve spaces.The results and discussion: PCR results from all orthogonal curve spaces and combined three curve space have similar predicting power as their R-squared value falling between 0.6 ~ 0.7. With a set of feature curves it is possible to further derive a 3D shape of the scalp. Although the three head measurements used in this study are effectively correlated to the first and second principal component in each group, other principal components contribute to subtle shape variations. This is an area need further analysis. We will also investigate methods to predicting 3D scalp shape from three feature curves and accuracy of the reconstruction of 3D scalp from those curves.

Over the last decade, it has become increasingly affordable to digitize 2D and 3D shape information using multiple modalities, such as (video) cameras, image-based reconstruction systems, laser-range scanners, or depth cameras. If these dense models can be processed and described in an efficient and informative way, they can be used in applications, such as ergonomic design, virtual shopping, scientific and medical visualization, realistic simulation, photo-realistic rendering, the design of natural user interfaces, and semantic scene understanding. Traditionally, the notion of shape has been studied either by analyzing projections of shapes in images or by analyzing a sparse set of marker positions on 3D shapes. Typical tasks in 2D shape analysis include segmenting objects in images and tracking objects across a sequence of images, and typical tasks in 3D shape analysis include reconstructing the three-dimensional object depth from input images and identifying corresponding points on different 3D models. The analysis and processing of shape data becomes especially challenging because of the increasing amount of data captured by sensors used to acquire shapes, and because modern applications such as natural user interfaces require real-time processing of the input shapes. Meeting these challenges requires models of shape analysis that are compact and informative, thereby allowing the development of algorithms that can process large datasets efficiently. To achieve these goals, interdisciplinary approaches are needed that use concepts from a variety of research areas, including numerical computing, differential geometry, deformable shape modeling, sparse data representation, and machine learning. On the algorithmic side, many shape analysis tasks are modeled using partial differential equations, which can be solved using tools from the field of numerical computing. The fields of differential geometry and deformable shape modeling have recently begun to influence shape analysis methods. Furthermore, tools from the field of sparse representations, which aim to describe input data using a compressible representation with respect to a set of carefully selected basis elements, have the potential to significantly reduce the amount of data that needs to be processed in shape analysis tasks. The related field of machine learning offers similar potential. This seminar brought together 28 researchers from North America and Europe engaged in recent and upcoming developments in shape analysis who view these challenges from different perspectives and who together discussed the pressing open problems and novel solutions to them.

In shape analysis, finding an optimal 1-1 correspondence between surfaces within a large class of admissible bijective mappings is of great importance. Such process is called surface registration. The difficulty lies in the fact that the space of all surface diffeomorphisms is a complicated functional space, making exhaustive search for the best mapping challenging. To tackle this problem, we propose a simple representation of bijective surface maps using Beltrami coefficients (BCs), which are complex-valued functions defined on surfaces with supreme norm less than 1. Fixing any 3 points on a pair of surfaces, there is a 1-1 correspondence between the set of surface diffeomorphisms between them and the set of BCs. Hence, every bijective surface map can be represented by a unique BC. Conversely, given a BC, we can reconstruct the unique surface map associated to it using the Beltrami Holomorphic flow (BHF) method. Using BCs to represent surface maps is advantageous because it is a much simpler functional space, which captures many essential features of a surface map. By adjusting BCs, we equivalently adjust surface diffeomorphisms to obtain the optimal map with desired properties. More specifically, BHF gives us the variation of the associated map under the variation of BC. Using this, a variational problem over the space of surface diffeomorphisms can be easily reformulated into a variational problem over the space of BCs. This makes the minimization procedure much easier. More importantly, the diffeomorphic property is always preserved. We test our method on synthetic examples and real medical applications. Experimental results demonstrate the effectiveness of our proposed algorithm for surface registration.

The fact that the physical shapes of man-made objects are subject to overlapping influences—such as technological, economic, geographic, and stylistic progressions—holds great information potential. On the other hand, it is also a major analytical challenge to uncover these overlapping trends and to disentagle them in an unbiased way. This article explores a novel mathematical approach to extract archaeological insights from ensembles of similar artifact shapes. We show that by considering all shape information in a find collection , it is possible to identify shape patterns that would be difficult to discern by considering the artifacts individually or by classifying shapes into predefined archaeological types and analyzing the associated distinguishing characteristics. Recently, a series of high-resolution digital representations of artifacts have become available. Such datasets enable the application of extremely sensitive and flexible methods of shape analysis. We explore this potential on a set of 3D models of ancient Greek and Roman sundials, with the aim of providing alternatives to the traditional archaeological method of “trend extraction by ordination” (typology). In the proposed approach, each 3D shape is represented as a point in a shape space —a high-dimensional, curved, non-Euclidean space. Proper consideration of its mathematical properties reduces bias in data analysis and thus improves analytical power. By performing regression in shape space, we find that for Roman sundials, the bend of the shadow-receiving surface of the sundials changes with the latitude of the location. This suggests that, apart from the inscribed hour lines, also a sundial’s shape was adjusted to the place of installation. As an example of more advanced inference, we use the identified trend to infer the latitude at which a sundial, whose location of installation is unknown, was placed. We also derive a novel method for differentiated morphological trend assertion, building upon and extending the theory of geometric statistics and shape analysis. Specifically, we present a regression-based method for statistical normalization of shapes that serves as a means of disentangling parameter-dependent effects (trends) and unexplained variability . In addition, we show that this approach is robust to noise in the digital reconstructions of the artifact shapes.

We consider the problem of constructing quasi-conformal mappings between surfaces by solving Beltrami equations. This is of great importance for shape registration.In the physical world, most surface deformations can be rigorously modeled as quasi-conformal maps. The local deformation is characterized by a complex-value function, Beltrami coefficient, which describes the deviation from conformality of the deformation at each point.We propose an effective algorithm to solve the quasi-conformal map from the Beltrami coefficient. The major strategy is to deform the conformal structure of the original surface to a new conformal structure by the Beltrami coefficient, such that the quasi-conformal map becomes a conformal map. By using holomorphic differential forms, conformal maps under the new conformal structure are calculated, which are the desired quasi-conformal maps.The efficiency and efficacy of the algorithms are demonstrated by experimental results. Furthermore, the algorithms are robust for surfaces scanned from real life, and general for surfaces with different topologies.KeywordsQuasic-Conformal MapBeltrami EquationRiemannian MetricUniformization

We used functional magnetic resonance imaging to investigate the human cortical areas involved in processing 3-dimensional (3D) shape from texture (SfT) and shading. The stimuli included monocular images of randomly shaped 3D surfaces and a wide variety of 2-dimensional (2D) controls. The results of both passive and active experiments reveal that the extraction of 3D SfT involves the bilateral caudal inferior temporal gyrus (caudal ITG), lateral occipital sulcus (LOS) and several bilateral sites along the intraparietal sulcus. These areas are largely consistent with those involved in the processing of 3D shape from motion and stereo. The experiments also demonstrate, however, that the analysis of 3D shape from shading is primarily restricted to the caudal ITG areas. Additional results from psychophysical experiments reveal that this difference in neuronal substrate cannot be explained by a difference in strength between the 2 cues. These results underscore the importance of the posterior part of the lateral occipital complex for the extraction of visual 3D shape information from all depth cues, and they suggest strongly that the importance of shading is diminished relative to other cues for the analysis of 3D shape in parietal regions.

Over the past few years, Generative Adversarial Networks (GANs) have garnered increased interest among researchers in Computer Vision, with applications including, but not limited to, image generation, translation, imputation, and super-resolution. Nevertheless, no GAN-based method has been proposed in the literature that can successfully represent, generate or translate 3D facial shapes (meshes). This can be primarily attributed to two facts, namely that (a) publicly available 3D face databases are scarce as well as limited in terms of sample size and variability (e.g., few subjects, little diversity in race and gender), and (b) mesh convolutions for deep networks present several challenges that are not entirely tackled in the literature, leading to operator approximations and model instability, often failing to preserve high-frequency components of the distribution. As a result, linear methods such as Principal Component Analysis (PCA) have been mainly utilized towards 3D shape analysis, despite being unable to capture non-linearities and high frequency details of the 3D face—such as eyelid and lip variations. In this work, we present 3DFaceGAN, the first GAN tailored towards modeling the distribution of 3D facial surfaces, while retaining the high frequency details of 3D face shapes. We conduct an extensive series of both qualitative and quantitative experiments, where the merits of 3DFaceGAN are clearly demonstrated against other, state-of-the-art methods in tasks such as 3D shape representation, generation, and translation.

Statistical Shape Analysis, that includes classic Geometric Morphometrics (GM), is often based on landmarks and frequently used to describe shape and shapes changes in biological studies. In this context changes in shape are typically analyzed separately from changes in size measured, in most cases, with centroid size (CS) [1]. Changes in shape are projected in a common linear space: the tangent space to the consensus and decomposed in affine and non affine components. The non affine component can be in turn de-composed in a series of local deformations (partial warps). This approach relies on the assumption that shapes are limitedly scattered in the shape space. In these conditions the difference between centroid size and m-volume is barely appreciable. In medical image, and in general in soft tissues, bodies can undergo very large deformations, involving also large changes in size strictly coupled with the change in shape from mechanical point of view. The cardiac example, analyzed in the present paper, shows changes in volume that can reach the 60% when comparing systole and diastole, coupled with severe longitudinal, radial and torsional strains. Because ventricle’s volume, together with its pressure, is one of the most important descriptors of the pumping heart function, it is natural considering the size change as volumes change. In fact, when dealing with such large size differences, CS and volume behave differently. In the last years the emerging disciplines of Diffeomorphometry and the related Functional Anatomy apply to replace classic GM through the use of diffeomorphisms, more suitable to describe soft tissues changes [2, 3]. On the other hand, these descriptions are very sophisticated but ignore some synthetic properties of the decomposition of deformation in some significant aspects. The goal of the present work is to show that standard GM tools (landmarks, Thin Plate Spline, and related decomposition of the deformation) can be generalized to better describe the very large deformations of biological tissues, without losing a synthetic description. In particular, the classical decomposition of the tangent space to the shape space in affine and non affine components [4] is enriched to include also the change in size, in such a way to give a complete description of the tangent space to the size-and-shape space. The proposed generalization is formulated by means of a new Riemannian metric describing the change in size as change in volume rather than change in CS. This lead to a redefinition of some aspect of the Kendalls size-and-shape space without abandoning Kendalls original formulation. This new formulation is discussed by analyzing 3D heart ventricular shapes coming from 3D Speckle Tracking Echocardiography. We evaluate the performances of different methods in recognizing Control (healthy) subjects from patients affected by Hypertrophic Cardiomyopathy.

Shape analysis finds many important applications in shape understanding, matching and retrieval. Among the various shape analysis methods, spectral shape analysis aims to study the spectrum of the Laplace–Beltrami operator of some well‐designed shape‐dependent equations and obtain a spectral shape descriptor that can in turn be used for shape analysis purposes. The success of such approaches depends greatly on the discriminating power of a shape descriptor. On the other hand, the medial axis of a shape is widely known for its complete shape representation. It is sensitive to small perturbation of the boundary of a shape which often poses difficulty in its effective use for shape analysis. In this paper, we propose a new spectral shape descriptor, called the medial axis spectrum for 2D shapes, which directly applies spectral analysis to the medial axes of the shapes. We extend the Laplace–Beltrami operator onto the medial axis, and take the solution to an extended Laplacian eigenvalue problem defined on the axis as the medial axis spectrum. The medial axis spectrum is robust in the presence of shape boundary noise, and is invariant under rigid transformations, uniform scaling and isometry of the medial axis. We demonstrate these benefits of such a medial axis spectrum representation through extensive experiments. The medial axis spectrum is further used for 2D shape retrieval, and its superiority over previous work is shown by comparison.

Abstract3D shapes provide substantially more information than 2D images. However, the acquisition of 3D shapes is sometimes very difficult or even impossible in comparison with acquiring 2D images, making it necessary to derive the 3D shape from 2D images. Although this is, in general, a mathematically ill-posed problem, it might be solved by constraining the problem formulation using prior information. Here, we present a new approach based on Kendall’s shape space to reconstruct 3D shapes from single monocular 2D images. The work is motivated by an application to study the feeding behavior of the basking shark, an endangered species whose massive size and mobility render 3D shape data nearly impossible to obtain, hampering understanding of their feeding behaviors and ecology. 2D images of these animals in feeding position, however, are readily available. We compare our approach with state-of-the-art shape-based approaches, both on human stick models and on shark head skeletons. Using a small set of training shapes, we show that the Kendall shape space approach is substantially more robust than previous methods and results in plausible shapes. This is essential for the motivating application in which specimens are rare and therefore only few training shapes are available.Keywords3D shape estimation2D-to-3DShape space approachKendall’s shape spaceSparse data

More and more analyses of biological shapes are using the techniques of geometric morphometrics based on configurations of landmarks in two or three dimensions. A fundamental concept at the core of these analyses is Kendall’s shape space and local approximations to it by shape tangent spaces. Kendall’s shape space is complex because it is a curved surface and, for configurations with more than three landmarks, multidimensional. This paper uses the shape space for triangles, which is the surface of a sphere, to explore and visualize some properties of shape spaces and the respective tangent spaces. Considerations about the dimensionality of shape spaces are an important step in understanding them, and can offer a coordinate system that can translate between positions in the shape space and the corresponding landmark configurations and vice versa. By simulation studies “walking” along that are great circles around the shape space, each of them corresponding to the repeated application of a particular shape change, it is possible to grasp intuitively why shape spaces are curved and closed surfaces. From these considerations and the available information on shape spaces for configurations with more than three landmarks, the conclusion emerges that the approach using a tangent space approximation in general is valid for biological datasets. The quality of approximation depends on the scale of variation in the data, but existing analyses suggest this should be satisfactory to excellent in most empirical datasets.

Example-based statistical framework for parametric modeling of human body shapes

ABSTRACTPrevious studies demonstrate the effectiveness of the Euclidean distance shape descriptor for characterizing individual tree crowns derived from Light Detection and Ranging (LiDAR) data. This paper evaluates the effectiveness of the longitude–latitude shape descriptor for tree crown shape characterization in comparison with the Euclidean distance descriptor using LiDAR data for 43 individual tree crowns. The three-dimensional (3D) shape signatures obtained from longitude–latitude and Euclidean distance measures of the tree crowns are compared with 3D shape signatures of simulated ellipsoid and hemisphere tree crowns. Kolmogorov–Smirnov test (K–S test) and Mean Square Error (MSE) are used to measure the similarity of longitude–latitude and Euclidean distance signatures. The experimental results from all 43 examined trees suggest that the longitude–latitude shape descriptor can determine tree crown shapes with different leaf traits, while Euclidean distance is only able to characterize the shape of evergreen trees. Furthermore, the longitude–latitude shape descriptor, unlike the Euclidean distance shape descriptor, is not sensitive to the number of LiDAR points from the tree crown.

We present O-CNN , an Octree-based Convolutional Neural Network (CNN) for 3D shape analysis. Built upon the octree representation of 3D shapes, our method takes the average normal vectors of a 3D model sampled in the finest leaf octants as input and performs 3D CNN operations on the octants occupied by the 3D shape surface. We design a novel octree data structure to efficiently store the octant information and CNN features into the graphics memory and execute the entire O-CNN training and evaluation on the GPU. O-CNN supports various CNN structures and works for 3D shapes in different representations. By restraining the computations on the octants occupied by 3D surfaces, the memory and computational costs of the O-CNN grow quadratically as the depth of the octree increases, which makes the 3D CNN feasible for high-resolution 3D models. We compare the performance of the O-CNN with other existing 3D CNN solutions and demonstrate the efficiency and efficacy of O-CNN in three shape analysis tasks, including object classification, shape retrieval, and shape segmentation.