On the Representation of Shapes Using Implicit Functions

We propose a markerless end-to-end training framework for parametric 3D human shape models. The training of statistical 3D human shape models with minimal supervision is an important problem in computer vision. Contrary to prior work, the whole training process (i) uses a differentiable shape model surface and (ii) is trained end-to-end by jointly optimizing all parameters of a single, self-contained objective that can be solved with slightly modified off-the-shelf non-linear least squares solvers. The training process only requires a compact model definition and an off-the-shelf 2D RGB pose estimator. No pre-trained shape models are required. For training (iii) a medium-sized dataset of approximately 1000 low-resolution human body scans is sufficient to achieve competitive performance on the challenging FAUST surface correspondence benchmark. The training and evaluation code will be made available for research purposes to facilitate end-to-end shape model training on novel datasets with minimal setup cost.

This paper proposed a method based on transformation optics, implicit function, and level set methods to solve the challenge of multi-physics simulation of a microwave heating cavity with two different motion modes. A 3D computation model with a rotating turntable, a lifting support rod, and a sample is proposed as a detailed demonstration. Based on the theory of transformation optics, the rotating turntable is surrounded by two circles with a time-varying, inhomogeneous and anisotropy medium, and the electric field in the moving region is rotated by controlling the two mediums. The implicit function and level set methods compute the lifting motion by setting the properties of the lifting region as a function of space and time. The correctness of the proposed method is verified by comparing the proposed method’s results with the discrete position’s results, and then its accuracy is further verified by experiment. Subsequently, compared with the implicit function and level set methods only, the proposed method is more accurate. Finally, the effects of lifting motion, rotating motion and lifting motion (i.e., spiral motion) on microwave heating uniformity and heating efficiency were analyzed, respectively.

We present a novel framework for learning a joint shape and appearance model from a large set of un-labelled training examples in arbitrary positions and orientations. The shape and intensity spaces are unified by implicitly representing shapes as “images” in the space of distance transforms. A stochastic chord-based matching algorithm is developed to align photo-realistic training examples under a common reference frame. Then dense local deformation fields, represented using the cubic B-spline based Free Form Deformations (FFD), are recovered to register the training examples in both shape and intensity spaces. Principal Component Analysis (PCA) is applied on the FFD control lattices to capture the variations in shape as well as on registered object interior textures. We show examples where we have built coupled shape and appearance prior models for the left ventricle and whole heart in short-axis cardiac tagged MR images, and used them to delineate the heart chambers in noisy, cluttered images. We also show quantitative validation on the automatic segmentation results by comparing to expert solutions.

The use of appearance and shape priors in image segmentation is known to improve accuracy; however, existing techniques have several drawbacks. For instance, most active shape and appearance models require landmark points and assume unimodal shape and appearance distributions, and the level set representation does not support construction of local priors. In this paper, we present novel appearance and shape models for image segmentation based on a differentiable implicit parametric shape representation called a disjunctive normal shape model (DNSM). The DNSM is formed by the disjunction of polytopes, which themselves are formed by the conjunctions of half-spaces. The DNSM's parametric nature allows the use of powerful local prior statistics, and its implicit nature removes the need to use landmarks and easily handles topological changes. In a Bayesian inference framework, we model arbitrary shape and appearance distributions using nonparametric density estimations, at any local scale. The proposed local shape prior results in accurate segmentation even when very few training shapes are available, because the method generates a rich set of shape variations by locally combining training samples. We demonstrate the performance of the framework by applying it to both 2-D and 3-D data sets with emphasis on biomedical image segmentation applications.

We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video. For the large differences between pareidolia face reenactment and traditional human face reenactment, two main challenges are introduced, i.e., shape variance and texture variance. In this work, we propose a novel Parametric Unsupervised Reenactment Algorithm to tackle these two challenges. Specifically, we propose to decompose the reenactment into three catenate processes: shape modeling, motion transfer and texture synthesis. With the decomposition, we introduce three crucial components, i.e., Parametric Shape Modeling, Expansionary Motion Transfer and Unsupervised Texture Synthesizer, to overcome the problems brought by the remarkably variances on pareidolia faces. Extensive experiments show the superior performance of our method both qualitatively and quantitatively. Code, model and data are available on our project page <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .

In the field of medical image analysis, the three-dimensional (3-D) shape representation and modeling of anatomic structures using only a few parameters is an important issue, and can be applied to computer assisted diagnosis, surgical simulations, visualization, and many other medical applications. In this paper, we show that the 3D anatomical structure such as the liver can be represented by a few coefficients of spherical harmonic functions (SPHARM). We also propose to use SPHARM based shape representation for statistical shape modeling. Since the dimension of SPHARM based shape representation vector is much lower than the conventional shape representation using coordinates of surface points, our proposed method can be used for small number of training samples and enhance the computation cost.

It is popular to use moving elements to improve heating uniformity during microwave heating under some situations. However, it is still a challenge to simulate moving elements during microwave heating due to the complexity of mathematical modeling. In this paper, a novel continuous algorithm based on implicit function and level set methods is developed to compute this heating process. By introducing the implicit function and level set methods, the moving region's properties are set as a function of space coordinate and time, which means that the movement of the elements is represented by the variation of the parameters. In addition, with the coordinate transformation, the coupling process of electromagnetic field and heat transfer in microwave heating can be performed. A 2-D applicator model and a 3-D applicator model with a rotating turntable are used to illustrate the proposed method in detail. Experiments are also conducted to testify the validity of the proposed method. The results obtained by the proposed method are in accordance with those obtained by experiments. Moreover, compared with conventional methods, the proposed method is more accurate and efficient.

The human body comes in many sizes and shapes. For design purposes, it is useful to be able to quickly simulate a virtual mannequin of a customer. A statistical shape model can be used for this purpose, because it describes the main variations of body shape inside the model’s population. From this model, the specific features of each person in the population are known. Therefore, a mapping between the shape model parameters and specific features can be calculated, which allows adjusting the body shape, in an intuitive way. In this work, we have investigated how accurate a body shape can be predicted based on a set of features and which features are most suitable for this purpose. Height, weight, and hip circumference appeared to be the most suitable features to accurately predict the body shape.

This paper presents a new deformable modeling strategy aimed at integrating shape and appearance in a unified space. If we think traditional deformable models as active contours or evolving curve fronts, the new deformable shape and appearance models we propose are deforming disks or volumes. Each model has not only boundary shape but also interior appearance. The model shape is implicitly embedded in a higher dimensional space of distance transforms, thus represented by a distance map image. In this way, both shape and appearance of the model are defined in the pixel space. A common deformation scheme, the Free Form Deformations (FFD), parameterizes warping deformations of the volumetric space in which the model is embedded in, hence deforming both model boundary and interior simultaneously.

One of the major goals of computer vision is the development of flexible and efficient methods for shape representation. This is true, especially for non-rigid 3D shapes where a great variety of shapes are produced as a result of deformations of a non-rigid object. Modeling these non-rigid shapes is a very challenging problem. Being able to analyze the properties of such shapes and describe their behavior is the key issue in research. Also, considering photometric features can play an important role in many shape analysis applications, such as shape matching and correspondence because it contains rich information about the visual appearance of real objects. This new information (contained in photometric features) and its important applications add another, new dimension to the problem's difficulty. Two main approaches have been adopted in the literature for shape modeling for the matching and retrieval problem, local and global approaches. Local matching is performed between sparse points or regions of the shape, while the global shape approaches similarity is measured among entire models. These methods have an underlying assumption that shapes are rigidly transformed. And Most descriptors proposed so far are confined to shape, that is, they analyze only geometric and/or topological properties of 3D models. A shape descriptor or model should be isometry invariant, scale invariant, be able to capture the fine details of the shape, computationally efficient, and have many other good properties. A shape descriptor or model is needed. This shape descriptor should be: able to deal with the non-rigid shape deformation, able to handle the scale variation problem with less sensitivity to noise, able to match shapes related to the same class even if these shapes have missing parts, and able to encode both the photometric, and geometric information in one descriptor. This dissertation will address the problem of 3D non-rigid shape representation and textured 3D non-rigid shapes based on local features. Two approaches will be proposed for non-rigid shape matching and retrieval based on Heat Kernel (HK), and Scale-Invariant Heat Kernel (SI-HK) and one approach for modeling textured 3D non-rigid shapes based on scale-invariant Weighted Heat Kernel Signature (WHKS). For the first approach, the Laplace-Beltrami eigenfunctions is used to detect a small number of critical points on the shape surface. Then a shape descriptor is formed based on the heat kernels at the detected critical points for different scales. Sparse representation is used to reduce the dimensionality of the calculated

In this study, a novel probabilistic, geometric and dynamic shape‐based level sets method is proposed. The shape prior is coupled with the intensity information to enhance the segmentation results. The two‐dimensional principal component analysis method is applied on the training shapes to represent the shape variation with enough number of shape projections in the training step. The shape model is constructed using the implicit representation of the projected shapes. A new energy functional is proposed (i) to embed the shape model into the image domain and (ii) to estimate the shape coefficients. The proposed method is validated on synthetic and clinical images with various challenges such as the noise, occlusion and missing information. The authors compare their method with some of related works. Experiments show that the proposed segmentation method is more accurate and robust than other alternatives under different challenges. ∗ Note: Colour figures are available in the online version of this paper.

Statistical shape modeling (SSM) is an essential tool for analyzing variations in anatomical morphology. In a typical SSM pipeline, 3D anatomical images, gone through segmentation and rigid registration, are represented using lower-dimensional shape features, on which statistical analysis can be performed. Various methods for constructing compact shape representations have been proposed, but they involve laborious and costly steps. We propose Image2SSM, a novel deep-learning-based approach for SSM that leverages image-segmentation pairs to learn a radial-basis-function (RBF)-based representation of shapes directly from images. This RBF-based shape representation offers a rich self-supervised signal for the network to estimate a continuous, yet compact representation of the underlying surface that can adapt to complex geometries in a data-driven manner. Image2SSM can characterize populations of biological structures of interest by constructing statistical landmark-based shape models of ensembles of anatomical shapes while requiring minimal parameter tuning and no user assistance. Once trained, Image2SSM can be used to infer low-dimensional shape representations from new unsegmented images, paving the way toward scalable approaches for SSM, especially when dealing with large cohorts. Experiments on synthetic and real datasets show the efficacy of the proposed method compared to the state-of-art correspondence-based method for SSM.

Chapter 2 - Shape Representation

Percutaneous screw fixation in pelvic trauma surgery is a challenging procedure that often requires long fluoroscopic exposure times and trial-and-error insertion attempts along narrow bone corridors of the pelvis. We report a method to automatically plan surgical trajectories using preoperative CT and assist device placement by augmenting the fluoroscopic scene with planned trajectories.A pelvic shape atlas was formed from 40 CT images and used to construct a statistical shape model (SSM). Each member of the atlas included expert definition of volumetric regions representing safe trajectory within bone corridors for fixating 10 common fracture patterns. Patient-specific planning is obtained by mapping the SSM to the (un-segmented) patient CT via active shape model (ASM) registration and free-form deformation (FFD), and the resulting transformation is used to transfer the atlas trajectory volumes to the patient CT. Fluoroscopic images acquired during K-wire placement are in turn augmented with projection of the planned trajectories via 3D–2D registration. Registration performance was evaluated via leave-one-out cross-validation over the 40-member atlas, computing the root mean square error (RMSE) in pelvic surface alignment (volumetric registration error), the positive predicted value (PPV) of volumetric trajectories within bone corridors (safety of the automatically planned trajectories), and the distance between trajectories within the planned volume and the bone cortex (absence of breach). A cadaver study was conducted in which K-wires were placed under fluoroscopic guidance to validate 3D–2D registration accuracy and evaluate the potential utility of augmented fluoroscopy with planned trajectories.The leave-one-out cross-validation achieved surface RMSE of 2.2 ± 0.3 mm after ASM registration and 1.8 ± 0.2 mm after FFD refinement. Automatically determined surgical plans conformed within bone corridors with PPV > 90% and centerline trajectory within 3–5 mm of the bone cortex. 3D–2D registration in the cadaver study achieved 0.3 ± 0.8 mm accuracy (in-plane translation) and <4° accuracy (in-plane rotation). Fluoroscopic images augmented with planning data exhibited >90% conformance of volumetric planning data overlay within bone, and all centerline trajectories were within safe corridors.The approach yields a method for both automatic planning of pelvic fracture fixation and augmentation of fluoroscopy for improved surgical precision and safety. The method does not require segmentation of the patient CT, operates without additional hardware (e.g. tracking systems), and is consistent with common workflow in fluoroscopically guided procedures. The approach has the potential to reduce operating time and radiation dose by minimizing trial-and-error attempts in percutaneous screw placement.

Statistical shape modelling versus linear scaling: Effects on predictions of hip joint centre location and muscle moment arms in people with hip osteoarthritis