Physically-based modeling: past, present, and future

My name is Demetri Terzopoulos and my co-chair, John Platt, and I would like to welcome you to the panel on Physically-Based Modeling -- Past, Present and Future. I'll start by introducing the panelists; the affiliations you see listed on the screen are somewhat out of date.

Deformable models are a useful tool in computer vision and computer graphics. A deformable model is a curve (in two dimensions) or a surface (in three dimensions), whose shape, position, and orientation are controlled through a set of parameters. Deformable models can represent manufactured objects, human faces and skeletons, and even bodies of fluid. In computer graphics, we use deformable models for animations and simulations, whereas in computer vision applications, such as tracking and fitting, deformable models help to restrict the family of possible solutions. We introduce the use of a directed acyclic graph (DAG) to describe the position and Jacobian of each point on the surface of deformable models. This data structure, combined with a topological description of the points, is simple, powerful, and extremely useful for both computer vision and computer graphics applications. We show a computer vision application, 3D deformable face tracking, and a computer graphics application, cyberglove data visualization and calibration.

This paper presents a novel software framework for quantitative validation of physically-based deformable models in computer graphics. In the majority of previous studies, validation is qualitative (through visual plausibility), which is necessarily user subjective. The proposed framework facilitates construction of a single scalar output that quantifies the agreement between the complete time histories of test and reference models. Different models and comparison metrics can be easily included within the general framework. The framework is shown to yield a high accuracy score for a simplified model that can be analytically derived from the reference model, indicating that the framework is reliable. A lower score results when evaluating a more approximate, yet still visually plausible, model, demonstrating the objective sensitivity of the framework. The software framework can thus provide an objective measure of accuracy and a standardised way to quantitatively compare the accuracy of one method against another, whilst also supplying a quantitative rationale for trading accuracy and performance.

This paper presents a video object motion segmentation method for object tracking in visual surveillance. In the first step, the frames are first decomposed into small facets (regions), using colour information. Then, based on the detected motion, the motion segmentation is performed at facet level. A Bayesian approach is applied in clustering facets into moving objects and tracking moving video objects. Experiments have verified that the proposed method can efficiently tackle the complexity of video motion tracking.

We present a deformable model to reconstruct a surface from a point cloud. The model is based on an explicit mesh representation composed of multiple competing evolving fronts. These fronts adapt to the local feature size of the target shape in a coarse–to–fine manner. Hence, they approach towards the finer (local) features of the target shape only after the reconstruction of the coarse (global) features has been completed. This conservative approach leads to a better control and interpretation of the reconstructed topology. The use of an explicit representation for the deformable model guarantees water‐tightness and simple tracking of topological events. Furthermore, the coarse–to–fine nature of reconstruction enables adaptive handling of non‐homogenous sample density, including robustness to missing data in defected areas.Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Digitizing and scanning. Keywords: surface reconstruction, deformable models

Deformable models are a powerful tool in both computer graphics and computer vision. The description and implementation of the deformations have to be simultaneously flexible and powerful, otherwise the technique may not satisfy the requirements of all the distinct applications. In this paper we introduce a new method for deformable model specification: deformable fields. Deformable fields are conceptually simple, lead to an easy implementation, and are suitable for adaptive models. We apply our new technique to describe an adaptive deformable face, and compare three different adaptation strategies. Additionally, we show how our technique is suitable to describe different individuals.

Augmented reality applications have rapidly spread across online retail platforms and social media, allowing consumers to virtually try‐on a large variety of products, such as makeup, hair dying, or shoes. However, parametrizing a renderer to synthesize realistic images of a given product remains a challenging task that requires expert knowledge. While recent work has introduced neural rendering methods for virtual try‐on from example images, current approaches are based on large generative models that cannot be used in real‐time on mobile devices. This calls for a hybrid method that combines the advantages of computer graphics and neural rendering approaches. In this paper, we propose a novel framework based on deep learning to build a real‐time inverse graphics encoder that learns to map a single example image into the parameter space of a given augmented reality rendering engine. Our method leverages self‐supervised learning and does not require labeled training data, which makes it extendable to many virtual try‐on applications. Furthermore, most augmented reality renderers are not differentiable in practice due to algorithmic choices or implementation constraints to reach real‐time on portable devices. To relax the need for a graphics‐based differentiable renderer in inverse graphics problems, we introduce a trainable imitator module. Our imitator is a generative network that learns to accurately reproduce the behavior of a given non‐differentiable renderer. We propose a novel rendering sensitivity loss to train the imitator, which ensures that the network learns an accurate and continuous representation for each rendering parameter. Automatically learning a differentiable renderer, as proposed here, could be beneficial for various inverse graphics tasks. Our framework enables novel applications where consumers can virtually try‐on a novel unknown product from an inspirational reference image on social media. It can also be used by computer graphics artists to automatically create realistic rendering from a reference product image.

Women and computing

This paper describes how the concept of imposing geometric constraints by minimizing cost functions may be used and extended to accomplish a variety of animated modelling tasks for computer graphics. In this approach a complex 3-D geometric problem is mapped into a scalar minimization formulation. The mapping provides a straightforward method for converting abstract geometric concepts into a construct that is easily computed. The minimization approach is demonstrated in three application areas: computer animation, visualization, and physically-based modelling. In the computer animation application, cost minimization may be used to generate motion paths and joint parameters for animated actors. The approach may also be used to generate deformable models that extract closed 3-D geometric models from volume data for visualization. In the final application, the approach provides the fundamental structure to a physically-based model of woven cloth. © 1997 John Wiley & Sons, Ltd.

We present a deformable graphical model for interactive modeling of three-dimensional objects representing organs and tissues in surgery simulations. The proposed model is a physically-based model, generating real-time deformations using a high resolution triangular surface mesh with minimal pre-processing of mesh data. The surface mesh of an object is modeled by a spring network according to the connectivity of mesh nodes. Object deformations are simulated by computing nodal deformations based on a force equation at each mesh node. A deformed node index table (DNIT) is proposed to model the deformation propagation driven by the displacement at a surface contact point (SCP). Surface nodes that undergo a deformation are added to the current DNIT according to the triangular mesh topology. A PHANToM device is integrated in the simulator to provide haptic feedback. Coupled with the real-time visual feedback generated by the proposed model, the simulator allows users to interactively manipulate virtual organs utilizing the PHANToM device. Results of a virtual kidney biopsy simulation are presented to illustrate the deformations.

<p>The objective of this study is to integrate a continuum-based deformable tire and terrain interaction model into a general-use physics-based simulation environment capable of off-road vehicle mobility analysis and high-performance computing potential. Specifically, the physics-based deformable tire and terrain models which were recently proposed and validated by Yamashita, et al. will be implemented into the structure of the multi-physics simulation engine Chrono. In off-road vehicle mobility analysis, empirical and analytical models have been commonly used for vehicle-terrain interaction. While these models utilize experimental data or terramechanics theories to create quick predictive mobility models, they are unable to capture the highly nonlinear behavior of soft soil deformation, which can lead to inaccurate or unreliable results. In order to resolve these limitations, the use of physics-based numerical approaches have been proposed. These methods make use of finite element and discrete element simulations to describe the interaction between the vehicle and deformable terrain. Continuum-based finite element models transfer tire forces to the terrain and model the deformation with elasto-plastic constitutive models. Discrete element soil uses a large number of small rigid body particles to describe the microscale behavior of granular terrain, with the deformation of the soil represented by the motion and contact of the particles. While these physics-based models offer a more accurate vehicle-terrain interaction model, the solution procedure can become complex and computationally expensive since co-simulation techniques are often used.</p><p>To address these issues, the analysis of physics-based full vehicle dynamics simulations utilizing high-fidelity deformable tire and terrain models in a multi-physics engine with high-performance computing capability is desired. To this end, a continuum mechanics based shear deformable laminated composite shell element proposed by Yamashita, et al. was integrated into the flexible body dynamics simulation framework of Chrono. This element was based on the absolute nodal coordinate formulation and is defined by the global position coordinates and the transverse gradient coordinates of its four nodes. Element lockings are eliminated with the incorporation of the enhanced assumed strain (EAS) and assumed natural strain approaches (ANS). The element formulation includes an extension to model laminated composite materials. Additionally, a locking-free 9-node brick element was integrated into the Chrono framework that makes use of the curvature coordinates at the center of the element. This element is formulated with the Hencky strain measure such that multiplicative finite strain plasticity theory can be used to incorporate soil plasticity models, such as the capped Drucker-Prager failed criterion.</p><p>With the shear deformable laminated composite shell element and plastic soil brick element integrated into the Chrono multi-physics simulation engine, an off-road deformable tire and terrain interaction model was developed using the vehicle dynamics simulation module Chrono::Vehicle. An off-road deformable tire model was parameterized based on commercial tire properties and generated as an interchangeable tire model option in the full vehicle dynamics system. Benchmark verification tests were performed to ensure the accuracy of tire deformation and tire force characteristics. Further tests were performed to validate a deformable tire model with a deformable tread pattern constructed from shear deformable shell elements and co-rotational tetrahedral elements. The deformable soil model was also integrated as a terrain option in Chrono::Vehicle and numerical tests were carried out to demonstrate its interaction with rigid and deformable tire models. To make use of the computational performance enhancements available in Chrono, Open Multi-Processing (OpenMP) and Advanced Vector Extensions (AVX) were applied to the evaluation of the elastic force/Jacobian matrix and large matrix operations of flexible bodies, respectively, in order to reduce the computation time by nearly 60%.</p>

Abstract3D reconstruction of deformable (ornon‐rigid) scenes from a set of monocular 2D image observations is a long‐standing and actively researched area of computer vision and graphics. It is an ill‐posed inverse problem, since—without additional prior assumptions—it permits infinitely many solutions leading to accurate projection to the input 2D images. Non‐rigid reconstruction is a foundational building block for downstream applications like robotics, AR/VR, or visual content creation. The key advantage of using monocular cameras is their omnipresence and availability to the end users as well as their ease of use compared to more sophisticated camera set‐ups such as stereo or multi‐view systems. This survey focuses on state‐of‐the‐art methods for dense non‐rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views. It reviews the fundamentals of 3D reconstruction and deformation modeling from 2D image observations. We then start from general methods—that handle arbitrary scenes and make only a few prior assumptions—and proceed towards techniques making stronger assumptions about the observed objects and types of deformations (e.g. human faces, bodies, hands, and animals). A significant part of this STAR is also devoted to classification and a high‐level comparison of the methods, as well as an overview of the datasets for training and evaluation of the discussed techniques. We conclude by discussing open challenges in the field and the social aspects associated with the usage of the reviewed methods.

Homogenized constitutive and fatigue nucleation models from crystal plasticity FE simulations of Ti alloys, Part 2: Macroscopic probabilistic crack nucleation model

To account for both surface and volume effects and to provide cross-validation when using in situ nondestructive evaluation (NDE) methods, a hybrid optico-acoustic approach combining digital image correlation (DIC) and acoustic emission (AE) was developed, validated, and tested. The designed system includes hardware and software linkages between the two sensing modalities with feedback both from the mechanical testing device as well as from other complementary NDE methods including ultrasonic testing (UT) and infrared thermography (IRT). This hybrid monitoring approach targets the early identification of damage precursors which vary depending on the material tested. To corroborate the reliability of the hybrid setup, various tensile, fracture and fatigue testing procedures were implemented for different types of materials. Of interest herein was the evaluation of each individual NDE method’s monitoring capabilities of deformation and damage in conjunction with their spatial and temporal resolutions. In carefully designed metal alloy microstructures with well-characterized mechanical properties, AE validated DIC-observed strain band formations on the sample’s surface, which further enabled targeted microscopic analyses that proved the origin of the NDE findings. In the case of fracture, DIC and UT validated AE information regarding both crack initiation and growth in Mode I testing of metallic alloy samples. Finally, in fatigue loading of fiber reinforced composites, IR measurements confirmed DIC indications of early damage formation consistent with AE recordings. Post-processing of NDE datasets combined with actual loading information allowed the calculation of mechanical parameters (elastic–plastic transition stress, strain localizations, traction separation laws, energy dissipation, localized Poisson ratios) that have been used to construct physics-based computational models for multi-scale deformation and damage evolution, as well as remaining life-estimations.

This work focuses on the analysis and review of most prominent advanced rendering and animation techniques in the computer graphics used for 3D games, some of which we then juxtapose with our physically based softbody simulation framework we designed and implemented earlier. We then discuss such implementation results and the future directions. The techniques we touch upon are various algorithms for animation and physical based modeling as well as GPU programming with OpenGL. We break down our contribution into two main parts: the analysis and review of the applicability of the techniques to softbody rendering and animation with the summary of the results to date.