Physically-based modeling: past, present, and future

Physically-based modeling: past, present, and future

Deformable models are a useful tool in computer vision and computer graphics. A deformable model is a curve (in two dimensions) or a surface (in three dimensions), whose shape, position, and orientation are controlled through a set of parameters. Deformable models can represent manufactured objects, human faces and skeletons, and even bodies of fluid. In computer graphics, we use deformable models for animations and simulations, whereas in computer vision applications, such as tracking and fitting, deformable models help to restrict the family of possible solutions. We introduce the use of a directed acyclic graph (DAG) to describe the position and Jacobian of each point on the surface of deformable models. This data structure, combined with a topological description of the points, is simple, powerful, and extremely useful for both computer vision and computer graphics applications. We show a computer vision application, 3D deformable face tracking, and a computer graphics application, cyberglove data visualization and calibration.

Facial modeling and animation are important research topics in computer graphics. During the last 20 years, a lot of research has been done in these areas, but it still remains a challenging task. The impact of previous and ongoing research has been felt in many applications, like games, Web-based 3D animations, 3D animation movies, etc. Two directions are investigated: precalculating animation with very realistic results for animated films and real-time animation for interactive applications. Correspondingly, the animation techniques vary from key-frame animations, where animators set each frame, to algorithmic parameterized mesh deformation. Many of the proposed deformation models use a parameterization scheme, which helps control the animation. Computer graphics have evolved to a relatively mature state. In parallel to the evolution of 3D graphics technologies, user and application requirements have also dramatically increased from simple virtual worlds to highly complex, interactive, and detailed virtual environments. Additionally, the targeted display platforms have widely broadened from dedicated graphics workstations or clusters of machines to standard desktop PCs, laptops, and mobile devices such as personal digital assistants (PDAs) or even mobile phones. Facial animation can be one illustration of such closely related evolutions of graphics techniques and corresponding applications and user’s requirements. Actually, despite much research and work on modeling, animation, and rendering techniques, it is still an important challenge to animate a highly realistic face with simulated hair and cloth, to display hundred of thousands of real-time animated humans on a standard computer, and it is still not possible to render animated characters on most mobile devices. The focus of this chapter is to present dynamically adaptive real-time facial animation techniques. We discuss methods to automatically and dynamically control the processing and memory loads together with the visual realism of rendered motions for real-time facial animation. Such approaches would theoretically allow us to free additional resources for hair or cloth animation; for instance, it should also achieve real-time performance for facial animation on multiplatform and on lightweight devices, as well as enable improvements to virtual environments with the addition of more and more facially animated humans in a single scene.

This paper presents a novel software framework for quantitative validation of physically-based deformable models in computer graphics. In the majority of previous studies, validation is qualitative (through visual plausibility), which is necessarily user subjective. The proposed framework facilitates construction of a single scalar output that quantifies the agreement between the complete time histories of test and reference models. Different models and comparison metrics can be easily included within the general framework. The framework is shown to yield a high accuracy score for a simplified model that can be analytically derived from the reference model, indicating that the framework is reliable. A lower score results when evaluating a more approximate, yet still visually plausible, model, demonstrating the objective sensitivity of the framework. The software framework can thus provide an objective measure of accuracy and a standardised way to quantitatively compare the accuracy of one method against another, whilst also supplying a quantitative rationale for trading accuracy and performance.

<title>ABSTRACT</title> <p>To realize the full potential of simulation-based evaluation and validation of autonomous ground vehicle systems, the next generation of modeling and simulation (M&S) solutions must provide real-time closed-loop environments that feature the latest physics-based modeling approaches and simulation solvers. Real-time capabilities enable seamless integration of human-in/on-the-loop training and hardware-in-the-loop evaluation and validation studies. Using an open modular architecture to close the loop between the physics-based solvers and autonomy stack components allows for full simulation of unmanned ground vehicles (UGVs) for comprehensive development, training, and testing of artificial intelligence vehicle-based agents and their human team members.</p> <p>This paper presents an introduction to a Proof of Concept for such a UGV M&S solution for severe terrain environments with a discussion of simulation results and future research directions. This conceptual approach features: 1) richly detailed severe terrain environments, 2) vehicle systems with multi-body dynamics, 3) Terramechanics-based tire-terrain interactions, 4) physics-based exteroceptive sensor models, 5) modular ROS autonomy components, 6) vehicle energy management and electric motor models, and 7) a user configurable dashboard for co-simulation coordination and model parameterization for automated M&S testing.</p> <p><bold>Citation:</bold> S.Misko, A. Free, S. Sivashankar, T. Kluge, V. Vantsevich, et al. “Real-time, Closed-Loop and Physics-based Modeling and Simulation System for Unmanned Ground Vehicles in Unstructured Terrain Environments,” In <italic>Proceedings of the Ground Vehicle Systems Engineering and Technology Symposium</italic> (GVSETS), NDIA, Novi, MI, Aug. 16-18, 2022.</p>

This paper presents a video object motion segmentation method for object tracking in visual surveillance. In the first step, the frames are first decomposed into small facets (regions), using colour information. Then, based on the detected motion, the motion segmentation is performed at facet level. A Bayesian approach is applied in clustering facets into moving objects and tracking moving video objects. Experiments have verified that the proposed method can efficiently tackle the complexity of video motion tracking.

Building deformation models using the motions captured from real video sequences is becoming a popular method in facial animation. In this paper, we propose an explanation-based facial motion tracking algorithm based on a piecewise Bezier volume deformation model (PBVD). The PBVD is a suitable model both for the synthesis and the analysis of facial images. It is linear and independent of the facial mesh structure. With this model, basic facial movements, or action units, are interactively defined. By changing the magnitudes of these action units, animated facial images are generated. The magnitudes of these action units can also be computed from real video sequences using a model-based tracking algorithm. However, in order to customize the articulation model for a particular face, the predefined PBVD action units need to be adaptively modified. In this paper, we first briefly introduce the PBVD model and its application in facial animation. Then a multi-resolution PBVD-based motion tracking algorithm is presented. Finally, we describe an explanation-based tracking algorithm that takes the predefined action units as the initial articulation model and adaptively improves them during the tracking process to obtain a more realistic articulation model. Experimental results on PBVD-based animation, model-based tracking, and explanation-based tracking are shown in this paper.

We present a deformable model to reconstruct a surface from a point cloud. The model is based on an explicit mesh representation composed of multiple competing evolving fronts. These fronts adapt to the local feature size of the target shape in a coarse–to–fine manner. Hence, they approach towards the finer (local) features of the target shape only after the reconstruction of the coarse (global) features has been completed. This conservative approach leads to a better control and interpretation of the reconstructed topology. The use of an explicit representation for the deformable model guarantees water‐tightness and simple tracking of topological events. Furthermore, the coarse–to–fine nature of reconstruction enables adaptive handling of non‐homogenous sample density, including robustness to missing data in defected areas.Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Digitizing and scanning. Keywords: surface reconstruction, deformable models

Deformable models are a powerful tool in both computer graphics and computer vision. The description and implementation of the deformations have to be simultaneously flexible and powerful, otherwise the technique may not satisfy the requirements of all the distinct applications. In this paper we introduce a new method for deformable model specification: deformable fields. Deformable fields are conceptually simple, lead to an easy implementation, and are suitable for adaptive models. We apply our new technique to describe an adaptive deformable face, and compare three different adaptation strategies. Additionally, we show how our technique is suitable to describe different individuals.

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

We present a deformable graphical model for interactive modeling of three-dimensional objects representing organs and tissues in surgery simulations. The proposed model is a physically-based model, generating real-time deformations using a high resolution triangular surface mesh with minimal pre-processing of mesh data. The surface mesh of an object is modeled by a spring network according to the connectivity of mesh nodes. Object deformations are simulated by computing nodal deformations based on a force equation at each mesh node. A deformed node index table (DNIT) is proposed to model the deformation propagation driven by the displacement at a surface contact point (SCP). Surface nodes that undergo a deformation are added to the current DNIT according to the triangular mesh topology. A PHANToM device is integrated in the simulator to provide haptic feedback. Coupled with the real-time visual feedback generated by the proposed model, the simulator allows users to interactively manipulate virtual organs utilizing the PHANToM device. Results of a virtual kidney biopsy simulation are presented to illustrate the deformations.

This paper describes how the concept of imposing geometric constraints by minimizing cost functions may be used and extended to accomplish a variety of animated modelling tasks for computer graphics. In this approach a complex 3-D geometric problem is mapped into a scalar minimization formulation. The mapping provides a straightforward method for converting abstract geometric concepts into a construct that is easily computed. The minimization approach is demonstrated in three application areas: computer animation, visualization, and physically-based modelling. In the computer animation application, cost minimization may be used to generate motion paths and joint parameters for animated actors. The approach may also be used to generate deformable models that extract closed 3-D geometric models from volume data for visualization. In the final application, the approach provides the fundamental structure to a physically-based model of woven cloth. © 1997 John Wiley & Sons, Ltd.

This paper proposes a unified facial motion tracking and expression recognition framework for monocular video. For retrieving facial motion, an online weight adaptive statistical appearance method is embedded into the particle filtering strategy by using a deformable facial mesh model served as an intermediate to bring input images into correspondence by means of registration and deformation. For recognizing facial expression, facial animation and facial expression are estimated sequentially for fast and efficient applications, in which facial expression is recognized by static anatomical facial expression knowledge. In addition, facial animation and facial expression are simultaneously estimated for robust and precise applications, in which facial expression is recognized by fusing static and dynamic facial expression knowledge. Experiments demonstrate the high tracking robustness and accuracy as well as the high facial expression recognition score of the proposed framework.

The Web provides a useful environment for simple surgical training simulations. A combination of VRML for 3D rendering, and Java code for the simulation engine, has been used for a range of simple neurosurgical demonstrators. However the elements in these simulators are rigid, to avoid the computational complexity of deformable modelling. In this paper we describe a variation of the ChainMail technique that allows us to provide real-time deformable modelling, even in a Web browser environment on a PC. Our new algorithm, SurfaceChainMail, has been used to develop a simulator for the cutting of two layers of tissue, and separating the layers by pulling them apart.

<p>The objective of this study is to integrate a continuum-based deformable tire and terrain interaction model into a general-use physics-based simulation environment capable of off-road vehicle mobility analysis and high-performance computing potential. Specifically, the physics-based deformable tire and terrain models which were recently proposed and validated by Yamashita, et al. will be implemented into the structure of the multi-physics simulation engine Chrono. In off-road vehicle mobility analysis, empirical and analytical models have been commonly used for vehicle-terrain interaction. While these models utilize experimental data or terramechanics theories to create quick predictive mobility models, they are unable to capture the highly nonlinear behavior of soft soil deformation, which can lead to inaccurate or unreliable results. In order to resolve these limitations, the use of physics-based numerical approaches have been proposed. These methods make use of finite element and discrete element simulations to describe the interaction between the vehicle and deformable terrain. Continuum-based finite element models transfer tire forces to the terrain and model the deformation with elasto-plastic constitutive models. Discrete element soil uses a large number of small rigid body particles to describe the microscale behavior of granular terrain, with the deformation of the soil represented by the motion and contact of the particles. While these physics-based models offer a more accurate vehicle-terrain interaction model, the solution procedure can become complex and computationally expensive since co-simulation techniques are often used.</p><p>To address these issues, the analysis of physics-based full vehicle dynamics simulations utilizing high-fidelity deformable tire and terrain models in a multi-physics engine with high-performance computing capability is desired. To this end, a continuum mechanics based shear deformable laminated composite shell element proposed by Yamashita, et al. was integrated into the flexible body dynamics simulation framework of Chrono. This element was based on the absolute nodal coordinate formulation and is defined by the global position coordinates and the transverse gradient coordinates of its four nodes. Element lockings are eliminated with the incorporation of the enhanced assumed strain (EAS) and assumed natural strain approaches (ANS). The element formulation includes an extension to model laminated composite materials. Additionally, a locking-free 9-node brick element was integrated into the Chrono framework that makes use of the curvature coordinates at the center of the element. This element is formulated with the Hencky strain measure such that multiplicative finite strain plasticity theory can be used to incorporate soil plasticity models, such as the capped Drucker-Prager failed criterion.</p><p>With the shear deformable laminated composite shell element and plastic soil brick element integrated into the Chrono multi-physics simulation engine, an off-road deformable tire and terrain interaction model was developed using the vehicle dynamics simulation module Chrono::Vehicle. An off-road deformable tire model was parameterized based on commercial tire properties and generated as an interchangeable tire model option in the full vehicle dynamics system. Benchmark verification tests were performed to ensure the accuracy of tire deformation and tire force characteristics. Further tests were performed to validate a deformable tire model with a deformable tread pattern constructed from shear deformable shell elements and co-rotational tetrahedral elements. The deformable soil model was also integrated as a terrain option in Chrono::Vehicle and numerical tests were carried out to demonstrate its interaction with rigid and deformable tire models. To make use of the computational performance enhancements available in Chrono, Open Multi-Processing (OpenMP) and Advanced Vector Extensions (AVX) were applied to the evaluation of the elastic force/Jacobian matrix and large matrix operations of flexible bodies, respectively, in order to reduce the computation time by nearly 60%.</p>