Real-Time Full-Body Pose Synthesis and Editing

Pose recognition in character animations is an important avenue of research in computer graphics. However, the current use of traditional artificial intelligence algorithms to recognize animation gestures faces hurdles such as low accuracy and speed. Therefore, to overcome the above problems, this paper proposes a real-time 3D pose recognition system, which includes both facial and body poses, based on deep convolutional neural networks and further designs a single-purpose 3D pose estimation system. First, we transformed the human pose extracted from the input image to an abstract pose data structure. Subsequently, we generated the required character animation at runtime based on the transformed dataset. This challenges the conventional concept of monocular 3D pose estimation, which is extremely difficult to achieve. It can also achieve real-time running speed at a resolution of 384 fps. The proposed method was used to identify multiple-character animation using multiple datasets (Microsoft COCO 2014, CMU Panoptic, Human3.6M, and JTA). The results indicated that the improved algorithm improved the recognition accuracy and performance by approximately 3.5% and 8–10 times, respectively, which is significantly superior to other classic algorithms. Furthermore, we tested the proposed system on multiple pose-recognition datasets. The 3D attitude estimation system speed can reach 24 fps with an error of 100 mm, which is considerably less than that of the 2D attitude estimation system with a speed of 60 fps. The pose recognition based on deep learning proposed in this study yielded surprisingly superior performance, proving that the use of deep-learning technology for image recognition has great potential.

Increasingly complex physics-based models enhance the realism of character animation in computer graphics, but they pose difficult motor control challenges. This is especially the case when controlling a biomechanically simulated virtual human with an anatomically realistic structure that is actuated in a natural manner by a multitude of contractile muscles. Graphics researchers have pursued machine learning approaches to neuromuscular control, but traditional neural network learning methods suffer limitations when applied to complex biomechanical models and their associated high-dimensional training datasets. We demonstrate that “deep learning” is a useful approach to training neuromuscular controllers for biomechanical character animation. In particular, we propose a deep neural network architecture that can effectively and efficiently control (online) a dynamic musculoskeletal model of the human neck-head-face complex after having learned (offline) a high-dimensional map relating head orientation changes to neck muscle activations. To our knowledge, this is the first application of deep learning to biomechanical human animation with a muscle-driven model.

Until relatively recently, researchers in computer graphics paid scant attention to the numerics of their computations. Computation was used as a simple tool to evaluate algorithms or transform data into some appropriate pictoral representation. Thus standard computer graphics texts have little to say about numerical methods, just as earlier numerical analysis textbooks had little to say about computer graphics. This is now changing, for the important reasons outlined in this paper.

The modeling of plants is an active area of research in computer graphics and the geometrical modeling of veins is essential for obtaining plant images with a realistic appearance. The venation patterns determine the anisotropy of plant tissues, and affect the shadowing and masking of light incident on these tissues, hence the images. The current trend in the area of plant image generation is to aim for realism using biologically-based and predictable algorithms. Such algorithms do not depend on ad hoc parameters that have to be tuned whenever a new image is being generated. A number of plant image generation algorithms are now available in the literature that have achieved an impressive level of sophistication in many aspects, generating realistic looking images. One notable exception is the geometrical modeling of veins. This remains as an open problem in computer graphics. In this paper we review the state of art of the realistic simulation of plant venation systems from a geometric modeling perspective and propose a specified set of requirements for evaluating possible solutions for this problem.

Performing accurate and precise collision detection method between objects in virtual environment application such as computer games and medical simulation is important in computer graphics research and development. Given pair of objects that near colliding, numerous mechanic has been developed by researchers in order to minimize computation time and increase accuracy of the detection. However, most of these techniques required a lot of computational cost, extra processing power and complex algebraic equations just to solve distance between near colliding objects. In this paper, we described an alternate technique, which is a theoretical framework of novel technique in order to find the optimum closest distance between two or more convex polyhedral in virtual environment application. Given pair of near colliding objects, we proposed an easy to implement mechanism using dynamic origin point by creating inner and middle Axis Aligned Bounding-Box just to find closest distance between objects. We believed that the technique is suitable to be used in any game engine tools for computer games and medical simulation.

Achieving an efficient realistic illumination is an important aim of research in computer graphics. In this paper a new parallel global illumination method for hybrid systems based on the hierarchical radiosity method is presented. Our solution allows the exploitation of systems that combine independent nodes with multiple cores per node. Thus, multiple nodes work in parallel in the computation of the global illumination for the same scene. Within each node, all the available computational cores are used through a shared-memory multithreading approach. The good results obtained in terms of speedup on several distributed-memory and shared-memory configurations show the versatility of our hybrid proposal.

Numerical simulations of vortex sheets have a long history. The major advantage of the vortex sheet model lies in the efficiency of its boundary integral formulation, which allows us to avoid expensive computations of volumetric equations in some important problems. Although some researchers in computer graphics have exploited this benefit, there seems to be ample room for further improvements to their methods in relation to new results achieved in the applied mathematics community. On the other hand, applied mathematicians also could gain insight into flexible treatment of moving surfaces, which are very common in the current CG community. In this paper, we give an overview of recent progress in numerical simulations of the motion of three-dimensional vortex sheets with surface tension and discuss open problems that interests researchers in both communities referring to important papers from graphics.

Research in human modeling and simulation has been one of the primary areas of research in computer graphics since the early 1970s. It involves creating human life on a computer, digital avatars that move, talk, and behave like humans. The complexity of simulating the human body and its behavior is directly proportional to the complexity of the human body itself, and is compounded by the vast number of movements it is capable of. Research in this area encompasses multi-disciplinary efforts which include: biomechanics; computer animation; posture and motion prediction; anatomical modeling and physiological simulation. In this chapter we present a structured view of over two decades of research on anatomical modeling and simulation of virtual humans. We pay special attention to the modeling of the skeletal structure and the muscles as well as the simulation of their interactions.

Spectral methods for mesh processing and analysis rely on the eigenvalues, eigenvectors, or eigenspace projections derived from appropriately defined mesh operators to carry out desired tasks. Early work in this area can be traced back to the seminal paper by Taubin in 1995, where spectral analysis of mesh geometry based on a combinatorial Laplacian aids our understanding of the low‐pass filtering approach to mesh smoothing. Over the past 15 years, the list of applications in the area of geometry processing which utilize the eigenstructures of a variety of mesh operators in different manners have been growing steadily. Many works presented so far draw parallels from developments in fields such as graph theory, computer vision, machine learning, graph drawing, numerical linear algebra, and high‐performance computing. This paper aims to provide a comprehensive survey on the spectral approach, focusing on its power and versatility in solving geometry processing problems and attempting to bridge the gap between relevant research in computer graphics and other fields. Necessary theoretical background is provided. Existing works covered are classified according to different criteria: the operators or eigenstructures employed, application domains, or the dimensionality of the spectral embeddings used. Despite much empirical success, there still remain many open questions pertaining to the spectral approach. These are discussed as we conclude the survey and provide our perspective on possible future research.

The Advanced Volume Visualization Display (AVVD) research program is a joint research program between the Fraunhofer Center for Research in Computer Graphics, Inc. and Innovative Research and Development Corp. It is dedicated the application of the human visual system to real-time visualization of high- resolution volumetric sensor data sets. The AVVD program has successfully demonstrated its application to undersea imaging using data from the Naval Undersea Warfare Center -- Division Newport's High Resolution Array (HRA), and from the Naval Surface Warfare Center -- Coastal System Stations's Toroidal Volume Search Sonar (TVSS).

Fast display of realistic dynamic cloud scene is a challenging task for researchers in computer graphics. This paper describes a novel method of simulating animations of cloud. We first model the motion of cloud based on the physical theory of cloud formation. Then, the physical model of cloud is solved in discrete sparse grids. Thus, we can get approximate motion of cloud. Next, we advect texture image using the calculated velocity field to add the local detail of dynamic cloud scene. Here the texture image we used is Perlin noise map. The color of arbitrary point in the simulation space is got by calculating the product of the density texture and the regenerated Perlin noise texture. Finally, by changing different noise maps, realistic dynamic cloud scenes in different light environment are generated at high rendering rates.

Separation of the diffuse and specular components of observed reflectance has been an active area of research in computer graphics and vision, with major applications in reflectance modeling and scene analysis. Traditionally, researchers have investigated diffusespecular separation under point or directional illumination conditions while employing polarization and, in the case of dielectric materials, color space analysis techniques. Recently, Ma et al. [2007] introduced a technique for estimating high quality diffuse and specular normals and albedo maps (see Fig. 1, (a) & (d)) of a specular object using polarized spherical gradient illumination. However, the employed polarization technique imposes view-point restriction, and results in insufficient light levels for performance capture with high speed acquisition. Hence, in this work, we look into an alternate diffuse-specular separation technique for spherical gradients based on a data-driven reflectance model. Traditional separation techniques based on color space analysis focus on removing specular reflections from the observation for scene analysis [Mallick et al. 2005]. In contrast, we focus on obtaining high quality estimates of both the diffuse and the specular reflectance components.

The efficient and practical representation and processing of geometrically or topologically complex shapes often demands a partitioning into simpler patches. Possibilities range from unstructured arrangements of arbitrarily shaped patches on the one end, to highly structured conforming networks of all‐quadrilateral patches on the other end of the spectrum. Due to its regularity, this latter extreme of conforming partitions with quadrilateral patches, called quad layouts, is most beneficial in many application scenarios, for instance enabling the use of tensor‐product representations based on splines or Bézier patches, grid‐based multi‐resolution techniques and discrete pixel‐based map representations. However, this type of partition is also most complicated to create due to the strict inherent structural restrictions. Traditionally often performed manually in a tedious and demanding process, research in computer graphics and geometry processing has led to a number of computer‐assisted, semi‐automatic, as well as fully automatic approaches to address this problem more efficiently. This survey provides a detailed discussion of this range of methods, treats their strengths and weaknesses and outlines open problems in this field of research.

The graphics processors (GPUs) have recently emerged as a low-cost alternative for parallel programming. Since modern GPUs have great computational power as well as high memory bandwidth, running ray tracing on them has been an active field of research in computer graphics in recent years. Furthermore, the introduction of CUDA, a novel GPGPU architecture, has removed several limitations that the traditional GPU-based ray tracing suffered. In this paper, an implementation of high per formance CUDA ray tracing is demonstrated. We focus on the perfor mance and show how our design choices in various optimization lead to an implementation that outperforms the previous works. For reasonably complex scenes with simple shading, our implementation achieves the performance of 30 to 43 million traced rays per second. Our implementation also includes the effects of recursive specular reflection and refraction, which were less discussed in previous GPU-based ray tracing works.KeywordsRay TracingProgrammable Graphics HardwareGPU ComputingCUDAMultithreaded Architectures

We describe a dataset of several thousand calibrated, geo-referenced, high dynamic range color images, acquired under uncontrolled, variable illumination in an outdoor region spanning hundreds of meters. All image, feature, calibration, and geo-referencing data are available at http://city.lcs.mit.edu/data. Calibrated imagery is of fundamental interest in a wide variety of applications. We have made this data available in the belief that researchers in computer graphics, computer vision, photogrammetry and digital cartography will find it useful in several ways: as a test set for their own algorithms; as a calibrated image set for applications such as image-based rendering, metric 3D reconstruction, and appearance recovery; and as controlled imagery for integration into existing GIS systems and applications. The Web-based interface to the data provides interactive viewing of high-dynamic-range images and mosaics; extracted edge and point features; intrinsic and extrinsic calibration, along with maps of the ground context in which the images were acquired; the spatial adjacency relationships among images; the epipolar geometry relating adjacent images; compass and absolute scale overlays; and quantitative consistency measures for the calibration data.