3DFaceGAN: Adversarial Nets for 3D Face Representation, Generation, and Translation

A new method for reconstructing the 3D shape of single cells in fruit

3D human shape and pose estimation from monocular images has been an active area of research in computer vision, having a substantial impact on the development of new applications, from activity recognition to creating virtual avatars. Existing deep learning methods for 3D human shape and pose estimation rely on relatively high-resolution input images; however, high-resolution visual content is not always available in several practical scenarios such as video surveillance and sports broadcasting. Low-resolution images in real scenarios can vary in a wide range of sizes, and a model trained in one resolution does not typically degrade gracefully across resolutions. Two common approaches to solve the problem of low-resolution input are applying super-resolution techniques to the input images which may result in visual artifacts, or simply training one model for each resolution, which is impractical in many realistic applications. To address the above issues, this paper proposes a novel algorithm called RSC-Net, which consists of a Resolution-aware network, a Self-supervision loss, and a Contrastive learning scheme. The proposed network is able to learn the 3D body shape and pose across different resolutions with a single model. The self-supervision loss encourages scale-consistency of the output, and the contrastive learning scheme enforces scale-consistency of the deep features. We show that both these new training losses provide robustness when learning 3D shape and pose in a weakly-supervised manner. Extensive experiments demonstrate that the RSC-Net can achieve consistently better results than the state-of-the-art methods for challenging low-resolution images.

This paper presents a novel method for face alignment under unknown head poses and nonrigid warp, within the framework of markov random field. The proposed method learns a 3D face shape model comprised of 31 facial features and a texture model for each facial feature from a 3D face database. The models are combined to serve as the unary, pairwise and high order constraints of the markov random field. The facial features are located by minimizing the potential function of markov random field, which is solved with dual decomposition. The main contribution of this paper is composed of three aspects. First, Random Project Tree is utilized to learn the manifold structure of the facial feature appearance under different view points. Second, a 3D face shape model is learned to capture the linear part of face shape distribution due to the change of identity and expression, which is served as the high order constraints. Third, Markov random field is introduced to model the nonlinear part of the face shape distribution, and also to deal with occlusion of facial features due to head pose variation or ornaments. Experiments was taken on the Texas 3D face database and face images downloaded from the Internet, results shows capability of adapting large head pose variations of the method.KeywordsFace ImageFacial FeatureMarkov Random FieldFace ShapeTexture ModelThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

We used functional magnetic resonance imaging to investigate the human cortical areas involved in processing 3-dimensional (3D) shape from texture (SfT) and shading. The stimuli included monocular images of randomly shaped 3D surfaces and a wide variety of 2-dimensional (2D) controls. The results of both passive and active experiments reveal that the extraction of 3D SfT involves the bilateral caudal inferior temporal gyrus (caudal ITG), lateral occipital sulcus (LOS) and several bilateral sites along the intraparietal sulcus. These areas are largely consistent with those involved in the processing of 3D shape from motion and stereo. The experiments also demonstrate, however, that the analysis of 3D shape from shading is primarily restricted to the caudal ITG areas. Additional results from psychophysical experiments reveal that this difference in neuronal substrate cannot be explained by a difference in strength between the 2 cues. These results underscore the importance of the posterior part of the lateral occipital complex for the extraction of visual 3D shape information from all depth cues, and they suggest strongly that the importance of shading is diminished relative to other cues for the analysis of 3D shape in parietal regions.

BackgroundAortic shape has been proposed as an important determinant of adverse haemodynamics following coarctation repair. However, previous studies have not demonstrated a consistent relationship between shape and vascular load. In this study, 3D aortic shape was evaluated using principal component analysis (PCA), allowing investigation of the relationship between 3D shape and haemodynamics.MethodsSixty subjects (38 male, 25.0 ± 7.8 years) with repaired coarctation were recruited. Central aortic haemodynamics including wave intensity analysis were measured noninvasively using a combination of blood pressure and phase contrast cardiovascular magnetic resonance (CMR). 3D curvature and radius data were derived from CMR angiograms. PCA was separately performed on 3D radius and curvature data to assess the role of arch geometry on haemodynamics. Clinical findings were corroborated using 1D vascular models.ResultsThere were no independent associations between 3D curvature and any hemodynamic parameters. However, the magnitude of the backwards compression wave was related to the 1st (r = − 0.36, p = 0.005), 3rd (r = 0.27, p = 0.036) and 4th (r = − 0.31, p = 0.017) principle components of radius. The 4th principle componentof radius also correlated with central aortic systolic pressure. These aortas had larger aortic roots, more transverse arch hypoplasia and narrower aortic isthmuses.ConclusionsThere are major modes of variation in 3D aortic shape after coarctation repair witha modest association between variation in aortic radius and pathological wave reflections, but not with 3D curvature. Taken together, these data suggest that shape is not the major determinant of vascular load following coarctation repair, and calibre is more important than curvature.

Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding

Computer vision is one of the hottest research fields in deep learning. The emergence of generative adversarial networks (GANs) provides a new method and model for computer vision. The idea of GANs using the game training method is superior to traditional machine learning algorithms in terms of feature learning and image generation. GANs are widely used not only in image generation and style transfer but also in the text, voice, video processing, and other fields. However, there are still some problems with GANs, such as model collapse and uncontrollable training. This paper deeply reviews the theoretical basis of GANs and surveys some recently developed GAN models, in comparison with traditional GAN models. The applications of GANs in computer vision include data enhancement, domain transfer, high-quality sample generation, and image restoration. The latest research progress of GANs in artificial intelligence (AI) based security attack and defense is introduced. The future development of GANs in computer vision is also discussed at the end of the paper with possible applications of AI in computer vision.

Recent researches in three-dimensional (3D) face recognition area have proved that, 3D face recognition methods, achieve better accuracy than its 2D counterpart. One of the main advantages of 3D face recognition methods is that it measures the geometry of rigid features on the human face, due to which it becomes an invariant to illumination, expressions and rotations of head (pose variation). But the main drawback of 3D face recognition methods is the acquisition of 3D face images which generally needs a range camera. It is where the research community is facing difficulty. Research in 3D face recognition also depends on available databases. In this paper, we present a qualitative review of all the available 3D face databases. A comparative table of all the databases is also given at the end with all the necessary parameters.

INTRODUCTION Traditionally, the perception of three-dimensional (3D) scenes and objects was assumed to be the result of the reconstruction of 3D surfaces from available depth cues, such as binocular disparity, motion parallax, texture, and shading (Marr, 1982). In this approach, the role of ‰gure-ground organization was kept to a minimum. Figure-ground organization refers to ‰nding objects in the image, diŠerentiating them from the background, and grouping contour elements into two-dimensional (2D) shapes representing the shapes of the 3D objects. According to Marr, solving ‰gureground organization was not necessary. e depth cues could provide the observer with local estimates of surface orientation at many points. ese local estimates were then integrated into the estimates of the visible parts of 3D shapes. Marr called this a 2.5D representation. If the objects in the scene are familiar, their entire 3D shapes can be identi‰ed (recognized) byCONTENTS Introduction 113 Figure-Ground Organization 114 Using Simplicity Principle to Recover 3D Shapes 118 Summary and Discussion 123matching the shapes in the observer's memory to the 2.5D sketch. If they are not familiar, the observer will have to view the object from more than one viewing direction and integrate the visible surfaces into the 3D shape representations.

Example‐based statistical face models using principle component analysis (PCA) have been widely deployed for three‐dimensional (3D) face reconstruction and face recognition. The two common factors that are generally concerned with such models are the size of the training dataset and the selection of different examples in the training set. The representational power (RP) of an example‐based model is its capability to depict a new 3D face for a given 2D face image. The RP of the model can be increased by correspondingly increasing the number of training samples. In this contribution, a novel approach is proposed to increase the RP of the 3D face reconstruction model by deforming a set of examples in the training dataset. A PCA‐based 3D face model is adapted for each new near frontal input face image to reconstruct the 3D face shape. Further an extended Tikhonov regularisation method has been employed to reconstruct 3D face shapes from a set of facial points. The results justify that the proposed adaptive PCA‐based model considerably improves the RP of the standard PCA‐based model and outperforms it with a 95% confidence level.

In this paper, we investigate a novel problem of using generative adversarial networks in the task of 3D shape generation according to semantic attributes. Recent works map 3D shapes into 2D parameter domain, which enables training Generative Adversarial Networks (GANs) for 3D shape generation task. We extend these architectures to the conditional setting, where we generate 3D shapes with respect to subjective attributes defined by the user. Given pairwise comparisons of 3D shapes, our model performs two tasks: it learns a generative model with a controlled latent space, and a ranking function for the 3D shapes based on their multi-chart representation in 2D. The capability of the model is demonstrated with experiments on HumanShape, Basel Face Model and reconstructed 3D CUB datasets. We also present various applications that benefit from our model, such as multi-attribute exploration, mesh editing, and mesh attribute transfer.

This paper describes a vector representation of 3D shapes acquired in the form of 2D range images produced using a range finder (NEC Danae-R). The vectorization was performed by first extracting feature points that correspond to each other as landmarks among different 3D face shapes, and then re-sampling each surface region of the face divided by the landmarks into other regions to obtain corresponding elements of the 3D shape vector. A linear description of the 3D shape spaces obtained by principal component analysis allows a smooth morphable modeling of a human face in terms of a smaller number of projection parameters. We scientifically solved the relations between shapes and impressions of faces and attempted face-image manipulations that transform impressions on facial attributes.

In this paper, we present a novel strategy to design disentangled 3D face shape representation. Specifically, a given 3D face shape is decomposed into identity part and expression part, which are both encoded and decoded in a nonlinear way. To solve this problem, we propose an attribute decomposition framework for 3D face mesh. To better represent face shapes which are usually nonlinear deformed between each other, the face shapes are represented by a vertex based deformation representation rather than Euclidean coordinates. The experimental results demonstrate that our method has better performance than existing methods on decomposing the identity and expression parts. Moreover, more natural expression transfer results can be achieved with our method than existing methods.

In perceiving 3D shape from ambiguous shading patterns, humans use the prior knowledge that the light is located above their head and slightly to the left. Although this observation has fascinated scientists and artists for a long time, the neural basis of this "light from above left" preference for the interpretation of 3D shape remains largely unexplored. Combining behavioral and functional MRI measurements coupled with multivoxel pattern analysis, we show that activations in early visual areas predict best the light source direction irrespective of the perceived shape, but activations in higher occipitotemporal and parietal areas predict better the perceived 3D shape irrespective of the light direction. These findings demonstrate that illumination is processed earlier than the representation of 3D shape in the visual system. In contrast to previous suggestions, we propose that prior knowledge about illumination is processed in a bottom-up manner and influences the interpretation of 3D structure at higher stages of processing.

3D face similarity is a critical issue in computer vision, computer graphics and face recognition and so on. Since Frechet distance is an effective metric for measuring curve similarity, a novel 3D face similarity measure method based on Frechet distances of geodesics is proposed in this paper. In our method, the surface similarity between two 3D faces is measured by the similarity between two sets of 3D curves on them. Due to the intrinsic property of geodesics, we select geodesics as the comparison curves. Firstly, the geodesics on each 3D facial model emanating from the nose tip point are extracted in the same initial direction with equal angular increment. Secondly, the Frechet distances between the two sets of geodesics on the two compared facial models are computed. At last, the similarity between the two facial models is computed based on the Frechet distances of the geodesics obtained in the second step. We verify our method both theoretically and practically. In theory, we prove that the similarity of our method satisfies three properties: reflexivity, symmetry, and triangle inequality. And in practice, experiments are conducted on the open 3D face database GavaDB, Texas 3D Face Recognition database, and our 3D face database. After the comparison with iso-geodesic and Hausdorff distance method, the results illustrate that our method has good discrimination ability and can not only identify the facial models of the same person, but also distinguish the facial models of any two different persons.