３次元映像情報メディア技術 フリーハンド撮影された多視点スナップショットからの顔形状復元

We propose a method that can reconstruct both a 3D facial shape and camera poses from freehand multi-viewpoint snapshots. This method is based on Active Shape Model (ASM) using a facial shape database. Most ASM methods require an image in which the camera pose is known, but our method does not require this information. First, we choose an initial shape by selecting the model from the database which is most suitable to input images. Then, we improve the model by morphing it to fit the input images. Next, we estimate the camera poses using the morphed model. Finally we repeat the process, improving both the facial shape and the camera poses until the error between the input images and the computed result is minimized. Through experimentation, we show that our method reconstructs the facial shape within 3.5 mm of the ground truth.KeywordsFeature PointInput ImageFacial ShapeActive Shape ModelWorld Coordinate SystemThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Facial expression synthesizing is a process of generating new face shapes from a given face. The earlier work on synthesizing facial expressions used 2D images. Only recently, the work has moved to using 3D face shapes due to the availability and improvement of 3D scanner acquisition technology. This paper presents a work on synthesizing neutral facial expression or neutralizing facial expression on realistic 3D face models that holds distinct facial characteristics of the subjects. We apply Active Shape Model (ASM) to these dense face shapes. This approach could be used to improve the performance of existing biometric face identification systems, or to enhance human recognition.

Image warping and morphing are important visual effect tools in entertainment industry and other research fields. We developed a prototypical automatic facial image manipulation system (AFIM) for face morphing and shape normalization (warping). In our AFIM system, there are two main functions: (1) warping a facial image to a target image (face shape normalization), (2) generation of inter- or intrapersonal face by morphing between faces of different people or different faces of an individual. Image morphing is done by mesh warping scheme or multilevel B-spline approximation. The feature points are extracted by using active shape model (automaticextraction) or by manually. As an application, the AFIM is applied to statistical facial texture analysis.We first use AFIM to normalize the face shape in the facial database and then applied principal component analysis (PCA) to the shape-normalized facial images in order to analysis the texture of facial images statistically.

In this paper, we present an automatic face replacement approach in photographs based on Active Shape Models (ASM). Our replacement algorithm has three main modules: face alignment, face morph, and seamless blending. First, given an input and target image, we automatically implement face alignment using ASM. Second, we fit a morphable 2D model to the input face, and warp its shape to match the target face shape. Last, we adjust the lighting and color of the target face image to match those of the input face image, and seamlessly blend the target face image into the input image. Our approach is 2D model-based, fully automatic, and generates natural results. The improved results demonstrate the effectiveness of our approach.

In this paper, we present a novel strategy to design disentangled 3D face shape representation. Specifically, a given 3D face shape is decomposed into identity part and expression part, which are both encoded and decoded in a nonlinear way. To solve this problem, we propose an attribute decomposition framework for 3D face mesh. To better represent face shapes which are usually nonlinear deformed between each other, the face shapes are represented by a vertex based deformation representation rather than Euclidean coordinates. The experimental results demonstrate that our method has better performance than existing methods on decomposing the identity and expression parts. Moreover, more natural expression transfer results can be achieved with our method than existing methods.

Virtual orthognathic surgical planning involves simulating surgical corrections of jaw deformities on 3D facial bony shape models. Due to the lack of necessary guidance, the planning procedure is highly experience-dependent and the planning results are often suboptimal. A reference facial bony shape model representing normal anatomies can provide an objective guidance to improve planning accuracy. Therefore, we propose a self-supervised deep framework to automatically estimate reference facial bony shape models. Our framework is an end-to-end trainable network, consisting of a simulator and a corrector. In the training stage, the simulator maps jaw deformities of a patient bone to a normal bone to generate a simulated deformed bone. The corrector then restores the simulated deformed bone back to normal. In the inference stage, the trained corrector is applied to generate a patient-specific normal-looking reference bone from a real deformed bone. The proposed framework was evaluated using a clinical dataset and compared with a state-of-the-art method that is based on a supervised point-cloud network. Experimental results show that the estimated shape models given by our approach are clinically acceptable and significantly more accurate than that of the competing method.

In this paper, we present a system for synthesizing 3D human face models containing different expressions from a single facial image. Given a frontal image of the target face with neutral expression, we first detect several key points denoting the shape of the face by Active Shape Model (ASM). Then we apply a RBF-based scattered data interpolation to reconstruct a 3D target face using a neutral expression 3D face model as reference. By analyzing a series of 3D expression face model, we segment the 3D reference model into regions automatically that each region is correspondent to a facial organ. From the expression set we construct a motion model for each facial action with respect to the target face in a local consistent manner. At last, the reconstructed 3D target face model with neutral expression and the facial action motion model are combined to generate 3D target face of various expressions. There are 3 contributions of our work: (1) We employ a set of registered 3D facial expression models as input, which enabled us to generate more complex and visual-realistic expressions than other parameter-based approaches and 2D image-based methods. (2) On the basis of a clustering-based segmentation, we developed a localized linear expression model, which make it possible for us to generate different facial expressions both locally and globally, thusly enlarge the space of synthesize output and break the limitation by the limited scale of the input expression model set. (3) A local space transform procedure is included that the output expression can fit distinct facial shapes regardless of the scarcity of variation of the facial shapes (fat or thin) in the input model set.

A reported 20% of dental staff will fail their fit test for a disposable FFP3 respirator. This needs to be factored into future pandemic workforce and PPE supply planning. At present there are no scientifically or universally accepted facial shape criteria to design and produce facial masks that will fit the entire work force. This study presents differences in facial shape, volume and surface area between individuals who passed on several FFP3 masks (pass group) and participants who passed on only one FFP3 mask (fail group). Three dimensional images of 50 individuals, 25 in each group, were taken at rest and at maximum smile using a DI4D SNAP 6200 camera system. The images were processed, and four “average faces” were produced—pass group at rest, fail group at rest, pass group at maximum smile and fail group at maximum smile. Simple Euclidian linear and angular measurements, geodesic surface distances and volume and surface area enclosed within the mask were analysed. The results of the study show that individuals who are more likely to pass a mask fit test have longer faces, wider mouths, greater geodesic surface distances and a greater volume and surface area of soft tissue enclosed within the mask boundary. This would suggest that some manufactures masks may be too large, and they need to reduce the size of their masks or produce a category of sizes, accepting the fact that one size does not fit all.

Based on the assumption that a class of objects or data can be represented as a vector space spanned by a set of examples, we present a general method to estimate vector components of a novel vector, given only a subset of its dimensions. We apply this method to recover 3D shape of human faces from 2D image positions of a small number of feature points. The application demonstrates two aspects of the estimation of novel vector components: (1) From 2D image positions, we estimate 3D coordinates, and (2) from a small set of points, we obtain vertex positions of a high-resolution surface mesh. We provide an evaluation of the technique on laser scans of faces, and present an example of 3D shape reconstruction from a photograph. Our technique involves a tradeoff between reconstruction of the given measurements, and plausibility of the result. This is achieved in a Bayesian approach, and with a statistical analysis of the examples.

Efficacy of long-term treatment with lurasidone in children and adolescents with bipolar depression: Interim analysis of a 2-year open-label study

Obtaining the segmentation of an object in a sequence of images is usually achieved using a tracking methodology. However, in some applications, the whole sequence is available beforehand. This means that the segmentations can be determined simultaneously for all the frames in the sequence and taking into account the motion of the object. This paper proposes a new framework to incorporate motion information in the segmentation of image sequences using an active shape model (ASM). The motion of the object is modeled using a vector field, which is learned and refined online as the segmentation algorithm proceeds. The vector field is determined from the trajectories described by ASM points throughout the sequence. The vector field, in turn, influences the estimation of the ASM parameters by acting as a regularizer, ensuring that the segmentations are in agreement with the expected motion. The results show that coupling these models during the segmentation leads to an increase in performance, in particular by guarantee more consistent segmentations and by avoid gross errors in more challenging frames.

This paper presents a novel method for face alignment under unknown head poses and nonrigid warp, within the framework of markov random field. The proposed method learns a 3D face shape model comprised of 31 facial features and a texture model for each facial feature from a 3D face database. The models are combined to serve as the unary, pairwise and high order constraints of the markov random field. The facial features are located by minimizing the potential function of markov random field, which is solved with dual decomposition. The main contribution of this paper is composed of three aspects. First, Random Project Tree is utilized to learn the manifold structure of the facial feature appearance under different view points. Second, a 3D face shape model is learned to capture the linear part of face shape distribution due to the change of identity and expression, which is served as the high order constraints. Third, Markov random field is introduced to model the nonlinear part of the face shape distribution, and also to deal with occlusion of facial features due to head pose variation or ornaments. Experiments was taken on the Texas 3D face database and face images downloaded from the Internet, results shows capability of adapting large head pose variations of the method.KeywordsFace ImageFacial FeatureMarkov Random FieldFace ShapeTexture ModelThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Laparoscopic surgery is a widely used minimally invasive technique to treat a variety of pathologies within the abdominopelvic region. Compared to conventional open surgery, laparoscopic surgery reduces iatrogenic trauma and hospital stay length. Nevertheless, surgeons have a limited field of view during operation because surgical instruments are inserted through the patient's body via small incisions. To improve surgeon's perception, surgical navigation techniques are implemented to improve visualization. A number of surgical navigation systems being developed are integrated with computer algorithms to enhance their efficiency. In this work, we generate novel views, depths, and camera poses of corresponding laparoscopic images to augment our dataset, which can then be used to train and evaluate learning-based computer vision algorithms such as simultaneous localization and mapping (SLAM) and visual odometry. Our existing dataset was generated from porcine tissue images using a laparoscope and an optical tracking system to capture camera poses. We performed tissue surface reconstruction of the dataset via the NeRF-SLAM algorithm. Camera poses were transformed to acquire novel views of RGB laparoscopic images and their corresponding depth images. The results show that the simulated RGB images have an average peak signal-to-noise ratios (PSNR) between 29.17 - 29.79 dB, structural similarity index measure (SSIM) value between 0.6061 - 0.6174, and learned perceptual image patch similarity (LPIPS) value between 0.568 - 0.592 compared to the corresponding input images. This work provides a foundation for the generation of synthetic datasets, which could be useful for learning-based autonomous surgical navigation systems.

We propose an approach to predict accuracy for three-dimensional reconstruction and camera pose using a generic RGB-D camera on a robotic platform. We initially create a ground truth of 3D points and camera poses using a set of smart markers that we specifically devised and constructed for our approach. Then, we compute actual errors and their accuracy during the motion of our mobile robotic platform. The error modeling is then provided, which is used as input to a deep multi-layer perceptron in order to estimate accuracy as a function of the camera’s distance, velocity, and vibration of the vision system. The network outputs are the root mean squared errors for the 3D reconstruction and the relative pose errors for the camera. Experimental results show that this approach has a prediction accuracy of ±1% for the 3D reconstruction and ±2.5% for camera poses, which shows a better performance in comparison with state-of-the-art methods.

Over the last decade, it has become increasingly affordable to digitize 2D and 3D shape information using multiple modalities, such as (video) cameras, image-based reconstruction systems, laser-range scanners, or depth cameras. If these dense models can be processed and described in an efficient and informative way, they can be used in applications, such as ergonomic design, virtual shopping, scientific and medical visualization, realistic simulation, photo-realistic rendering, the design of natural user interfaces, and semantic scene understanding. Traditionally, the notion of shape has been studied either by analyzing projections of shapes in images or by analyzing a sparse set of marker positions on 3D shapes. Typical tasks in 2D shape analysis include segmenting objects in images and tracking objects across a sequence of images, and typical tasks in 3D shape analysis include reconstructing the three-dimensional object depth from input images and identifying corresponding points on different 3D models. The analysis and processing of shape data becomes especially challenging because of the increasing amount of data captured by sensors used to acquire shapes, and because modern applications such as natural user interfaces require real-time processing of the input shapes. Meeting these challenges requires models of shape analysis that are compact and informative, thereby allowing the development of algorithms that can process large datasets efficiently. To achieve these goals, interdisciplinary approaches are needed that use concepts from a variety of research areas, including numerical computing, differential geometry, deformable shape modeling, sparse data representation, and machine learning. On the algorithmic side, many shape analysis tasks are modeled using partial differential equations, which can be solved using tools from the field of numerical computing. The fields of differential geometry and deformable shape modeling have recently begun to influence shape analysis methods. Furthermore, tools from the field of sparse representations, which aim to describe input data using a compressible representation with respect to a set of carefully selected basis elements, have the potential to significantly reduce the amount of data that needs to be processed in shape analysis tasks. The related field of machine learning offers similar potential. This seminar brought together 28 researchers from North America and Europe engaged in recent and upcoming developments in shape analysis who view these challenges from different perspectives and who together discussed the pressing open problems and novel solutions to them.