Selecting stable keypoints and local descriptors for person identification using 3D face scans

We propose and evaluate a three-dimensional (3D) face recognition approach that applies the speeded up robust feature (SURF) algorithm to the depth representation of shape index map, under real-world conditions, using only a single gallery sample for each subject. First, the 3D scans are preprocessed, then SURF is applied on the shape index map to find interest points and their descriptors. Each 3D face scan is represented by keypoints descriptors, and a large dictionary is built from all the gallery descriptors. At the recognition step, descriptors of a probe face scan are sparsely represented by the dictionary. A multitask sparse representation classification is used to determine the identity of each probe face. The feasibility of the approach that uses the SURF algorithm on the shape index map for face identification/authentication is checked through an experimental investigation conducted on Bosphorus, University of Milano Bicocca, and CASIA 3D datasets. It achieves an overall rank one recognition rate of 97.75%, 80.85%, and 95.12%, respectively, on these datasets.

Face recognition has lot of challenges in biometrics. The challenges are addressed effectively by local pattern descriptors. The idea of local descriptors is to determine the feature vector and then compute the difference between test images with training images by using similarity measure. Based on the observation, local approaches attained better performance rate than other approaches in face recognition. Due to that, researchers made a significant attention on local descriptors for face recognition. For nonlinear subspace, the local descriptors will achieve better result than holistic approach. Local pattern descriptor follows simple procedure to extract the facial features. The steps are face alignment, face representation, and matching. Face alignment is the first step of local descriptor, which is used to divide the image into several blocks. Face representation is used to extract the meaningful information from each region. This local feature extraction method carries discriminant information of the region; it will improve the classification rate and matching rate. Local descriptors extract the discriminant information from the neighbors by setting a threshold value as center pixel value, and it is not capable of extracting the detailed information from microstructure. Finally, matching by classification techniques or distance measure is used to identify or verify the person. The local pattern descriptors are more robust against pose, lighting, and scale variations. This chapter describes the various local pattern descriptors and shows the effectiveness of the descriptor. The results of local pattern descriptors are experimented on standard benchmark databases such as FERET, Extended Yale-B, ORL, CAS-PEAL, LFW, JAFFE, and Cohn–Kannade.

Three-dimensional object construction has seen a great deal of activity in the past decade, as has been pointed out in recent surveys, with the advancement of technology and easy availability of the depth sensors the 3D scanning technology has taken off. A wide range of commercial sensors such as Intel RealSense, Microsoft Kinect are being widely used for real time 3D capturing. Efficient 3D face scanning is one of the important areas of 3D scanning. 3D printer compatible texture supported scanning has a wide range of commercial applications. Such methods are also being used for 3D avatars and characterizations for games. Even though several commercial grade applications are available, most of the techniques suffer from background and light variations. Therefore an efficient face scanning technique is of extreme importance. In this paper we propose a 3D face scanning method based on Intel RealSense technology that combines 3D face detection, background segmentation and 3D mesh mapping to produce realistic 3D face model along with texture export. Further the models then can be manipulated using any 3D editing software along with texture and wireframe manipulation.

OBJECTIVE To compare the registration accuracy of three-dimensional (3D) facial scans for the design of full-arch implant supported restoration by five methods and to explore the suitable registration method. METHODS According to the criteria, ten patients with maxillary edentulous jaw or end-stage dentition requiring implant supported restorations were enrolled in this study. A special rim with individual feature marks reflected appropriate occlusal relationship and esthetic characteristics was made for each patient. Both 3D facial scan data of natural laughter and with opener traction to expose the teeth or occlusal rim of each patient were acquired by facial scan and input to the digital analysis software Geomagic Qualify 2012. The dataset was superimposed by five different methods: seven facial anatomical landmark points alignment, facial immobile area alignment (forehead and nasal area), facial anatomical landmark points and immobile area combining alignment, facial feature points alignment, facial and intraoral feature points alignment with the same local coordinate system. The three-dimensional deviation of the same selected area was calculated, the smaller the deviation, the higher the registration accuracy. The 3D deviation was compared among the three registration methods of facial anatomical landmark points, facial immobile area alignment and the combination of the above two methods. Friedman test was performed to analyze the difference among the three methods (α=0.05). The effect of the aid of the facial and intraoral feature points were evaluated. Paired t test were performed to analyze the difference (P<0.05). RESULTS The average three-dimensional deviation of the selected area after alignment with the facial anatomical landmarks was (1.501 2±0.406 1) mm, significantly larger than that of the facial immobile area best-fit alignment [(0.629 1±0.150 6) mm] and the combination of the two methods[(0.629 1±0.150 6) mm] (P < 0.001). The aid of the facial feature points could significantly reduce the deviation (t=1.001 3, P < 0.001). There was no significant statistical difference in the remaining groups. CONCLUSION The forehead area of the 3D facial scan can be exposed as much as possible. The establishment of facial characteristic landmark points and the use of the invariant area alignment can improve the accuracy of registration. It should be clinically feasible to apply three-dimensional facial scan to the design of full-arch implant supported restoration with the registration of the immobile area on the face especially the forehead area.

The use of three-dimensional (3D) facial scans for facial analysis is increasing in maxillofacial treatment. The aim of this study was to investigate the consistency of two-dimensional (2D) and 3D facial analyses performed by multiple raters. Six men and four women (25–36-year-old) participated in this study. The 2D images of the smiling and resting faces in the frontal and sagittal planes were obtained. The 3D facial and intraoral scans were merged to generate virtual 3D faces. Ten clinicians performed facial analyses by investigating 14 indices of 2D and 3D faces. Intra- and interrater agreements of the results of 2D and 3D facial analyses within and among the participants were evaluated. The intrarater agreement between the 2D and 3D facial analyses varied according to the indices. The highest and lowest agreements were found for the dental crowding index (0.94) and smile line curvature index (0.56) in the frontal plane, and Angle's classification (canine) index (0.98) and occlusal plane angle index (0.55) in the profile plane. In the frontal plane, the interrater agreements were generally higher for the 3D images than for the 2D images, while in the profile plane, the interrater agreements were high in the Angle's classification (canine) index however low in the other indices. Several occlusion-related indices were missing in the 2D images because the posterior teeth were not observed. Esthetic analysis results between 2D and 3D face images can differ according to the evaluation indices. The use of 3D faces is recommended over 2D images to increase the reliability of facial analyses, as it can fully assess both esthetic and occlusion-related indices.

Abstract-In order to eliminate the effect of the facial expression and illumination condition, as well as to speed up the recognition procedure, we propose a face recognition approach based on sparse representation. First, preprocessing and segmenting the face area from three dimensional (3D) face scans, we also apply coarse to fine registration to ensure the alignment of range images; second, mapping the 3D model to range image through a kind of geometry-based resampling method; finally, employ sparse representation classification method to identify 3D face. The experiment results in actual 3D face database demonstrate the effectiveness of the proposed method.

The first handicap in 3D faces recognizing under unconstrained problem is the largest variability of the visual aspect when we use various sources. This great variability complicates the task of identifying persons from their 3D facial scans and it is the most reason that bring to face detection and recognition of the major problems in pattern recognition fields, biometrics and computer vision. We propose a new 3D face identification and recognition method based on Gappy Wavelet Neural Network (GWNN) that is able to provide better accuracy in the presence of facial occlusions. The proposed approach consists of three steps: the first step is face detection. The second step is to identify and remove occlusions. Occluded regions detection is done by considering that occlusions can be defined as local face deformations. These deformations are detected by a comparison between the input facial test wavelet coefficients and wavelet coefficients of generic face model formed by the mean data base faces. They are beneficial for neighborhood relationships between pixels rotation, dilation and translation invariant. Then, occluded regions are refined by removing wavelet coefficient above a certain threshold. Finally, the last stage of processing and retrieving is made based on wavelet neural network to recognize and to restore 3D occluded regions that gathers the most. The experimental results on this challenging database demonstrate that the proposed approach improves recognition rate performance from 93.57 to 99.45 % which represents a competitive result compared to the state of the art.

Foetal alcohol syndrome (FAS) is a preventable condition caused by maternal alcohol consumption during pregnancy. The FAS facial phenotype is an important factor for diagnosis, alongside central nervous system impairments and growth abnormalities. Current methods for analysing the FAS facial phenotype rely on 3D facial image data, obtained from costly and complex surface scanning devices. An alternative is to use 2D images, which are easy to acquire with a digital camera or smart phone. However, 2D images lack the geometric accuracy required for accurate facial shape analysis. Our research offers a solution through the reconstruction of 3D human faces from single or multiple 2D images. We have developed a framework for evaluating 3D human face reconstruction from a single-input 2D image using a 3D face model for potential use in FAS assessment. We first built a generative morphable model of the face from a database of registered 3D face scans with diverse skin tones. Then we applied this model to reconstruct 3D face surfaces from single frontal images using a model-driven sampling algorithm. The accuracy of the predicted 3D face shapes was evaluated in terms of surface reconstruction error and the accuracy of FAS-relevant landmark locations and distances. Results show an average root mean square error of 2.62 mm. Our framework has the potential to estimate 3D landmark positions for parts of the face associated with the FAS facial phenotype. Future work aims to improve on the accuracy and adapt the approach for use in clinical settings. Significance: Our study presents a framework for constructing and evaluating a 3D face model from 2D face scans and evaluating the accuracy of 3D face shape predictions from single images. The results indicate low generalisation error and comparability to other studies. The reconstructions also provide insight into specific regions of the face relevant to FAS diagnosis. The proposed approach presents a potential cost-effective and easily accessible imaging tool for FAS screening, yet its clinical application needs further research.

In this paper, we present an automatic point matching method to overcome the dense point alignment of 3D face scans. We adopt TPS (thin plate spline) transformation to model the deformation of different 3D faces, because TPS is a type of non-rigid transformation with good smooth property and suitable for formulating the complexities of human facial morphology. Generally, TPS is derived from the interpolation between two sets of aligned controlling points. To obtain fully automatic point matching method, a random point selecting method is proposed to get the controlling points of 3D faces. Integrating the point generating method with an iterative closest point searching strategy, we achieve a point-to-point alignment solution for dense 3D face scans. The point matching tests on BJUT 3D face database reveal that the TPS based method has flexible shape deformation ability. Comparing with the hand-selecting controlling point method our method has better point matching accuracy and stability. It is proved that the presented method is efficient for dense point registration with non-rigid deformation.

Face recognition research mainly focuses on traditional 2D color images, which is extremely susceptible to be affected by external factors such as various viewpoints and has limited recognition accuracy. In order to achieve improved recognition performance, as well as the 3D face holds more abundant information than 2D, we present a 3D human face recognition algorithm using the Microsoft's Kinect. The proposed approach integrates the depth data with the RGB data to generate 3D face raw data and then extracts feature points, identifies the target via a two-level cascade classifier. Also, we build a 3D-face database including 16 individuals captured exclusively using Kinect. The experimental results indicate that the introduced algorithm can not only achieve better recognition accuracy in comparison to existing 2D and 3D face recognition algorithms when the probe face is exactly in front of Kinect sensor, but also can increase 9.3% of recognition accuracy compared to the PCA-3D algorithm when it is not confronting the camera.

Analysis of the influence of the facial scanning method on the transfer accuracy of a maxillary digital scan to a 3D face scan for a virtual facebow technique: An in vitro study

Facial symmetry is one of the number of aesthetic traits associated with health, physical attractiveness and beauty of a person. In this paper, we present a novel method for enhancing the symmetry of a scanned 3D face automatically. To handle detailed areas of the face, we developed a new local 3D shape descriptor based on facial features and surface curvatures. Our shape descriptor can improve the accuracy when deforming a 3D face towards a symmetric configuration, because it provides accurate point pairing with respect to the plane of symmetry. It can also reduce the computational complexity by dividing the given face into partitioned slices and facial feature regions. In addition, we use point‐based representation over all stages of symmetrization without generating a consistent triangle mesh or texture parameterization, which makes it much easier to support discrete processes such as symmetric deformation and to be combined with more sophisticated manipulations later such as cutting of surfaces. Finally, we performed a statistical analysis to assess subjects' preference for the symmetrized faces by our approach. Copyright © 2009 John Wiley &amp; Sons, Ltd.

Recently, the integration of 3D face scanning into smartphones has raised vast interest in plastic surgery. With the release of smartphones featuring 3D face scanning technology, users now can capture detailed 3D models of their faces using their smartphones. However, trueness and precision of this system is less well established. PubMed, Cochrane Library, Embase, ScienceDirect, Scopus, and Web of Science databases were searched for studies evaluating 3D scanning of smartphone devices and conventional 3D imaging systems from January 1, 2017, to June 1, 2023. A qualitative systematic review was conducted by two review authors after independently selecting studies, extracting data, and assessing the risk of bias of included studies. A total of 11 studies were included, all focusing on the accuracy of smartphone 3D facial scanning. The results show that although smartphones perform poorly on deep and irregular surfaces, they are accurate enough for clinical applications and have the advantage of being economical and portable. Smartphone-based 3D facial scanning has been basically validated for clinical application, showing broad clinical application prospects in plastic surgery. This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors www.springer.com/00266 .

The impact of natural movement/sway while standing still during the capture of a 3D face model for biometric applications has previously been believed to have a negligible impact on biometric performance. Utilizing a newly captured dataset this paper demonstrates a significant negative impact of standing. A 0.5 improvement in d' (test of correct/incorrect match distribution separation) per 3D face region and noticeable improvement to match distributions are shown to result from eliminating movement during the scanning process. By comparing these match distributions to those in the FRGC dataset this paper presents an argument for improving the accuracy of 3D face models by eliminating motion during the capture process.

Objective The aim of the current study was to investigate correlations between measurements on 3D facial soft tissue scan and CBCT skeletal structures. Methods The study sample consisted of 20 subjects living in Beijing area. A 3D face scan analysis including two distance values and 22 angle values based on 14 landmarks, and the corresponding CBCT analysis with three distance values and 9 angle values were performed for all subjects. All measurements were operated twice by the same researcher. Then, the reproducibility was evaluated by intraclass correlation coefficient (ICC), and the correlations were evaluated by Pearson correlation coefficient analysis. Besides, the thickness of facial soft tissue at relative points to explore possible reasons between the correlations. Results The reproducibility of measurements related to soft tissue gonion is medium. For sagittal measurement, medium correlations were found between soft tissue subspinale and hard tissue point A, as well as the relative position of maxilla and mandible. For vertical measurements, soft tissue gonions, mandibular plane angles showed medium correlations with hard tissue, and medium correlation were found between anterior upper and lower facial heights with corresponding hard tissue measurements and hard tissue mandibular plane angle. And the thickness of facial soft tissue at subspinale was larger than that at alare. Conclusions Soft tissue measurements of alare, the relative position between maxilla and mandible, upper anterior facial height and lower anterior facial height are reliable in providing a soft tissue diagnosis reference to the CBCT analysis. While the measurements related to soft tissue gonion should be used with caution. Key words: Three-dimensional facial soft tissue scan; CBCT; Correlation assessment between soft tissue and hard tissue; Skeletal class Ⅲ