Hierarchical Feature-Pair Relation Networks for Face Recognition

Facial age estimation is of interest due to its potential to be applied in many real-life situations. However, recent age estimation efforts do not consider juveniles. Consequently, we introduce a juvenile age detection scheme called LaGMO, which focuses on the juvenile aging cues of facial shape and appearance. LaGMO is a combination of facial landmark points and Term Frequency Inverse Gravity Moment (TF-IGM). Inspired by the formation of words from morphemes, we obtained facial appearance features comprising facial shape and wrinkle texture and represented them as terms that described the age of the face. By leveraging the implicit ordinal relationship between the frequencies of the terms in the face, TF-IGM was used to compute the weights of the terms. From these weights, we built a matrix that corresponds to the possibilities of the face belonging to the age. Next, we reduced the reference matrix according to the juvenile age range (0–17 years) and avoided the exhaustive search through the entire training set. LaGMO detects the age by the projection of an unlabeled face image onto the reference matrix; the value of the projection depicts the higher probability of the image belonging to the age. With Mean Absolute Error (MAE) of 89% on the Face and Gesture Recognition Research Network (FG-NET) dataset, our proposal demonstrated superior performance in juvenile age estimation.

Human face recognition is one of the most important research areas in biometrics. However, the robust face recognition under a drastic change of the facial pose, expression, and illumination is a big challenging problem for its practical application. Such variations make face recognition more difficult. In this paper, we propose a novel face recognition method, called Attentional Feature-pair Relation Network (AFRN), which represents the face by the relevant pairs of local appearance block features with their attention scores. The AFRN represents the face by all possible pairs of the 9x9 local appearance block features, the importance of each pair is considered by the attention map that is obtained from the low-rank bilinear pooling, and each pair is weighted by its corresponding attention score. To increase the accuracy, we select top-K pairs of local appearance block features as relevant facial information and drop the remaining irrelevant. The weighted top-K pairs are propagated to extract the joint feature-pair relation by using bilinear attention network. In experiments, we show the effectiveness of the proposed AFRN and achieve the outstanding performance in the 1:1 face verification and 1:N face identification tasks compared to existing state-of-the-art methods on the challenging LFW, YTF, CALFW, CPLFW, CFP, AgeDB, IJB-A, IJB-B, and IJB-C datasets.

Recently, facial expression recognition has attracted much attention in machine vision research because of its various applications. Accordingly, many facial expression recognition systems have been proposed. However, the majority of existing systems suffer from a critical problem: geometric variability. It directly affects the performance of geometric feature-based facial expression recognition approaches. Furthermore, it is a crucial challenge in appearance feature-based techniques. This variability appears in both neutral faces and facial expressions. Appropriate face geometry normalization can improve the accuracy of each facial expression recognition system. Therefore, this paper proposes different geometric models or shapes for normalization. Face geometry normalization removes geometric variability of facial images and consequently, appearance feature extraction methods can be accurately utilized to represent facial images. Thus, some expression-based geometric models are proposed for facial image normalization. Next, local binary patterns and local phase quantization are used for appearance feature extraction. A combination of an effective geometric normalization with accurate appearance representations results in more than a 4% accuracy improvement compared to several state-of-the-arts in facial expression recognition. Moreover, utilizing the model of facial expressions which have larger mouth and eye region sizes gives higher accuracy due to the importance of these regions in facial expression.

In this paper, we propose a novel face recognition method based on local appearance feature extraction using dual-tree complex wavelet transform (DT-CWT). It provides a local multiscale description of images with good directional selectivity and invariance to shifts and in-plane rotations. In the dual-tree implementation, two parallel discrete wavelet transform (DWT) with different lowpass and highpass filters in different scales were used. The linear combination of subbands generated by two parallel DWT is used to generate 6 different directional subbands with complex coefficients. It is insensitive to illumination variations and facial expression changes. 2-D dual-tree complex wavelet transform is less redundant and computationally efficient. The local DT-CWT coefficients are used to extract the facial features which improve the face recognition with small sample size in less computation. The local features based methods have been successfully applied to face recognition and achieved state-of-the-art performance. Normally most of the local appearance based methods the facial features are extracted from several local regions and concatenated into an enhanced feature vector as a face descriptor. In this approach we divide the face into several (m ? m) non-overlapped parallelogram blocks instead of square or rectangle blocks. The local mean, standard deviation and energy of complex wavelet coefficients are used to describe the face image. Experiments, on two well-known databases, namely, Yale and ORL databases, shows the Local DT-CWT approach performs well on illumination, expression and perspective variant faces with single sample compared to PCA and global DT-CWT. Furthermore, in addition to the consistent and promising classification performances, our proposed Local DT-CWT based method has a really low computational complexity.

Local fusion of complex dual-tree wavelet coefficients based face recognition for single sample problem

OBJECTIVE To compare the registration accuracy of three-dimensional (3D) facial scans for the design of full-arch implant supported restoration by five methods and to explore the suitable registration method. METHODS According to the criteria, ten patients with maxillary edentulous jaw or end-stage dentition requiring implant supported restorations were enrolled in this study. A special rim with individual feature marks reflected appropriate occlusal relationship and esthetic characteristics was made for each patient. Both 3D facial scan data of natural laughter and with opener traction to expose the teeth or occlusal rim of each patient were acquired by facial scan and input to the digital analysis software Geomagic Qualify 2012. The dataset was superimposed by five different methods: seven facial anatomical landmark points alignment, facial immobile area alignment (forehead and nasal area), facial anatomical landmark points and immobile area combining alignment, facial feature points alignment, facial and intraoral feature points alignment with the same local coordinate system. The three-dimensional deviation of the same selected area was calculated, the smaller the deviation, the higher the registration accuracy. The 3D deviation was compared among the three registration methods of facial anatomical landmark points, facial immobile area alignment and the combination of the above two methods. Friedman test was performed to analyze the difference among the three methods (α=0.05). The effect of the aid of the facial and intraoral feature points were evaluated. Paired t test were performed to analyze the difference (P<0.05). RESULTS The average three-dimensional deviation of the selected area after alignment with the facial anatomical landmarks was (1.501 2±0.406 1) mm, significantly larger than that of the facial immobile area best-fit alignment [(0.629 1±0.150 6) mm] and the combination of the two methods[(0.629 1±0.150 6) mm] (P < 0.001). The aid of the facial feature points could significantly reduce the deviation (t=1.001 3, P < 0.001). There was no significant statistical difference in the remaining groups. CONCLUSION The forehead area of the 3D facial scan can be exposed as much as possible. The establishment of facial characteristic landmark points and the use of the invariant area alignment can improve the accuracy of registration. It should be clinically feasible to apply three-dimensional facial scan to the design of full-arch implant supported restoration with the registration of the immobile area on the face especially the forehead area.

In this paper, we present an extensive evaluation of face recognition and verification approaches performed by the European COST Action MULTI-modal Imaging of FOREnsic SciEnce Evidence (MULTI-FORESEE). The aim of the study is to evaluate various face recognition and verification methods, ranging from methods based on facial landmarks to state-of-the-art off-the-shelf pre-trained Convolutional Neural Networks (CNN), as well as CNN models directly trained for the task at hand. To fulfill this objective, we carefully designed and implemented a realistic data acquisition process, that corresponds to a typical face verification setup, and collected a challenging dataset to evaluate the real world performance of the aforementioned methods. Apart from verifying the effectiveness of deep learning approaches in a specific scenario, several important limitations are identified and discussed through the paper, providing valuable insight for future research directions in the field.

Video Face Detection Using Bayesian Technique

Face recognition methods, which usually represent face images using holistic or local facial features, rely heavily on alignment. Their performances also suffer a severe degradation under variations in expressions or poses, especially when there is one gallery per subject only. With the easy access to high-resolution (HR) face images nowadays, some HR face databases have recently been developed. However, few studies have tackled the use of HR information for face recognition or verification. In this paper, we propose a pose-invariant face-verification method, which is robust to alignment errors, using the HR information based on pore-scale facial features. A new keypoint descriptor, namely, pore-Principal Component Analysis (PCA)-Scale Invariant Feature Transform (PPCASIFT)-adapted from PCA-SIFT-is devised for the extraction of a compact set of distinctive pore-scale facial features. Having matched the pore-scale features of two-face regions, an effective robust-fitting scheme is proposed for the face-verification task. Experiments show that, with one frontal-view gallery only per subject, our proposed method outperforms a number of standard verification methods, and can achieve excellent accuracy even the faces are under large variations in expression and pose.

The main challenge of Multiple Object Tracking (MOT) is that there is great uncertainty in data association when using the tracked predicted values and tracked trajectories. Meanwhile, the MOT is complex and time‐consuming. When equipment resources are limited or real‐time requirement is high, its application is very limited. Therefore, we propose a Lightweight Deep Appearance Embedding (LDAE) to assist the association of trajectories. Firstly, in addition to motion information in data association, we also introduce more discriminative appearance features to participate in the affinity measure to effectively distinguish similar targets. Secondly, according to the idea of feature mapping, we design a lightweight deep appearance embedding module. It can help extract appearance features with less computation. Finally, we propose a simulated occlusion strategy for the training of the LDAE, which helps improve the ability to recognise different targets in dense scenes. The LDAE dramatically reduces the computational cost and improves the accuracy of data association. Extensive experiments are conducted on the MOT datasets (MOT16, MOT17 and MOT20), which prove that the LDAE outperforms several state‐of‐the‐art trackers in the tracking accuracy and anti‐occlusion performance. Furthermore, we apply the LDAE to escalators, which can achieve fast and stable tracking effect.

For the past years, studies on face recognition were focused on the effects and influence of illumination, pose, and angle to the performance, which, in fairness, has been proven by different research works as the dominant factor of adding complexity to face recognition across age progression observed through photographs, specifically headshots (Ling, Soatto, Ramanathan, & Jacobs, 2007; Ramanathan & Chellappa, 2006; Ramanathan & Chellappa, 2005). However, studies on face verification and recognition with time lapses between pairs of photographs and the interaction on the sexual orientation of the identifier was somehow defocused on the factor of effects of age progression or the time interval between pairs of facial images on face recognition. Thus, the researchers aimed to resolve the gap on the prevailing literatures on face recognition by finding the effects of age interval of the stimuli and sexual orientation of the participants on face recognition using Filipino college students as sample. Results showed that the greater the age gaps between the pair of images, the lesser the chance of identifying it. In addition, it is also not important to consider whether identifier is male or female in terms of recognizing picture because based from the result of the experiment, sexual orientation does not have an effect on the performance of the participants in face recognition. The study was guided by the theory of Biederman (1987) and Recognition By Components (RBC) to analyze the findings.

Multi-shot human re-identification is a major challenge because of the large variations in a human’s appearance caused by different types of noise such as occlusion, viewpoint and illumination variations. In this paper, we presented a novel Gabor-LBP based video covariance descriptor, called GL-VC descriptor, which considers image sequences to extract appearance features, captures moving regions of interest and find the correlation between video frames. Therefore, it implicitly encodes the described human motion by the integration of temporal information and decreases the effect of occlusion. To deal with the changes of view points and illumination, the Local binary pattern (LBP) operators and Gabor bank were integrated into the spatio-temporal covariance features. We evaluated our GL-VC approach on the publicly available CAVIAR4REID multi-shot dataset and demonstrated superior performance in comparison with the current state-of-the-art.

Aiming at the problem of low pedestrian tracking accuracy caused by illumination and occlusion in intelligent surveillance videos, a pedestrian tracking algorithm based on Work-Yolo detection combined with DeepSORT is proposed. To improve the accuracy of the detector, the attention module CBAM is used to fuse with the Backbone and Neck parts of Yolov5s network for enhancing pedestrian features. The BiAdd structure is used to fuse features of different scales about BiFPN, Dilated ConV is proposed to reduce the number of model parameters and extract better shallow features. Work-Yolo head decoupled head separates prediction classification and regression tasks, solving the problem of missed detection when pedestrians are obscured. The Lite-shufflenet lightweight structure is proposed to extract appearance features, and retrain the pedestrian re-identification dataset to reduce the identity switching caused by pedestrian occlusion. Pedestrian detection experiments are conducted on 3247 intersection pedestrian datasets, and the final detection accuracy rate was 94.2% and the recall rate was 90.6%. The video taken at the indoor entrance and exit of the scene, four videos are randomly selected for multi-target tracking experiments, the MOTA is improved by 10%, and the detection speed reaches 15fps, which meets the requirements of industrial applications.

The multi-object tracking (MOT) algorithms of the joint detection and embedding (JDE) approach estimate bounding boxes and re-identification (re-ID) features of objects with the single network, which balance the tracking accuracy and inference speed. However, when the appearance information between different objects is highly similar, these algorithms are usually easy to cause identity switches, and the comprehensive tracking performance is poor in crowded scenes. Aiming at the above problems, we propose a stronger multi-object tracking algorithm termed as ReIMOT, based on FairMOT. A joint loss function of combining normalized Softmax Loss and the center distance penalty term is designed to supervise the re-ID branch, which increases the intra-class similarity and makes the extracted appearance features more discriminative. To further improve the tracking performance, we introduce coordinate attention to make the encoder-decoder network focus more on features of interest. The experimental results show that the proposed ReIMOT is more effective than the other advanced multi-object tracking algorithms, and decreases the number of ID switches by 13.8% compared to FairMOT on the MOT17 dataset.

Video instance segmentation (VIS) mainly has two methods in human-targeted tasks, one is based on mask propagation method, and the other is based on detection tracking mode method. Existing datasets pay less attention to all the characters appearing in the video, but focus on the key characters appearing in the video. Therefore, in this paper, an online joint detection and tracking model is proposed. Based on the single-stage segmentation method SOLO, appearance feature extraction and data association are added to realize the segmentation and tracking of human body in video. At the same time, in order to better evaluate the method, we extract a single person type on the basis of existing multi-type data sets, filter out high-quality annotations and various video scenes, and obtain the challenging data set PVIS. As a result, the method based on the PVIS dataset can achieve satisfied performance both for segmentation accuracy and tracking accuracy.KeywordsSOLOVideo instance segmentationPerson Re-IDMulti-object tracking