Detection of Occluded Face Image using Mean Based Weight Matrix and Support Vector Machine

The purpose of the paper is to study the holistic image transformation methods for person's identity authentication from a thermographic image of faces. Within this study the datasets of images of faces in the far infrared range (LWIR) have been collected. The novelty of the study consists in the features of the dataset of images which collected in real conditions that affect the quality of authentication, such as facial expressions, wearing glasses or a medical mask, applying makeup / cosmetics, different illumination and temperature conditions of the environment, head turns. The methods under the study are based on the construction and selection of image features while reducing the dimension and converting the image into another form of representation. These methods are used to solve the problem of distinguishing features in images, in authentication person identity by 2D image of his face and allows solving other computer vision problems. This paper discusses classical methods of integral image transformation: principal component analysis, principal component analysis using a kernel, linear descriptor analysis, independent component analysis, truncated singular value decomposition, discrete cosine transformation. As a measure of the proximity of images, the Euclidean distance between the vectors of image features is used. Testing of the methods was performed on a set of thermograms consisting of 632 thousand images of the faces of 158 people. The F-measure was used as a metric to assess the quality of comparison of the selected methods. As a result of the experiment, the independent component analysis method showed the highest values of the F-measure metric – 0.72. The results of the study can find applications in access control and control systems to increase the fault tolerance of authentication of persons. The use of the considered methods is effective in the tasks of processing thermographic images for authenticating a person by secondary signs, by the pattern of his veins and vessels on the face, in cases of changes in facial expressions and appearance by means of applying makeup and wearable accessories.

A gender classification system robust to occlusion using Gabor features based (2D)2PCA

In this paper, a novel occlusion invariant face recognition algorithm based on Mean based weight matrix (MBWM) technique is proposed. The proposed algorithm is composed of two phases—the occlusion detection phase and the MBWM based face recognition phase. A feature based approach is used to effectively detect partial occlusions for a given input face image. The input face image is first divided into a finite number of disjointed local patches, and features are extracted for each patch, and the occlusion present is detected. Features obtained from the corresponding occlusion-free patches of training images are used for face image recognition. The SVM classifier is used for occlusion detection for each patch. In the recognition phase, the MBWM bases of occlusion-free image patches are used for face recognition. Euclidean nearest neighbour rule is applied for the matching. GTAV face database that includes many occluded face images by sunglasses and hand are used for the experiment. The experimental results demonstrate that the proposed local patch-based occlusion detection technique works well and the MBWM based method shows superior performance to other conventional approaches.

The challenges imposed by the Single Sample Per Person face recognition problem become especially critical when face images are captured in uncontrolled environments, which typically involve variations in illumination, facial expression, pose, occlusion, and more. In particular, illumination changes caused by poor lighting conditions can significantly degrade the performance of face recognition systems. In this paper, we investigate the SSPP face recognition problem in the presence of potential illumination variation and propose a two-fold approach. Firstly, we analyse and quantify the illumination variations in face images and then normalise these variations based on a probabilistic image enhancement approach. Subsequently, the feature embedding process was performed using the deep learning-based FaceNet system, and finally, faces were classified using the classical Support Vector Machine. The performance of the proposed approach is demonstrated through comprehensive experiments compared to state-of-the-art techniques. The proposed method achieved significant accuracy improvements, attaining 96.17% and 95.68% for 20 and 30 subjects, respectively, on the Extended Yale-B dataset. The performance of the proposed method is evaluated and exhibits superiority in handling illumination compared to state-of-the-art counterparts.

The process of aging causes significant alterations in the facial appearance of individuals. When compared with other sources of variation in face images, appearance variation due to aging displays some unique characteristics. Changes in facial appearance due to aging can even affect discriminatory facial features, resulting in deterioration of the ability of humans and machines to identify aged individuals. We describe how the effects of aging on facial appearance can be explained using learned age transformations and present experimental results to show that reasonably accurate estimates of age can be made for unseen images. We also show that we can improve our results by taking into account the fact that different individuals age in different ways and by considering the effect of lifestyle. Our proposed framework can be used for simulating aging effects on new face images in order to predict how an individual might look like in the future or how he/she used to look in the past. The methodology presented has also been used for designing a face recognition system, robust to aging variation. In this context, the perceived age of the subjects in the training and test images is normalized before the training and classification procedure so that aging variation is eliminated. Experimental results demonstrate that, when age normalization is used, the performance of our face recognition system can be improved.

Over the course of the last decade, infrared (IR) and particularly thermal IR imaging based face recognition has emerged as a promising complement to conventional, visible spectrum based approaches which continue to struggle when applied in the real world. While inherently insensitive to visible spectrum illumination changes, IR images introduce specific challenges of their own, most notably sensitivity to factors which affect facial heat emission patterns, e.g. emotional state, ambient temperature, and alcohol intake. In addition, facial expression and pose changes are more difficult to correct in IR images because they are less rich in high frequency detail which is an important cue for fitting any deformable model. In this paper we describe a novel method which addresses these major challenges. Specifically, to normalize for pose and facial expression changes we generate a synthetic frontal image of a face in a canonical, neutral facial expression from an image of the face in an arbitrary pose and facial expression. This is achieved by piecewise affine warping which follows active appearance model (AAM) fitting. This is the first publication which explores the use of an AAM on thermal IR images; we propose a pre-processing step which enhances detail in thermal images, making AAM convergence faster and more accurate. To overcome the problem of thermal IR image sensitivity to the exact pattern of facial temperature emissions we describe a representation based on reliable anatomical features. In contrast to previous approaches, our representation is not binary; rather, our method accounts for the reliability of the extracted features. This makes the proposed representation much more robust both to pose and scale changes. The effectiveness of the proposed approach is demonstrated on the largest public database of thermal IR images of faces on which it achieved 100% identification rate, significantly outperforming previously described methods.

Emotion detection is one of the most complex and challenging computer visuals due to significant variations caused by changes in facial appearance, lighting, and facial expressions. The results of tests conducted by Amanullah et al. on the classification of emotions using the Convolutional Neural Network show an accuracy value of 81.92% for training and 81.69% for testing but still has a fairly large error rate of 75.60% based on the loss validation results. In this study, the research to be conducted is to study more deeply about detecting human emotions based on facial expressions using the Convolutional Neural Network method. To detect human emotions based on faces using a Canon EOS 60D camera with good output quality. The camera is placed in front of the class. The computer user uses the Windows 10 64bit operating system, with a Core i5 processor, 16BG RAM, and GTX1060 VGA. Four LED lights are used every 18 watts to overcome the room lighting. Every 60 seconds, the system will carry out the process of detecting and classifying emotions and storing them in the database. The data obtained from the test results will be analyzed to obtain data in the form of accuracy, precision, and recall. The dataset for the training data used is FER-2013 which contains 28,000 datasets taken from Kaggle. The research results obtained were to obtain a high accuracy value that exceeded the results tested by Amanullah et al. The research results that will be carried out are also expected to be used by teachers to detect students' emotional conditions during class learning.

A bi-modal face recognition framework integrating facial expression with facial appearance

This study was carried out with the aim of exploring the full-convolution Siamese network (SiamFC) in the application of neonatal facial video image tracking, achieving accurate recognition of neonatal pain and helping doctors evaluate neonatal emotions in an automatic manner. The current technology shows low accuracy on facial image recognition of newborns, so the SiamFC algorithm under the deep learning was optimized in this study. Besides, a newborn facial video image tracking model (FVIT model) was constructed based on the SiamFC algorithm in combination with the attention mechanism with face tracking algorithm, and the facial features of newborns were tracked and recognized. In addition, a newborn face database was constructed based on the adult face database to evaluate performance of the FVIT model. It was found that the accuracy of the improved algorithm is 0.889, higher by 0.036 in contrast to other models; the area under the curve (AUC) of success rate reaches 0.748, higher by 0.075 compared with other algorithms. What’s more, the improved algorithm shows good performance in tracking the facial occlusion, facial expression changes, and scale conversion of newborns. Therefore, the improved algorithm shows higher accuracy and success rate and has good effect in capturing and tracking the facial images of newborns, thereby providing an experimental basis for facial recognition and pain assessment of newborns in the later stage.

This paper presents support vector machine (SVM) with local summation kernel for robust face recognition. In recent years, the effectiveness of SVM and local features is reported. However, conventional methods apply one kernel to global features. The effectiveness of local features is not used in those methods. In order to use the effectiveness of local features in SVM, one kernel is applied to local features. It is necessary to compute one kernel value from local kernels in order to use the local kernels in SVM. In this paper, the summation of local kernels is used because it is robust to occlusion. The robustness of the proposed method under partial occlusion is shown by the experiments using the occluded face images. In addition, the proposed method is compared with the global kernel based SVM. The recognition rate of the proposed method is over 80% under large occlusion, while the recognition rate of the SVM with global Gaussian kernel decreases dramatically.

In spite of the differences in visual stimulus of human beings such as ageing, changing conditions of a person, and occlusion, recognition can even be done at a glance by the human eye many years after the previous encounter. It has been established that facial differences like the hairstyle changes, growing of one’s beard, wearing of glasses, and other forms of occlusions can hardly hinder the power of the human brain from making a face recognition. However, the same cannot easily be said about automated intelligent systems which have been developed to mimic the skill of the human brain to aid in recognition. There have been growing interests in developing a resilient and efficient recognition system mainly because of its numerous application areas (access control, entertainment/leisure, security system based on biometric data, and user-friendly human-machine interfaces). Although there have been numerous researches on face recognition under varying pose, illumination, expression, and image degradations, problems caused by occlusions are mostly ignored. This study thus focuses on facial occlusions and proposes an enhancement mechanism through face image augmentation to improve the recognition of occluded face images. This study assessed the performance of Principal Component Analysis with Singular Value Decomposition using Fast Fourier Transform (FFT-PCA/SVD) for preprocessing face recognition algorithm on face images with missingness and augmented face image database. It was found that the average recognition rates for the FFT-PCA/SVD algorithm were the same ( 90 % ) when face images with missingness and augmented face images were used as test images, respectively. The statistical evaluation revealed that there exists a significant difference in the average recognition distances for the face images with missingness and augmented face images when FFT-PCA/SVD is used for recognition. Augmented face images tend to have a relatively lower average recognition distance when used as test images. This finding is contrary to the equal performance assessment by the adopted numerical technique. The MICE algorithm is therefore recommended as a suitable imputation mechanism for enhancing/improving the performance of the face recognition system.

Facial appearance changes with the movements of bony segments in orthognathic surgery of patients with craniomaxillofacial (CMF) deformities. Conventional bio-mechanical methods, such as finite element modeling (FEM), for simulating such changes, are labor intensive and computationally expensive, preventing them from being used in clinical settings. To overcome these limitations, we propose a deep learning framework to predict post-operative facial changes. Specifically, FC-Net, a facial appearance change simulation network, is developed to predict the point displacement vectors associated with a facial point cloud. FC-Net learns the point displacements of a pre-operative facial point cloud from the bony movement vectors between pre-operative and simulated post-operative bony models. FC-Net is a weakly-supervised point displacement network trained using paired data with strict point-to-point correspondence. To preserve the topology of the facial model during point transform, we employ a local-point-transform loss to constrain the local movements of points. Experimental results on real patient data reveal that the proposed framework can predict post-operative facial appearance changes remarkably faster than a state-of-the-art FEM method with comparable prediction accuracy.

Facial occlusions, due for example to sunglasses, hats, scarf, beards etc., can significantly affect the performance of any face recognition system. Unfortunately, the presence of facial occlusions is quite common in real-world applications especially when the individuals are not cooperative with the system such as in video surveillance scenarios. While there has been an enormous amount of research on face recognition under pose/illumination changes and image degradations, problems caused by occlusions are mostly overlooked. The focus of this paper is thus on facial occlusions, and particularly on how to improve the recognition of faces occluded by sunglasses and scarf. We propose an efficient approach which consists of first detecting the presence of scarf/sunglasses and then processing the non-occluded facial regions only. The occlusion detection problem is approached using Gabor wavelets, PCA and support vector machines (SVM), while the recognition of the non-occluded facial part is performed using block-based local binary patterns. Experiments on AR face database showed that the proposed method yields significant performance improvements compared to existing works for recognizing partially occluded and also non-occluded faces. Furthermore, the performance of the proposed approach is also assessed under illumination and extreme facial expression changes, demonstrating interesting results.

Facial expressions play a crucial role in the diagnosis of mental illnesses characterized by mood changes. The Facial Action Coding System (FACS) is a comprehensive framework that systematically categorizes and captures even subtle changes in facial appearance, enabling the examination of emotional expressions. In this study, we investigated the association between facial expressions and depressive symptoms in a sample of 59 older adults without cognitive impairment. Utilizing the FACS and the Korean version of the Beck Depression Inventory-II, we analyzed both "posed" and "spontaneous" facial expressions across six basic emotions: happiness, sadness, fear, anger, surprise, and disgust. Through principal component analysis, we summarized 17 action units across these emotion conditions. Subsequently, multiple regression analyses were performed to identify specific facial expression features that explain depressive symptoms. Our findings revealed several distinct features of posed and spontaneous facial expressions. Specifically, among older adults with higher depressive symptoms, a posed face exhibited a downward and inward pull at the corner of the mouth, indicative of sadness. In contrast, a spontaneous face displayed raised and narrowed inner brows, which was associated with more severe depressive symptoms in older adults. These findings suggest that facial expressions can provide valuable insights into assessing depressive symptoms in older adults.

Unconstrained authentication is an important component for personal automated systems and human-computer interfaces. Existing solutions mostly use face as the primary object of analysis. The performance of face-based systems is largely determined by the extent of deformation caused in the facial region and amount of useful information available in occluded face images. Periocular region is a useful portion of face with discriminative ability coupled with resistance to deformation. A reliable portion of periocular area is available for occluded images. The present work demonstrates that joint representation of periocular texture and periocular structure provides an effective expression and poses invariant representation. The proposed methodology provides an effective and compact description of periocular texture and shape. The method is tested over four benchmark datasets exhibiting varied acquisition conditions.