Online Multi-Face Tracking With Multi-Modality Cascaded Matching

Real-time face detection and recognition using hybrid-information extracted from face space and facial features

Personality computing lies on the effort of designing computational models to recognize personality traits from different sources of input such as visual (image), verbal (text) and vocal (audio). To recognize personality traits from facial images, the recognition model needs to extract facial features such as eyes, mouth, nose, etc. Facial features are relevant for personality judgements where they are providing information about human expression and behaviours. For example, by detecting eye’s movement, dimension and size, machines can describe people’s personality. Basically, people with higher score of conscientiousness have greater fluctuations in their pupil size while people who blink faster are more neurotic. The potential of each facial feature to automatically describe human personality has prompted research in personality recognition using CNN-based techniques. On the other hand, convolutional neural networks (CNN) have proven to be greatly success in the field of image processing including face recognition and detection. Since face detection is a key task in personality recognition using facial features, adoption of CNN model is a reliable choice. This study aimed to explore CNN-based models, which have been used and modified in the personality computing field. In this paper, we review several CNN-based models which have been employed for personality traits recognition, with an emphasis on extraction of facial features. Based on the finding, there are four models (VGGNet, ResNet, FaceNet and OpenFace) which are widely used and frequently adopted by previous studies in personality trait recognition. The adoption of CNN- based models in facial features extraction for personality traits recognition helps in achieving a good accuracy result. Thus, there is a huge potential to enhance and modify CNN-based model to generate more efficient model for facial features extraction tasks. Initially, this paper briefly explains personality computing and facial features extraction for personality recognition task. Next, this paper discusses the characteristics of network layer in CNN-based models for facial features extraction and personality recognition. Finally, this paper concludes that each of these models has its own features strength that are potentially useful for facial features extraction in development of personality traits recognition model.

Face recognition technology automatically identifies an individual from image or video sources. The detection process can be done by attaining facial characteristics from the image of a subject face. Recent developments in deep learning (DL) and computer vision (CV) techniques enable the design of automated face recognition and tracking methods. This study presents a novel Harris Hawks Optimization with deep learning-empowered automated face detection and tracking (HHODL-AFDT) method. The proposed HHODL-AFDT model involves a Faster region based convolution neural network (RCNN)-based face detection model and HHO-based hyperparameter optimization process. The presented optimal Faster RCNN model precisely recognizes the face and is passed into the face-tracking model using a regression network (REGN). The face tracking using the REGN model uses the features from neighboring frames and foresees the location of the target face in succeeding frames. The application of the HHO algorithm for optimal hyperparameter selection shows the novelty of the work. The experimental validation of the presented HHODL-AFDT algorithm is conducted using two datasets and the experiment outcomes highlighted the superior performance of the HHODL-AFDT model over current methodologies with maximum accuracy of 90.60% and 88.08% under PICS and VTB datasets, respectively.

Understanding how the human visual system recognizes objects is one of the key challenges in neuroscience. Inspired by a large body of physiological evidence, a general class of recognition models has emerged, which is based on a hierarchical organization of visual processing, with succeeding stages being sensitive to image features of increasing complexity. However, these models appear to be incompatible with some well-known psychophysical results. Prominent among these are experiments investigating recognition impairments caused by vertical inversion of images, especially those of faces. It has been reported that faces that differ 'featurally' are much easier to distinguish when inverted than those that differ 'configurally'; a finding that is difficult to reconcile with the physiological models. Here, we show that after controlling for subjects' expectations, there is no difference between 'featurally' and 'configurally' transformed faces in terms of inversion effect. This result reinforces the plausibility of simple hierarchical models of object representation and recognition in the cortex.

Face detection is an important step in face recognition. Ineffective algorithm used for face detection will have a negative impact on the performance of face recognition. Face detection technology is not only a key step in face recognition technology, but also is an independent widely-used technology. It is important to design a suitable feature for face detection technology. This paper proposes a new face feature, which can be used in face detection. The face feature has the property of rotation and scaling invariant under some certain conditions. The algorithm regards an image as an elastic surface and regards the grey level as deformation. The algorithm calculates local elastic potential vectors and uses them as the input to the adaboost classifier. The output of the classifier is the final result. Theoretical analysis and experiments demonstrate that the feature proposed in the paper can improve accuracy to some certain extent and is robust to rotation and scaling transformation.

We propose an effective facial features detection method for human– robot interaction (HRI) with indoor mobile robot. In case of human-robot interaction, its vision system has to manage such difficult problems as pose variations, illumination changes, and complex backgrounds, which is caused by the mobility of robot. In this paper, in order to overcome such difficult problems, we suggest a facial feature detection method based on local image area and direct pixel-intensity distributions, in which we propose two novel concepts; the directional template for evaluating intensity distributions and the edge-like blob map with multiple strength intensity. Using this proposed blob map, we show that the locations of major facial features – two eyes and a mouth – can be successfully estimated. Without the information of facial area boundary, final candidate face region is determined by both obtained locations of facial features and weighted correlations with stored facial templates. Our approach is also a flexible algorithm that can be applicable to both color and gray image. In case of color image, faster detection of both facial features and face is feasible by using the chromatic property of facial color. Experimental results from many color images and well-known gray level face database images prove the usefulness of proposed algorithm in human– robot interaction applications.

The problem of online face tracking from unconstrained videos is still unresolved. Challenges range from coping with severe online appearance variations to coping with occlusion. We propose RFTD (Robust Face Tracking-by-Detection), a system which combines tracking and detection into a single framework to robustly track a face from unconstrained videos. RFTD is based on the idea that adaptive and stable algorithmic components can complement each other in the task of online tracking. An online Structured Output SVM (SO-SVM) is combined with an offline trained face detector to break the self-learning loop typical in tracking. In turn, the face detector is supervised by a Deformable Part Model (DPM) landmark detector to asses the reliability of the face detection output. Extensive evaluation shows that RFTD delivers consistently good tracking performances across different scenarios, i.e., high mean success rate and lowest standard deviation across benchmark videos.

The complexity and applicability of facial recognition in the growing domains of security and authentication, payment, advertising and health care have accelerated the research and development of this niche in the artificial intelligence (AI) field. A facial recognition system utilizes technologies to recognize a human face through mapping of distinct facial features from raw data such as photography and video to information of known faces stored in the database. Recognizing faces in real-world images is a challenging task. Generally, a facial recognition system involves multiple processes such as facial detection, feature extraction, face matching and confidence scoring in which various techniques and algorithms are available in the form of libraries and frameworks. In this paper, an investigation is conducted using two pipelines to identify the effectiveness of two prevalent open-source libraries, OpenCV and OpenFace, on real-time face recognition system. In the OpenCV pipeline, Viola –Jones algorithm is chosen for face detection, while feature extraction is conducted using local binary pattern histogram (LBPH). Face detection is performed using dlib, while feature extraction is conducted using deep learning algorithm in the OpenFace pipeline. Performance measures of both pipelines are evaluated using Labelled Faces in the Wild (LFW) as a benchmark data set. In addition, a locally populated data set consisting of images of personnel who are given access to the office door system is used to experiment on the applicability of these pipelines on real streaming data. OpenFace achieved an AUROC of 0.98, while OpenCV scored 0.94. Both experiments inferred that facial recognition system with OpenFace pipeline outperforms OpenCV in all aspects of performance measures. Further development demonstrates the successful implementation of OpenFace pipeline to a functional prototype in replacing the manual door access control.

Real-time face detection and tracking for mobile videoconferencing

Nowadays, there is a growing interest in security applications using embedded smart cameras. Despite the rising attention to re-identifying the people in a non-overlapping embedded camera network, there exist no comparative evaluation of the existing schemes, specially facial representations. Though, facial features offer the advantage of remaining stable over much larger time intervals in contrast to commonly exploited features for person re-identification, such as whole body appearance. However, the challenge in using faces for such applications, apart from low captured face resolutions, is that their appearance is largely influenced by changes in illumination, pose, viewpoint, background and partial occlusions. Therefore, in this paper, we present a comparative study of facial local features for the task of person re-identification using embedded smart cameras in a realistic surveillance scenario. The experiments are performed on a novel data set which we make publicly available for future research in this area. The contributions of this work are as follows: 1) evaluation of eight baseline facial local features for person re-identification using embedded smart cameras; 2) an empirical study of the performance of person re-identification systems based on local facial features when fused with auxiliary information such as face shape, face skin color tone, hair color, and cloth color; 3) comparative study of single- and multi-shot approaches; 4) a novel data set using embedded cameras for person re-identification with properties unavailable in exiting data sets.

Face-based video retrieval (FBVR) is the task of retrieving videos that containing the same face shown in the query image. In this article, we present the first end-to-end FBVR pipeline that is able to operate on large datasets of unconstrained, multi-shot, multi-person videos. We adapt an existing audiovisual recognition dataset to the task of FBVR and use it to evaluate our proposed pipeline. We compare a number of deep learning models for shot detection, face detection, and face feature extraction as part of our pipeline on a validation dataset made of more than 4000 videos. We obtain 97.25% mean average precision on an independent test set, composed of more than 1000 videos. The pipeline is able to extract features from videos at sim 7 times the real-time speed, and it is able to perform a query on thousands of videos in less than 0.5 s.

From the past two decades, various spheres of activity are in the aspect of ‘Face recognition’ as an essential tool. The complete series of actions of face recognition is composed of 3 stages: Face Detection, Feature Extraction and Recognition. In this paper, we make an effort to put forth a new application of face recognition and detection in education. The proposed system scans the classroom and detects the face of the students in class and matches the scanned face with the templates that is available in the database and updates the attendance of the respective students.

This paper describes a trainable system capable of tracking faces and facial features like eyes and nostrils and estimating basic mouth features such as degrees of openness and smile in real time. In developing this system, we have addressed the twin issues of image representation and algorithms for learning. We have used the invariance properties of image representations based on Haar wavelets to robustly capture various facial features. Similarly, unlike previous approaches this system is entirely trained using examples and does not rely on a priori (hand-crafted) models of facial features based on an optical flow or facial musculature. The system works in several stages that begin with face detection, followed by localization of facial features and estimation of mouth parameters. Each of these stages is formulated as a problem in supervised learning from examples. We apply the new and robust technique of support vector machines (SVM) for classification in the stage of skin segmentation, face detection and eye detection. Estimation of mouth parameters is modeled as a regression from a sparse subset of coefficients (basis functions) of an overcomplete dictionary of Haar wavelets.

Face detection technology has garnered much interest due to its potential uses in industries like face identification and video surveillance systems. In addition to being a component of the automatic face recognition framework, real-time face detection is now becoming its study area. There are numerous methods to handle the problem of face detection. Traditional methods of manual attendance tracking in lecture halls are time-consuming, prone to errors, and lack efficiency. Next-generation attendance tracking systems are incorporating automated face detection technology to address these challenges. This paper explores the implementation of automated face detection for lecture attendance, leveraging computer vision and facial recognition algorithms. Initially, students’ data were gathered and preprocessed using a Median filter (MF). Then the preprocessed data features were extracted using a histogram of gradients (HOG). And finally, extracted features were classified using a Backpropagation Bayesian neural network (BBNN). To prove the efficiency of the suggested method, that is compared and contrasted with specific conventional methods. Experimental outcomes show that the recommended method performs superior when contrasted to traditional methods.

The purpose of this chapter is to describe the application of the problem solving ontology component of HCVM in the area of face detection and annotation. computer-based aspects of an embedded feature-based multi-agent face detection and annotation system are described in this chapter. Unlike similar feature-based systems that start from searching for facial organs in the images and group them to find faces, the system solves the problem using the problem solving agents of the HCVM in a top-down fashion. The embedded mtilti-agent face detection system makes hypotheses of face locations and seeks evidences to verify them. The five problem solving agents follow a coarse grain to fine grain methodology and work on various features of color images. The coarse to fine top-down methodology is more akin to human perceptions than a bottom-up approach. The higher level problem solving agents like decomposition work as background knowledge for the control and decision agents resulting in improvement of the accuracy and speed of detection. It is also distinct from many other top-down systems in that no re-scaling of the images is needed in the searching process. To improve the detection rate, a unique iterative region-partitioning algorithm is developed in this multi-agent system. The problem solving agents model aspects related to skin-tone region segmentation, noise reduction, candidate face regions location and facial feature extraction/face detection, and annotation. The detection algorithm is invariant to scale and rotation to some degree. The performance of the multi-agent system has been tested with a large number of images and the results obtained thus far are very encouraging.