Local Binary Patterns and Its Application to Facial Image Analysis: A Survey

Facial images are of critical importance in many real-world applications from gaming to surveillance. The current literature on facial image analysis, from face detection to face and facial expression recognition, are mainly performed in either RGB, Depth (D), or both of these modalities. But, such analyzes have rarely included Thermal (T) modality. This paper paves the way for performing such facial analyzes using synchronized RGB-D-T facial images by introducing a database of 51 persons including facial images of different rotations, illuminations, and expressions. Furthermore, a face recognition algorithm has been developed to use these images. The experimental results show that face recognition using such three modalities provides better results compared to face recognition in any of such modalities in most of the cases.

Face recognition is an area of computer vision that has attracted a lot of interest from the research community. A growing demand for robust face recognition software in security applications has driven the development of interesting approaches in this field. A large quantity of research in face recognition deals with visible face images. In the visible spectrum the illumination and face expressions changes represent a significant challenge for the recognition system. To avoid these problems, researchers proposed recently the use of 3D and infrared imaging for face recognition. In this work, we introduce a new framework for infrared face recognition using texture descriptors. This framework exploits linear and non linear dimensionality reduction techniques for face learning and recognition in the texture space. Active and passive infrared imaging modalities are used and comparison with visible face recognition is performed. Two multispectral face recognition databases were used in our experiments: Equinox Database (Visible, SWIR, MWIR, LWIR) and Laval University Multispectral Database (Visible, NIR, MWIR, LWIR). The obtained results show high increase in recognition performance when texture descriptors like LBP (Local Binary Pattern) and LTP (Local Ternary Pattern) are used. The best result was obtained in the short wave infrared spectrum (SWIR) using non linear dimensionality reduction techniques.

The field of medical image analysis has greatly influenced the practice of neuroscience. Many studies aim to find new disease related anatomical characteristics based on the analysis of Magnetic Resonance images (MRI). Image segmentation is usually a fundamental step, in which brain structures of interest are delineated from MRI. Neuroscientists then categorize segmentations of anatomical structures by attributes such as shape, size, or location. Besides medical images, biometric images such as facial images are gaining more and more attention from clinical research because certain biometric features are also related to neurological diseases. Similar to brain morphology analysis, facial features are collected for studies testing the hypothesis that the disease alters the facial shape. Despite the rapid development of image analysis technologies, most clinical practice still relies on laborious manual. The aim of this thesis is to develop a shape modeling framework for brain and facial image analysis, for the purpose of clinical practice. Specifically, it focuses on the following three problems: (1) brain structure segmentation from MRI, where several deformable model based segmentation methods are proposed; (2) quantitative analysis of brain morphology, where methods for directly comparing 3D shapes of brain structures are proposed, and clinical applications are demonstrated; (3) facial morphology analysis, where an automatic facial feature localization method is proposed, and applied to the shape analysis of the eyes.

Understanding how the human visual system recognizes objects is one of the key challenges in neuroscience. Inspired by a large body of physiological evidence, a general class of recognition models has emerged, which is based on a hierarchical organization of visual processing, with succeeding stages being sensitive to image features of increasing complexity. However, these models appear to be incompatible with some well-known psychophysical results. Prominent among these are experiments investigating recognition impairments caused by vertical inversion of images, especially those of faces. It has been reported that faces that differ 'featurally' are much easier to distinguish when inverted than those that differ 'configurally'; a finding that is difficult to reconcile with the physiological models. Here, we show that after controlling for subjects' expectations, there is no difference between 'featurally' and 'configurally' transformed faces in terms of inversion effect. This result reinforces the plausibility of simple hierarchical models of object representation and recognition in the cortex.

The texture is an essential property of an image that tells about the arrangements of pixels of different intensities in an image matrix. Researchers have used many descriptors for texture classification of the images in computer vision and image processing. Local Binary Pattern (LBP) is one of the descriptors, which is computationally efficient and straightforward for texture classification. In recent years, researchers have made many enhancements to the basic LBP method to improve the quality of extracted features for an image. This paper presents a Local Roughness Binary Pattern (LRBP) descriptor for feature extraction of an image to increase LBP discriminative capability. This new descriptor extracts features from the local region of an image called a partial feature vector and concatenates these local features to obtain a feature vector of an image. This feature vector is used as input to a classifier to classify the class of an image. We tested the proposed descriptor for the images available on CUReT and KTHTIPS2b databases with Support Vector Machine (SVM) and k-nearest Neighbors (KNN) classifiers. We have presented the experimental results obtained from the analysis process regarding classification accuracy and confusion matrix. Finally, we have presented a performance comparison between the proposed method, basic LBP descriptor, and its variants. Results show that the proposed descriptor gives much better results than other descriptors on the KTHTIPS2b Database when we use the KNN classifier.KeywordsLBPLocal binary patternLRBPLocal roughness binary patternSVMSupport vector machineKNN K-nearest neighbors

Based on neural network and local binary pattern algorithm, this paper builds a lightweight artificial face recognition system on chip Firefly-RK3399, with high speed, strong robustness and high recognition accuracy. Our embedded artificial intelligent face recognition system mainly consists of face detection, feature extraction and recognition. Multi-task convolutional neural network (MTCNN) under the CaffeOnACL framework is utilized for face detection, and the local binary pattern (LBP) is applied as face recognition algorithm. Experiments illustrate that our artificial intelligent embedded face recognition system has high speed and accuracy, which is easy-carrying and of high commercial value as well.

This paper proposes a novel face representation approach, local Gabor binary mapping pattern (LGBMP), for multi-view gender classification. In this approach, a face image is first represented as a series of Gabor magnitude pictures (GMP) by applying multi-scale and multi-orientation Gabor filters. Each GMP is then encoded as a LGBP image where a uniform local binary pattern (LBP) operator is used. After that, each LGBP image is divided into non-overlapping rectangular regions, from which spatial histograms are extracted. Although an LGBP feature vector can be obtained by fitting together the regional histograms, it can not be employed in pattern classification due to its high dimension. We propose that each regional LGBP feature be mapped onto a one-dimensional subspace independently before they are concatenated as a whole feature vector. This is attractive since we reduce the feature dimension and also preserve the spatial information of LGBP image. Two ways have been proposed to map the regional LGBP feature in this paper. One is so-called LGBMP-LDA using linear discriminant analysis (LDA) for dimensionality reduction while the other is to project the regional LGBP feature onto the class center connecting line, namely, LGBMP-CCL. As a result, despite several decades of Gabor filters, the final feature dimension is even less than that of the feature extracted by using LBP directly on gray-scale images. The classification tasks in our work are performed by support vector machines (SVM). The experimental results on the CAS-PEAL face database indicate that the proposed approach achieves higher accuracy than the others such as SVMs+Gray-scale pixel, SVMs+Gabor and SVMs+LBP approach, more particularly, it has the lowest dimension of feature vector.

Visible face recognition systems, being vulnerable to illumination, expression, and pose, can not achieve robust performance in unconstrained situations. Meanwhile, near infrared face images, being light- independent, can avoid or limit the drawbacks of face recognition in visible light, but its main challenges are low resolution and signal noise ratio (SNR). Therefore, near infrared and visible fusion face recognition has become an important direction in the field of unconstrained face recognition research. In this paper, a novel fusion algorithm in non-subsampled contourlet transform (NSCT) domain is proposed for Infrared and visible face fusion recognition. Firstly, NSCT is used respectively to process the infrared and visible face images, which exploits the image information at multiple scales, orientations, and frequency bands. Then, to exploit the effective discriminant feature and balance the power of high-low frequency band of NSCT coefficients, the local Gabor binary pattern (LGBP) and Local Binary Pattern (LBP) are applied respectively in different frequency parts to obtain the robust representation of infrared and visible face images. Finally, the score-level fusion is used to fuse the all the features for final classification. The visible and near infrared face recognition is tested on HITSZ Lab2 visible and near infrared face database. Experiments results show that the proposed method extracts the complementary features of near-infrared and visible-light images and improves the robustness of unconstrained face recognition.

Face recognition is efficiently used in computer vision, pattern recognition, biometric, law enforcement, surveillance, criminal investigations, and missing person detection. However, face recognition is still a challenging task under variant illuminations, pose, expression, occlusion, and background. In this paper, we proposed a cascading of histogram of oriented gradients (HOG) and local binary pattern (LBP) feature extraction method on ORL face dataset, which can improve recognition rates while also addressing pose, scale, expression, and variant illumination issues. The facial recognition network was trained using various classifiers such as with KNN, SVM, and random forest (RF). The results have been shown that the accuracy rate of the combined LBP and HOG feature extraction method is better than individual LBP or HOG features for ORL database face recognition.

Facial recognition has been used in several areas such as security, detection of criminals or suspects, etc. The face is considered the most important part of the human body in the person identification process since it can be considered the most important perspective in human identification. This paper proposed a real-time monitoring security system for high secured areas rather than relying on human security officers. The proposed system engages with capturing and detecting the human face using the real-time image segment captured by the web camera through the Viola-Jones algorithm. After detecting the face in the captured image, face identification can be made through the LBP by finding the similarities. Once the identification is made, automatically it recognizes the face. Open Computer Vision (OpenCV) is one of the most efficient machine learning libraries used for Object Detection. OpenCV library is used for detection, and Haar - Cascades are trained to detect certain objects. LBP is used for the face recognition process.

Face recognition describes a surface framework, which is capable of processing image and detection. The proposed paper demonstrates three contributions: the first is to introduce the image representation, known as an integral image, the second application of Ada Boost learning algorithm, and the third is the cascaded framework. This includes observation, bio-metrics and video coding. Here, the primary objective is to implement a real-time system using a field-programmable gate array (FPGA) to track and detect human expression. The expression recognition involves colour-shaped coating separation and image purifying. Moreover, it involves different types of search engines which are based on computer vision. A computer controls Pan Tilt Zoom (PTZ) cameras in CCTV for surveillance systems for the face. The multi-view face tracking on a mobile platform has three methods (local binary patterns) and enhancing procedure. These face detectors are adaptable, extended for multi-opinion face recognition over the revolution ability and face matrix partition scheme is proposed to accelerate the face tracking process.

Digital games are played by a large variety of players with different profiles and backgrounds. Understanding their reactions and experiences while playing is crucial to support game expansion or creation. This paper presents a tool to support affective states assessment based on facial analysis. It allows the configuration of different testing scenarios to gather in-game information as well as webcam feed for the facial expressions analysis and further affective reaction processing. Also, it outputs all the data collected and processed into three different formats: graphs, videos and CSV files. Experimental tests with a 2D puzzle game were made to demonstrate that the tool is able to gather and properly provide information about players’ affective experience.

Face Recognition is the most popular and trending technology in the present era. It is an effective way to provide vision to a machine for better interaction with humans. The way of living will be reflected if machines can read our faces. The face recognition system will move the world in a new dimension. It will be beneficial in many ways to find the identity and security. In this paper, a face recognition system is proposed for advanced applications such as access and security, payments, criminal identifications etc. The process of identification will be based on face recognition which is further divided into three steps: detection of face, extractions of the features and classification, and real time recognition. Detection of face is recognized as the essential step of our system. It is used to extract a face in a frame, which is based on the Viola-Jones object detection algorithm that uses AdaBoost classifier with Haar and LBP features. Local Binary Patterns (LBP) is utilized to extract the unique features of the face like eyes, nose, and mouth in the feature extraction phase. The facial image is correlated with the images available in the database for the classification. The system is implemented in Python using OpenCV library. Kivy is used to create a user interface and also to build executables for different platforms.

A comparative study on local binary pattern (LBP) based face recognition: LBP histogram versus LBP image

In this paper we present a new static descriptor for facial image analysis. We combine Gaussian derivatives with Local Binary Patterns to provide a robust and powerful descriptor especially suited to extracting texture from facial images. Gaussian features in the form of image derivatives form the input to the Linear Binary Pattern(LBP) operator instead of the original image. The proposed descriptor is tested for face recognition and smile detection. For face recognition we use the CMU-PIE and the YaleB+extended YaleB database. Smile detection is performed on the benchmark GENKI 4k database. With minimal machine learning our descriptor outperforms the state of the art at smile detection and compares favourably with the state of the art at face recognition.