Facial expression analysis using local directional stigma mean patterns and convolutional neural networks

Automated recognition of facial expression has attracted significant attention in recent years due to its potential applicability in security and surveillance, human computer interaction, social robotics, and animation. This paper presents a new facial expression recognition method that utilizes bit plane specific local image description in a weighted score level fusion. The motivation is to utilize bit plane slicing to highlight the contribution of a particular bit plane made to the holistic facial appearance, which is then used in a weighted score level fusion in order to boost the recognition performance. A new local image descriptor is proposed specifically to extract local features from bit plane representations that utilizes Fisher linear discriminant to maximize the inter-class distance, while minimizing the intra-class variance. Two well-known facial expression databases, namely the Cohn-Kanade (CK) and the Japanese female facial expression (JAFFE) database have been used to evaluate the performance of the proposed method against existing facial appearance descriptors, such as local binary pattern (LBP), local ternary pattern (LTP), local directional pattern (LDP), and linear discriminant analysis (LDA). Experiments with a total of seven prototypic facial expressions show promising results for the proposed method, as compared with the other existing methods.

In this paper, local directional position pattern (LDPP) and local ternary pattern (LTP) are selected for facial recognition method which are having many advantages over previous techniques like local binary pattern (LBP) and local directional pattern (LDP). The selected techniques of LDPP and LTP are estrangement in their algorithms which help solely to extract features out of an image. LDPP is a revised form of LDP. In a typical LDP, only the top edge direction was taken into consideration, but strength sign of the pixel was not considered which may result in same code for opposite kind of edge pixel. This snag is overcome by LDPP which is further concatenated with LTP for better feature extraction. Once features are extracted they are trained using deep belief network. In the experimental work 10 images of each expression i.e. angry, surprise, disgust, neutral, sad, smile are selected. LDPP and LTP are concatenated followed by principal component analysis (PCA) and general discriminant analysis (GDA). Further for training, Deep Belief Network (DBN) is used which eventually increases the recognition rate and achieve accuracy of 95.3% which was 89.3% without concatenating.

This paper proposes a new pattern-based feature called local mesh ternary pattern for biomedical image indexing and retrieval. The standard local binary patterns (LBP) and local ternary patterns (LTP) encode the greyscale relationship between the centre pixel and its surrounding neighbours in two-dimensional (2D) local region of an image, whereas the proposed method encodes the greyscale relationship among the neighbours for a given centre pixel with three selected directions of mesh patterns which is generated from 2D image. The novelty of the proposed method is that it uses ternary patterns from mesh patterns of an image to encode more spatial structure information which leads to better retrieval. The experiments have been carried out for proving the worth of proposed algorithm on three different types of benchmarked biomedical databases; (i) computed tomography (CT) scanned lung image databases named as LIDC-IDRI-CT and VIA/I–ELCAP-CT, (ii) brain magnetic resonance imaging (MRI) database named as OASIS-MRI. The results demonstrate that the proposed method yields better performance in terms of average retrieval precision and average retrieval rate over state-of-the-art feature extraction techniques like LBP, LTP, local mesh pattern, etc.

Extraction of human emotions from facial expression has attracted significant attention in computer vision community. There are several appearance based techniques like local binary patterns (LBP), local directional patterns (LDP), local ternary patterns (LTP) and gradient local ternary patterns (GLTP). Recently, many investigations have been done to improve these feature extraction techniques. Although GLTP has achieved an improvement in robustness to noise and illumination, it encodes image gradient in four directions and two orientations only. This paper proposes to improve GLTP to directional gradient local ternary patterns (DGLTP) by encoding image gradient on eight directions and four orientations. The eight directional Kirsch mask is used to encode the image gradient followed by dimensionality reduction using linear discriminant analysis (LDA) and AVG, MAX and MIN pooling techniques are compared for fusing facial expression features. The proposed technique was experimented on JAFFE facial expression dataset with support vector machine (SVM). The experimental results show that proposed technique improved accuracy of GLTP.

Recently, there has been a huge demand for assistive technology for industrial, commercial, automobile, and societal applications. In some of these applications, there is a requirement of an efficient and accurate system for automatic facial expression recognition (FER). Therefore, FER has gained enormous interest among computer vision researchers. Although there has been a plethora of work available in the literature, an automatic FER system has not yet reached the desired level of robustness and performance. In most of these works, there has been the dominance of appearance-based methods primarily consisting of local binary pattern (LBP), local directional pattern (LDP), local ternary pattern (LTP), gradient local ternary pattern (GLTP), and improved local ternary pattern (IGLTP). Keeping in view the popularity of appearance-based methods, in this paper, we have proposed an appearance-based descriptor called Improved Adaptive Local Ternary Pattern (IALTP) for automatic FER. This new descriptor is an improved version of ALTP, which has been proved to be effective in face recognition. We have investigated ALTP in more details and have proposed some improvements like the use of uniform patterns and dimensionality reduction via principal component analysis (PCA). The reduced features are then classified using kernel extreme learning machine (K-ELM) classifier. In order to validate the performance of the proposed method, experiments have been conducted on three different FER datasets using well-known evaluation measures such as accuracy, precision, recall, and F1-Score. The proposed approach has also been compared with some of the state-of-the-art works in literature and found to be more accurate and efficient.

Texture is the surface property that is used to identify and recognise objects. This property is widely used in many applications including texture-based face recognition systems, surveillance, identity verification and so on. The Local binary pattern (LBP) texture method is most successful for face recognition. Owing to the great success of LBP, recently many models, which are variants of LBP have been proposed for texture analysis. Some of the derivatives of LBPs are multivariate local binary pattern, centre symmetric local binary pattern, local binary pattern variance, dominant local binary pattern, advanced local binary pattern, local texture pattern (LTP) and local derivative pattern (LDP). In this scenario, it is essential to review, whether LBP or their derivatives perform better for face recognition. The real-time challenges such as illumination changes, rotations, angle variations and facial expression variations are evaluated by different LBP-based models. Experiments were conducted on the Japanese female facial expression, YALE and FRGC version2 databases. The results show that LDP and LTP perform much better than the other LBP-based models.

Children’s emotions can affect the learning process, especially positive emotions, making them more focused on learning. In addition, in terms of identifying someone’s emotions, we can represent them through facial expressions by combining the local binary pattern (LBP) and the local ternary patterns (LTP) method, known as Co-ChiLeRFE. The reasons for combining the two methods are that the LBP has proven to be very good at performing feature extraction, especially in describing textures. At the same time, LTP is adept at dealing with uniform motifs such as those on the face area. Subsequently, in this study, we used the NIMH child emotional faces picture set (NIMH-ChEFS), which has five-class expressions: sad, neutral, happy, angry, and afraid. To achieve optimal results in the Co-ChiLeRFE method, we set the LBP parameter as P = 8, R = 8, and the LTP parameter threshold value of one. The results we got from this experiment achieved a system performance superior accuracy of 92.51%.

Handcrafted vs. non-handcrafted features for computer vision classification

Local line directional pattern for palmprint recognition

Local binary pattern (LBP) is sensitive to noise. Local ternary pattern (LTP) partially solves this problem. Both LBP and LTP, however, treat the corrupted image patterns as they are. In view of this, we propose a noise-resistant LBP (NRLBP) to preserve the image local structures in presence of noise. The small pixel difference is vulnerable to noise. Thus, we encode it as an uncertain state first, and then determine its value based on the other bits of the LBP code. It is widely accepted that most of the image local structures are represented by uniform codes and noise patterns most likely fall into the non-uniform codes. Therefore, we assign the value of an uncertain bit hence as to form possible uniform codes. Thus, we develop an error-correction mechanism to recover the distorted image patterns. In addition, we find that some image patterns such as lines are not captured in uniform codes. Those line patterns may appear less frequently than uniform codes, but they represent a set of important local primitives for pattern recognition. Thus, we propose an extended noise-resistant LBP (ENRLBP) to capture line patterns. The proposed NRLBP and ENRLBP are more resistant to noise compared with LBP, LTP, and many other variants. On various applications, the proposed NRLBP and ENRLBP demonstrate superior performance to LBP/LTP variants.

In this paper, we describe a novel local descriptor of image texture representation for visual recognition. The image features based on micro-descriptors such as local binary patterns (LBP) and local ternary patterns (LTP) have been very successful in a number of applications including face recognition, object detection, and texture analysis. Instead of binary quantization in LBP, LTP thresholds the differential values between a focused pixel and its neighborhood pixels into three gray levels, which can be explained as the active status (i.e., positively activated, negatively activated, and not activated) of the neighborhood pixels compared to the focused pixel. However, regardless of the magnitude of the focused pixel, the thresholding strategy remains fixed, which would violate the principle of human perception. Therefore, in this study, we design LTP with a data-driven threshold according to Weber’s law, a human perception principle; further, our approach incorporates the contexts of spatial and orientation co-occurrences (i.e., co-occurrence context) among adjacent Weber-based local ternary patterns (WLTPs, i.e., data-driven quantized LTPs) for texture representation. The explored WLTP is formulated by adaptively quantizing differential values between neighborhood pixels and the focused pixel as negative or positive stimuli if the normalized differential values are large; otherwise, the stimulus is set to 0. Our approach here is based on the fact that human perception of a distinguished pattern depends not only on the absolute intensity of the stimulus but also on the relative variance of the stimulus. By integrating co-occurrence context information, we further propose a rotation invariant co-occurrence WLTP (RICWLTP) approach to be more discriminant for image representation. In order to validate the efficiency of our proposed strategy, we apply this to three different visual recognition applications including two texture datasets and one food image dataset and prove the promising performance that can be achieved compared with the state-of-the-art approaches.

The recognition rate of person-independent facial expression is generally not high, which limits the practical application of facial expression recognition. Aiming at this problem, this paper analyzes the reasons for the low recognition rate of person-independent facial expression, and proposes a recognition algorithm of person-independent facial expression based on improved LBP (Local Binary Pattern) and HOSVD (Higher-Order Singular Value Decomposition). The algorithm has the following contributions of facial expression recognition framework. In the stage of facial expression feature extraction, the transient features extracted by LDP(Local Directional Pattern) and the persistent features extracted by CBP(Centralized Binary Pattern) are integrated to improve the discrimination of facial expression features. Moreover, in the stage of facial expression classification and recognition, the traditional nearest neighbor classification is changed into k-nearest neighbor pre-classification, and the regional energy calculated by HOSVD is used to determine the similarity of two images for secondary classification. Finally, in the extended Cohn-Kanade dataset and Oulu-CASIA NIR&VIS facial expression database, the theoretical analysis and experimental results show that the method has better recognition effect for solving the problem of person-independent facial expression recognition.

Although face recognition systems have wide application, they are vulnerable to presentation attack samples (fake samples). Therefore, a presentation attack detection (PAD) method is required to enhance the security level of face recognition systems. Most of the previously proposed PAD methods for face recognition systems have focused on using handcrafted image features, which are designed by expert knowledge of designers, such as Gabor filter, local binary pattern (LBP), local ternary pattern (LTP), and histogram of oriented gradients (HOG). As a result, the extracted features reflect limited aspects of the problem, yielding a detection accuracy that is low and varies with the characteristics of presentation attack face images. The deep learning method has been developed in the computer vision research community, which is proven to be suitable for automatically training a feature extractor that can be used to enhance the ability of handcrafted features. To overcome the limitations of previously proposed PAD methods, we propose a new PAD method that uses a combination of deep and handcrafted features extracted from the images by visible-light camera sensor. Our proposed method uses the convolutional neural network (CNN) method to extract deep image features and the multi-level local binary pattern (MLBP) method to extract skin detail features from face images to discriminate the real and presentation attack face images. By combining the two types of image features, we form a new type of image features, called hybrid features, which has stronger discrimination ability than single image features. Finally, we use the support vector machine (SVM) method to classify the image features into real or presentation attack class. Our experimental results indicate that our proposed method outperforms previous PAD methods by yielding the smallest error rates on the same image databases.

Biometrics is the study of methods of recognizing humans based on their behavioral and physical characteristics or traits. Face recognition is one of the biometric modalities that received a great amount of attention from many researchers during the past few decades because of its potential applications in a variety of security domains. Face recognition however is not only concerned with recognizing human faces, but also with recognizing faces of non-biological entities or avatars. Fortunately, the need for secure and affordable virtual worlds is attracting the attention of many researchers who seek to find fast, automatic and reliable ways to identify virtual worlds’ avatars. In this work, I propose new techniques for recognizing avatar faces, which also can be applied to recognize human faces. Proposed methods are based mainly on a well-known and efficient local texture descriptor, Local Binary Pattern (LBP). I am applying different versions of LBP such as: Hierarchical Multi-scale Local Binary Patterns and Adaptive Local Binary Pattern with Directional Statistical Features in the wavelet space and discuss the effect of this application on the performance of each LBP version. In addition, I use a new version of LBP called Local Difference Pattern (LDP) with other well-known descriptors and classifiers to differentiate between human and avatar face images. The original LBP achieves high recognition rate if the tested images are pure but its performance gets worse if these images are corrupted by noise. To deal with this problem I propose a new definition to the original LBP in which the LBP descriptor will not threshold all the neighborhood pixel based on the central pixel value. A weight for each pixel in the neighborhood will be computed, a new value for each pixel will be calculated and then using simple statistical operations will be used to compute the new threshold, which will change automatically, based on the pixel’s values. This threshold can be applied with the original LBP or any other version of LBP and can be extended to work with Local Ternary Pattern (LTP) or any version of LTP to produce different versions of LTP for recognizing noisy avatar and human faces images.

Weed identification techniques that are effective can lower the cost of weed control while also improving crop quality and output. Herbicides are currently the most frequent method of weed management. In order to deal with the issues of weed man-agement in agriculture, accurate crop and weed discrimination is essential. This work deals with the performance comparison of different classification algorithms namely Convolutional Neural Network(CNN), Graph Convolutional Networks(GCN), k-Nearest Neighbor(k-NN), and Support Vector Machine(SVM). Feature extraction algorithms, Local Directional Pattern(LDP) and Local Directional Relation Pattern(LDRP) are used alongside SVM and kNN. For data expansion, Generative Adversarial Network(GAN) was used, which generates synthetic samples of images from the dataset. The result of the study shows that CNN gives the maximum accuracy of 90.58% in ‘Crop and weed detection’ dataset when compared with other classification models used for investigation in our paper.