Facial Expression Recognition Using Directional Gradient Local Ternary Patterns

Recently, there has been a huge demand for assistive technology for industrial, commercial, automobile, and societal applications. In some of these applications, there is a requirement of an efficient and accurate system for automatic facial expression recognition (FER). Therefore, FER has gained enormous interest among computer vision researchers. Although there has been a plethora of work available in the literature, an automatic FER system has not yet reached the desired level of robustness and performance. In most of these works, there has been the dominance of appearance-based methods primarily consisting of local binary pattern (LBP), local directional pattern (LDP), local ternary pattern (LTP), gradient local ternary pattern (GLTP), and improved local ternary pattern (IGLTP). Keeping in view the popularity of appearance-based methods, in this paper, we have proposed an appearance-based descriptor called Improved Adaptive Local Ternary Pattern (IALTP) for automatic FER. This new descriptor is an improved version of ALTP, which has been proved to be effective in face recognition. We have investigated ALTP in more details and have proposed some improvements like the use of uniform patterns and dimensionality reduction via principal component analysis (PCA). The reduced features are then classified using kernel extreme learning machine (K-ELM) classifier. In order to validate the performance of the proposed method, experiments have been conducted on three different FER datasets using well-known evaluation measures such as accuracy, precision, recall, and F1-Score. The proposed approach has also been compared with some of the state-of-the-art works in literature and found to be more accurate and efficient.

In this paper, local directional position pattern (LDPP) and local ternary pattern (LTP) are selected for facial recognition method which are having many advantages over previous techniques like local binary pattern (LBP) and local directional pattern (LDP). The selected techniques of LDPP and LTP are estrangement in their algorithms which help solely to extract features out of an image. LDPP is a revised form of LDP. In a typical LDP, only the top edge direction was taken into consideration, but strength sign of the pixel was not considered which may result in same code for opposite kind of edge pixel. This snag is overcome by LDPP which is further concatenated with LTP for better feature extraction. Once features are extracted they are trained using deep belief network. In the experimental work 10 images of each expression i.e. angry, surprise, disgust, neutral, sad, smile are selected. LDPP and LTP are concatenated followed by principal component analysis (PCA) and general discriminant analysis (GDA). Further for training, Deep Belief Network (DBN) is used which eventually increases the recognition rate and achieve accuracy of 95.3% which was 89.3% without concatenating.

The recognition rate of person-independent facial expression is generally not high, which limits the practical application of facial expression recognition. Aiming at this problem, this paper analyzes the reasons for the low recognition rate of person-independent facial expression, and proposes a recognition algorithm of person-independent facial expression based on improved LBP (Local Binary Pattern) and HOSVD (Higher-Order Singular Value Decomposition). The algorithm has the following contributions of facial expression recognition framework. In the stage of facial expression feature extraction, the transient features extracted by LDP(Local Directional Pattern) and the persistent features extracted by CBP(Centralized Binary Pattern) are integrated to improve the discrimination of facial expression features. Moreover, in the stage of facial expression classification and recognition, the traditional nearest neighbor classification is changed into k-nearest neighbor pre-classification, and the regional energy calculated by HOSVD is used to determine the similarity of two images for secondary classification. Finally, in the extended Cohn-Kanade dataset and Oulu-CASIA NIR&VIS facial expression database, the theoretical analysis and experimental results show that the method has better recognition effect for solving the problem of person-independent facial expression recognition.

Automated recognition of facial expression has attracted significant attention in recent years due to its potential applicability in security and surveillance, human computer interaction, social robotics, and animation. This paper presents a new facial expression recognition method that utilizes bit plane specific local image description in a weighted score level fusion. The motivation is to utilize bit plane slicing to highlight the contribution of a particular bit plane made to the holistic facial appearance, which is then used in a weighted score level fusion in order to boost the recognition performance. A new local image descriptor is proposed specifically to extract local features from bit plane representations that utilizes Fisher linear discriminant to maximize the inter-class distance, while minimizing the intra-class variance. Two well-known facial expression databases, namely the Cohn-Kanade (CK) and the Japanese female facial expression (JAFFE) database have been used to evaluate the performance of the proposed method against existing facial appearance descriptors, such as local binary pattern (LBP), local ternary pattern (LTP), local directional pattern (LDP), and linear discriminant analysis (LDA). Experiments with a total of seven prototypic facial expressions show promising results for the proposed method, as compared with the other existing methods.

This paper represents automatic facial expression analysis method named Local Directional Stigma Mean Patterns (LDSMP) for automatic facial expression analysis and image retrieval using content based facial expression image retrieval and CNN. The traditional local patterns such as Local Binary Patterns (LBP) and Local Ternary Patterns (LTP) are applied for face recognition and expression analysis, calculated using relationship between the center pixel and neighboring pixels. The proposed method calculates the eight directional difference values then divided into the three ranges based on threshold values. Thus, the values are substituted with basic three positive values (+3, +2, +1) and three negative values (-3, -2, -1) to get more sensitive information from an image rather than aforementioned methods. The threshold can be select either static which is selected by user or dynamic is evaluated from image itself and supports to improve the efficiency. The performance of the proposed method is further improved by giving this patterns as input to the Convolutional Neural Networks (CNN) and compared with the existing methods LBP, LTP, and Directional Binary Code (DBC) in terms of Average Precision (AP), Average Recall (AR), and Average Retrieval Rate (ARR) using standard databases COREL 10K (DB1) and JAFFE (The Japanese Female Facial Expression) (DB2) and Extended Cohn-Kanade (CK +) (DB3) dataset.

Local line directional pattern for palmprint recognition

Face recognition and facial expression recognition using local patterns is the order of the day. Local directional number pattern (LDNP) is one of the prominent descriptor for face recognition. LDNP assigns a 3 bit code for each pixel in the image. The resultant LDNP labeled image is divided into regions to form histogram based descriptor. The histogram bins of all the regions are concatenated to form the final descriptor. In contrast to LDNP, a dimensionality reduced local directional number pattern (DR-LDNP) is proposed in this paper. The proposed descriptor computes single code for each block. This is done by X-ORing of the LDNP codes obtained in a single block. During the process, restructuring of the patterns is done by slightly modifying the LDNP coding pattern constraints. The resultant DR-LDNP descriptor outperforms the existing methods. The experimentation is carried out on standard databases like FERET, YALE, ORL, Cohn–Kannade and JAFFEE and obtained good recognition rates compared to other methods.

Facial expression recognition is an important part of the study in man-machine interaction. Principal component analysis (PCA) is an extraction method based on statistical features which were extracted from the global grayscale features of the whole image .But the grayscale global features are environmentally sensitive. In order to recognize facial expression accurately, a fused method of principal component analysis and local direction pattern (LDP) is introduced in this paper. First, PCA extracts the global features of the whole grayscale image; LDP extracts the local grayscale texture features of the mouth and eyes region, which contribute most to facial expression recognition, to complement the global grayscale features of PCA. Then we adopt Support Vector Machine (SVM) classifier for expression classification. Experimental results demonstrate that this method can classify different expressions more effectively and get higher recognition rate compared with the traditional method.

Children’s emotions can affect the learning process, especially positive emotions, making them more focused on learning. In addition, in terms of identifying someone’s emotions, we can represent them through facial expressions by combining the local binary pattern (LBP) and the local ternary patterns (LTP) method, known as Co-ChiLeRFE. The reasons for combining the two methods are that the LBP has proven to be very good at performing feature extraction, especially in describing textures. At the same time, LTP is adept at dealing with uniform motifs such as those on the face area. Subsequently, in this study, we used the NIMH child emotional faces picture set (NIMH-ChEFS), which has five-class expressions: sad, neutral, happy, angry, and afraid. To achieve optimal results in the Co-ChiLeRFE method, we set the LBP parameter as P = 8, R = 8, and the LTP parameter threshold value of one. The results we got from this experiment achieved a system performance superior accuracy of 92.51%.

This study focuses on a comprehensive study of Automatic Signature Verification (ASV) for off-line Thai signatures; an investigation was carried out to characterise the challenges in Thai ASV and to baseline the performance of Thai ASV employing baseline features, being Local Binary Pattern, Local Directional Pattern, Local Binary and Directional Patterns combined (LBDP), and the baseline shape/feature-based hidden Markov model. As there was no publicly available Thai signature database found in the literature, the authors have developed and proposed a database considering real-world signatures from Thailand. The authors have also identified their latent challenges and characterised Thai signature-based ASV. The database consists of 5,400 signatures from 100 signers. Thai signatures could be bi-script in nature, considering the fact that a single signature can contain only Thai or Roman characters or contain both Roman and Thai, which poses an interesting challenge for script-independent SV. Therefore, along with the baseline experiments, the investigation on the influence and nature of bi-script ASV was also conducted. From the equal error rates and Bhattacharyya distance, the score achieved in the experiments indicate that the Thai SV scenario is a script-independent problem. The open research area on this subject of research has also been addressed.

Weed identification techniques that are effective can lower the cost of weed control while also improving crop quality and output. Herbicides are currently the most frequent method of weed management. In order to deal with the issues of weed man-agement in agriculture, accurate crop and weed discrimination is essential. This work deals with the performance comparison of different classification algorithms namely Convolutional Neural Network(CNN), Graph Convolutional Networks(GCN), k-Nearest Neighbor(k-NN), and Support Vector Machine(SVM). Feature extraction algorithms, Local Directional Pattern(LDP) and Local Directional Relation Pattern(LDRP) are used alongside SVM and kNN. For data expansion, Generative Adversarial Network(GAN) was used, which generates synthetic samples of images from the dataset. The result of the study shows that CNN gives the maximum accuracy of 90.58% in ‘Crop and weed detection’ dataset when compared with other classification models used for investigation in our paper.

The most expressive manner which human can reveal his emotional states is facial expression. Automatic facial expression recognition is an emerging field of study having extensive applications among which the human-computer interaction (HCI) has received lots of attentions in recent years. The features extracted from facial images, in order to recognize facial expressions, play an essential role in effectiveness of the facial image descriptors. Local binary pattern (LBP) texture descriptors have been known as simple, yet efficient descriptors which are noticeably used for extracting facial patterns from images. Recently, a generalized form of local binary pattern has been introduced which can offer a more precise image description than simple LBP descriptors. Consequently, it would be expected that taking the advantage of using these new LBP texture descriptors will produce more promising results in comparison with use of simple local binary pattern descriptors. In this paper, a novel method has been proposed for image feature extraction using these new image texture descriptors (generalized LBP); then, the obtained results have been compared to the results produced when applying simple LBP descriptors. Furthermore, K-NN and SVM have been used as classifiers in the proposed approach. Finally, a comparison between the proposed method and the existing local binary pattern algorithms for facial expression recognition concludes the superiority of the proposed algorithm over its existing counterparts.

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

In this paper, we describe a novel local descriptor of image texture representation for visual recognition. The image features based on micro-descriptors such as local binary patterns (LBP) and local ternary patterns (LTP) have been very successful in a number of applications including face recognition, object detection, and texture analysis. Instead of binary quantization in LBP, LTP thresholds the differential values between a focused pixel and its neighborhood pixels into three gray levels, which can be explained as the active status (i.e., positively activated, negatively activated, and not activated) of the neighborhood pixels compared to the focused pixel. However, regardless of the magnitude of the focused pixel, the thresholding strategy remains fixed, which would violate the principle of human perception. Therefore, in this study, we design LTP with a data-driven threshold according to Weber’s law, a human perception principle; further, our approach incorporates the contexts of spatial and orientation co-occurrences (i.e., co-occurrence context) among adjacent Weber-based local ternary patterns (WLTPs, i.e., data-driven quantized LTPs) for texture representation. The explored WLTP is formulated by adaptively quantizing differential values between neighborhood pixels and the focused pixel as negative or positive stimuli if the normalized differential values are large; otherwise, the stimulus is set to 0. Our approach here is based on the fact that human perception of a distinguished pattern depends not only on the absolute intensity of the stimulus but also on the relative variance of the stimulus. By integrating co-occurrence context information, we further propose a rotation invariant co-occurrence WLTP (RICWLTP) approach to be more discriminant for image representation. In order to validate the efficiency of our proposed strategy, we apply this to three different visual recognition applications including two texture datasets and one food image dataset and prove the promising performance that can be achieved compared with the state-of-the-art approaches.

Breast density is considered to be one of the major risk factors in developing breast cancer. High breast density can also affect the accuracy of mammographic abnormality detection due to the breast tissue characteristics and patterns. We reviewed variants of local binary pattern descriptors to classify breast tissue which are widely used as texture descriptors for local feature extraction. In our study, we compared the classification results for the variants of local binary patterns such as classic LBP (Local Binary Pattern), ELBP (Elliptical Local Binary Pattern), Uniform ELBP, LDP (Local Directional Pattern) and M-ELBP (Mean-ELBP). A wider comparison with alternative texture analysis techniques was studied to investigate the potential of LBP variants in density classification. In addition, we investigated the effect on classification when using descriptors for the fibroglandular disk region and the whole breast region. We also studied the effect of the Region-of-Interest (ROI) size and location, the descriptor size, and the choice of classifier. The classification results were evaluated based on the MIAS database using a ten-run ten-fold cross validation approach. The experimental results showed that the Elliptical Local Binary Pattern descriptors and Local Directional Patterns extracted most relevant features for mammographic tissue classification indicating the relevance of directional filters. Similarly, the study showed that classification of features from ROIs of the fibroglandular disk region performed better than classification based on the whole breast region.