MMI-based optimal LBP code selection for facial expression recognition

This paper presents an analysis of the statistical texture representation of the Local Binary Pattern (LBP) variants in the classification of wood defect images. The basic and variants of the LBP feature set that was constructed from a stage of feature extraction processes with the Basic LBP, Rotation Invariant LBP, Uniform LBP, and Rotation Invariant Uniform LBP. For significantly discriminating, the wood defect classes were further evaluated with the use of different classifiers. By comparing the results of the classification performances that had been conducted across the multiple wood species, the Uniform LBP was found to have demonstrated the highest accuracy level in the classification of the wood defects.

Texture is the surface property that is used to identify and recognise objects. This property is widely used in many applications including texture-based face recognition systems, surveillance, identity verification and so on. The Local binary pattern (LBP) texture method is most successful for face recognition. Owing to the great success of LBP, recently many models, which are variants of LBP have been proposed for texture analysis. Some of the derivatives of LBPs are multivariate local binary pattern, centre symmetric local binary pattern, local binary pattern variance, dominant local binary pattern, advanced local binary pattern, local texture pattern (LTP) and local derivative pattern (LDP). In this scenario, it is essential to review, whether LBP or their derivatives perform better for face recognition. The real-time challenges such as illumination changes, rotations, angle variations and facial expression variations are evaluated by different LBP-based models. Experiments were conducted on the Japanese female facial expression, YALE and FRGC version2 databases. The results show that LDP and LTP perform much better than the other LBP-based models.

Local binary pattern (LBP) is sensitive to image noise. Noise-resistant LBP (NRLBP) improves the robustness to noise by incorporating the prior knowledge of images and information of other LBP bits into encoding process. However, it encodes the small pixel difference in such a way that its sign and magnitude are ignored. Although the small pixel difference may be easily distorted by noise, some of its information is still useful for LBP encoding. In this letter, we propose two enhanced NRLBPs that jointly utilize the sign and the magnitude of the current pixel difference, and also the information of other LBP bits. The proposed approaches are validated on two benchmark databases and demonstrate a superior performance compared with NRLBP and other LBP variants. The performance gain is significant when the noise level is high.

<p>Various face recognition methods are derived using local features among them the Local Binary Pattern (LBP) approach is very famous. The histogram techniques based on LBP is a complex task. Later Uniform Local Binary Pattern (ULBP) is derived on LBP, based on the bitwise transitions and ULBP’s are treated as the fundamental property of texture. The ULBP approach treated all Non-Uniform Local Binary Patterns’ (NULBP) into one miscellaneous label. Recently we have derived Prominent LBP (PLBP), Maximum PLBP (MPLBP) and Smallest PLBP (SPLBP). The PLBP consists of the majority of the ULBP’s and some of the NULBP’s. The basic disadvantage of these various variants of LBP’s is they are basically local approaches and completely failed in representing features derived from large regions or macrostructures, which are very much essential for faces. This paper derives PLBP’s on the large region. The rectangular region of this paper is assumed with a size of multiples of three and PLBPs are evaluated on dividing each region into multiple regions. The proposed Multi Region-PLBP (MR-PLBP) approach is tested on three facial databases namely Yale, Indian and AT&T ORL. The experimental results show the proposed approach significantly outperforms the other LBP based face recognition methods.</p>

Local binary patterns (LBP) are considered among the most computationally efficient high-performance texture features. However, the LBP method is very sensitive to image noise and is unable to capture macrostructure information. To best address these disadvantages, in this paper, we introduce a novel descriptor for texture classification, the median robust extended LBP (MRELBP). Different from the traditional LBP and many LBP variants, MRELBP compares regional image medians rather than raw image intensities. A multiscale LBP type descriptor is computed by efficiently comparing image medians over a novel sampling scheme, which can capture both microstructure and macrostructure texture information. A comprehensive evaluation on benchmark data sets reveals MRELBP's high performance-robust to gray scale variations, rotation changes and noise-but at a low computational cost. MRELBP produces the best classification scores of 99.82%, 99.38%, and 99.77% on three popular Outex test suites. More importantly, MRELBP is shown to be highly robust to image noise, including Gaussian noise, Gaussian blur, salt-and-pepper noise, and random pixel corruption.

Particularly in the field of biometric security using human face has been widely implemented in the real world. Currently the human face is one of the guidelines in the security system. Nowadays the challenge is how to detect data falsification; such an attack is called spoofing. Spoofing occurs when someone is trying to pretend to be someone else by falsifying the original data and then that person may gain illegal access and benefit him. For example one can falsify the face recognition system using photographs, video, masks or 3D models. In this paper image spoofing human face detection using texture analysis on input image is proposed. Texture analysis used in this paper is the Local Binary Pattern (LBP) and Local Binary Pattern Variance (LBPV). To classified input as original or spoof K-Nearest Neighbor (KNN) used. Experiment used 5761 spoofs and 3362 original from NUAA Imposter dataset. The experimental result yielded a best success rate of 87.22% in term of accuracy with configuration of the system using LBPV and histogram equalization with ratio 𝑅 = 7 and 𝑃 = 8.

Local binary features for texture classification: Taxonomy and experimental study

Texture classification plays a major role in many computer vision applications. Local binary patterns (LBP) encoding schemes have largely been proven to be very effective for this task. Improved LBP (ILBP) are conceptually simple, easy to implement, and highly effective LBP variants based on a point-to-average thresholding scheme instead of a point-to-point one. We propose the use of this encoding scheme for extracting intra- and interchannel features for color texture classification. We experimentally evaluated the resulting improved opponent color LBP alone and in concatenation with the ILBP of the local color contrast map on a set of image classification tasks over 9 datasets of generic color textures and 11 datasets of biomedical textures. The proposed approach outperformed other grayscale and color LBP variants in nearly all the datasets considered and proved competitive even against image features from last generation convolutional neural networks, particularly for the classification of biomedical images.

Texture classification algorithms using local binary pattern (LBP) and its variants usually can achieve attractive results. However, the selected rotation invariant structural patterns in numerous LBP variants are not absolutely continuous invariant to any rotation angle. To improve the classification effectiveness on this occasion, in this paper, we introduce a robust descriptor based on the principal curvatures (PCs) and rotation invariant version of CLBP_Sign operator in completed LBP (CLBP), namely PC-LBP. Different from the original LBP and many LBP variants, PCs are employed in this paper to represent each local structure information due to their continuous rotation invariance. Simultaneously, both micro-and macro-structure texture information can also be captured through PCs, which comprise maximum and minimum curvatures. Inspired by the similar coding strategy of the CLBP_Sign operator, a new operator CLBP_PC is developed. By exploiting complementary information resulting from the two operators combination, the final PC-LBP descriptor has the properties of conspicuous rotation invariance, strong discriminativeness, gray scale invariance, needless of pretraining, and high computational efficiency. In addition, to improve the robustness of texture classification with multiresolution, a multiscale sampling approach is designed by adjusting three parameters accordingly. Experimental results demonstrate that the proposed multiresolution PC-LBP approach achieves comparable performance or outperforms a large number of state-of-the-art methods. Impressively, the classification accuracy of the proposed method performed on Outex_TC_00010 test suite is 100%.

The most expressive manner which human can reveal his emotional states is facial expression. Automatic facial expression recognition is an emerging field of study having extensive applications among which the human-computer interaction (HCI) has received lots of attentions in recent years. The features extracted from facial images, in order to recognize facial expressions, play an essential role in effectiveness of the facial image descriptors. Local binary pattern (LBP) texture descriptors have been known as simple, yet efficient descriptors which are noticeably used for extracting facial patterns from images. Recently, a generalized form of local binary pattern has been introduced which can offer a more precise image description than simple LBP descriptors. Consequently, it would be expected that taking the advantage of using these new LBP texture descriptors will produce more promising results in comparison with use of simple local binary pattern descriptors. In this paper, a novel method has been proposed for image feature extraction using these new image texture descriptors (generalized LBP); then, the obtained results have been compared to the results produced when applying simple LBP descriptors. Furthermore, K-NN and SVM have been used as classifiers in the proposed approach. Finally, a comparison between the proposed method and the existing local binary pattern algorithms for facial expression recognition concludes the superiority of the proposed algorithm over its existing counterparts.

This paper presents a novel variant of Local Binary Patterns (LBP) so called Mean-Elliptical Local Binary Patterns (M-ELBP) for breast density classification. The basic idea of (E)LBP is that an image texture consists of micropatterns and the histogram of these micropatterns could give information about the local features in an image. To include intensity based information along with texture features, we improve the ELBP descriptor by including the mean intensity around the central pixel. We use multiple orientations which facilitates capturing sufficient micropattern features from the mammographic images. The M-ELBP is computed for the fibroglandular disk area instead of the whole breast region. A Bayesian network classifier was used for performing mammogram density classification and a stratified ten-fold cross-validation scheme was used for performance evaluation of the proposed method. The proposed method achieved an accuracy of 74% which is comparable with other methods in the literature using the same dataset.

Local binary pattern (LBP) is sensitive to noise. Local ternary pattern (LTP) partially solves this problem. Both LBP and LTP, however, treat the corrupted image patterns as they are. In view of this, we propose a noise-resistant LBP (NRLBP) to preserve the image local structures in presence of noise. The small pixel difference is vulnerable to noise. Thus, we encode it as an uncertain state first, and then determine its value based on the other bits of the LBP code. It is widely accepted that most of the image local structures are represented by uniform codes and noise patterns most likely fall into the non-uniform codes. Therefore, we assign the value of an uncertain bit hence as to form possible uniform codes. Thus, we develop an error-correction mechanism to recover the distorted image patterns. In addition, we find that some image patterns such as lines are not captured in uniform codes. Those line patterns may appear less frequently than uniform codes, but they represent a set of important local primitives for pattern recognition. Thus, we propose an extended noise-resistant LBP (ENRLBP) to capture line patterns. The proposed NRLBP and ENRLBP are more resistant to noise compared with LBP, LTP, and many other variants. On various applications, the proposed NRLBP and ENRLBP demonstrate superior performance to LBP/LTP variants.

With the work and study pressure on people increasing and the importance of mental health problems on the rise, facial expression analysis plays an important role in mental health auxiliary diagnosis and treatment (MHADT). This study proposes a facial expression localization and recognition model that integrates Main Directional Mean Optical Flow (MDMO) with the Transformer architecture. It addresses the problems of insufficient generalization ability and limited temporal modeling ability of traditional methods in facial expression recognition. The study is based on the authoritative Audio/Visual Emotion Challenge 2019 Depression Detection Sub-challenge (AVEC2019 DDS) dataset in the mental health field, which contains 163 training samples, 56 validation samples, and 56 test samples. With Explicit Shape Regression, Local Binary Patterns, Mnemonic Descent Method, Convolutional Neural Network, and benchmark models as comparison objects, it systematically evaluates the model's performance in error, accuracy, and processing speed. The results show that the model achieves the best performance in both facial expression localization and recognition tasks. The validation set errors are 7.26 and 6.85, the localization accuracy reaches 89.6%, and the recognition accuracy reaches 88.9%, which is significantly better than other methods. At the same time, its image processing time is 55 milliseconds (ms) and 44ms, balancing high precision and real-time performance. The study indicates that the fusion of MDMO and Transformer can effectively capture the spatial and temporal features of facial expressions. Thus, this fusion method provides an efficient, stable, and scalable technical solution for emotion recognition in MHADT. This improves both the FER and localization accuracy and effect. Besides, it provides a novel reference in both approaches and application level for the intelligent development of mental health evaluation.

Facial expression which carries rich information of body behavior is the leading carrier of human affective and the symbol of intelligence. The main purpose of this paper is to recognize 3D human facial expression. The research in this paper includes the expression feature extraction algorithm and fusion with different kinds of feature. To contain more local texture feature information, we proposed a new feature of 3D facial expression named Local Threshold Binary Pattern (LTBP) which based on Local Binary Pattern (LBP). We calculate the difference of gray value standard between neighboring pixels and the center pixel as a threshold to binary instead of the traditional LBP operation which only comparison of size between neighboring pixels and the center pixel. After we get the LTBP feature, we fuse the LTBP and HOG (Histogram of Oriented Gradient) features to get multi-feature fusion for 3D facial expression recognition. Our algorithm of 3D facial expression recognition comprises three steps: (1) extracting two sets of feature vectors and establishing the correlation criterion function between the two sets of feature vectors; (2) solving the two sets canonical projective vectors and extracting their canonical correlation features by the framework of canonical correlation analysis algorithm; (3) doing feature fusion for classification by using proposed strategy. We have performed comprehensive experiments on the BU-3DFE database which is presently the largest available 3D face database. We have achieved verification rates of more than 90% for the 3D facial expression recognition.

Facial expression recognition based on Local Binary Patterns: A comprehensive study