Robust Facial Expression Recognition using Gabor and LDP Feature Fusion using CCA

Facial expression recognition is an important part of the study in man-machine interaction. Principal component analysis (PCA) is an extraction method based on statistical features which were extracted from the global grayscale features of the whole image .But the grayscale global features are environmentally sensitive. In order to recognize facial expression accurately, a fused method of principal component analysis and local direction pattern (LDP) is introduced in this paper. First, PCA extracts the global features of the whole grayscale image; LDP extracts the local grayscale texture features of the mouth and eyes region, which contribute most to facial expression recognition, to complement the global grayscale features of PCA. Then we adopt Support Vector Machine (SVM) classifier for expression classification. Experimental results demonstrate that this method can classify different expressions more effectively and get higher recognition rate compared with the traditional method.

A feature extraction and fusion algorithm was constructed by combining principal component analysis (PCA) and linear discriminant analysis (LDA) to detect a fault state of the induction motor. After yielding a feature vector with PCA and LDA from current signal that was measured by an experiment, the reference data were used to produce matching values. In a diagnostic step, two matching values that were obtained by PCA and LDA, respectively, were combined by probability model, and a faulted signal was finally diagnosed. As the proposed diagnosis algorithm brings only merits of PCA and LDA into relief, it shows excellent performance under the noisy environment. The simulation was executed under various noisy conditions in order to demonstrate the suitability of the proposed algorithm and showed more excellent performance than the case just using conventional PCA or LDA.

In this paper, local directional position pattern (LDPP) and local ternary pattern (LTP) are selected for facial recognition method which are having many advantages over previous techniques like local binary pattern (LBP) and local directional pattern (LDP). The selected techniques of LDPP and LTP are estrangement in their algorithms which help solely to extract features out of an image. LDPP is a revised form of LDP. In a typical LDP, only the top edge direction was taken into consideration, but strength sign of the pixel was not considered which may result in same code for opposite kind of edge pixel. This snag is overcome by LDPP which is further concatenated with LTP for better feature extraction. Once features are extracted they are trained using deep belief network. In the experimental work 10 images of each expression i.e. angry, surprise, disgust, neutral, sad, smile are selected. LDPP and LTP are concatenated followed by principal component analysis (PCA) and general discriminant analysis (GDA). Further for training, Deep Belief Network (DBN) is used which eventually increases the recognition rate and achieve accuracy of 95.3% which was 89.3% without concatenating.

Steady State Visual Evoked Potential (SSVEP) has been commonly adopted in Brain Computer Interface (BCI) applications. For wearable BCI applications, several aspects of SSVEP-based BCI systems, such as speed, subject variability, and accurate target detection, are under ongoing research investigations. Up to date, Canonical Correlation Analysis (CCA) has been considered the state-of-the-art feature extraction method for SSVEP-based BCI systems. Nevertheless, although CCA outperforms traditional SSVEP detection methods, such as Power Spectral Density Analysis (PSDA), achieving high accuracies when detecting target frequencies is still a challenging task due to user variation and physiological changes in the human body. In this paper, we investigate an SSVEP-based BCI application using wireless EEG recording and an Android tablet-based user interface. We propose a fusion of CCA and PSDA solutions at the score level by dividing their score space into multiple partitions, and extract and combine their complementary discriminative information to minimize the detection error in a linear fashion. We investigated transforming the fusion score space to lower dimensions with the purpose of alleviating redundancy. As such, we employed Principal Component Analysis (PCA), and Linear Discriminant Analysis (LDA). Our experimental results demonstrated that our proposed score fusion method is effective in reducing the effect of noise and non-stationary elements in EEG dynamics. Average detection accuracies improved from 63% for CCA to 72% for fusion+PCA and further improved to 98% for fusion+LDA.

Feature fusion is important for providing enhancements of data authenticity in both traditional and deep learning pattern analysis. Classical serial fusion concatenates multiple feature sets followed by dimensionality reduction using principal component analysis (PCA), linear discriminant analysis (LDA), canonical correlation analysis (CCA) etc. CCA-based feature fusion is a main technique for exploring the mutual relationships of multiple feature sets. It considers the correlation of multiple feature sets during dimensionality reduction. In traditional CCA-based feature fusion and extraction, the natural groupings of features are directly used. It is still unclear whether the natural groupings of features are optimal for CCA-based fusion. In this paper, we propose a feature regrouping algorithm for CCA-based feature fusion and extraction through normalized cut (FR-NC). Feature correlation analysis is incorporated into normalized cut, in which the intra-group correlation is maximized, and the extra-group correlation is minimized simultaneously. CCA-based feature fusion is performed on the regrouped features. The proposed feature regrouping algorithm aims to provide enhanced fused features for pattern classification. Extensive experiments have proved its effectiveness.

Recently, there has been a huge demand for assistive technology for industrial, commercial, automobile, and societal applications. In some of these applications, there is a requirement of an efficient and accurate system for automatic facial expression recognition (FER). Therefore, FER has gained enormous interest among computer vision researchers. Although there has been a plethora of work available in the literature, an automatic FER system has not yet reached the desired level of robustness and performance. In most of these works, there has been the dominance of appearance-based methods primarily consisting of local binary pattern (LBP), local directional pattern (LDP), local ternary pattern (LTP), gradient local ternary pattern (GLTP), and improved local ternary pattern (IGLTP). Keeping in view the popularity of appearance-based methods, in this paper, we have proposed an appearance-based descriptor called Improved Adaptive Local Ternary Pattern (IALTP) for automatic FER. This new descriptor is an improved version of ALTP, which has been proved to be effective in face recognition. We have investigated ALTP in more details and have proposed some improvements like the use of uniform patterns and dimensionality reduction via principal component analysis (PCA). The reduced features are then classified using kernel extreme learning machine (K-ELM) classifier. In order to validate the performance of the proposed method, experiments have been conducted on three different FER datasets using well-known evaluation measures such as accuracy, precision, recall, and F1-Score. The proposed approach has also been compared with some of the state-of-the-art works in literature and found to be more accurate and efficient.

Dimensionality reduction of image features using the canonical contextual correlation projection

The recognition rate of person-independent facial expression is generally not high, which limits the practical application of facial expression recognition. Aiming at this problem, this paper analyzes the reasons for the low recognition rate of person-independent facial expression, and proposes a recognition algorithm of person-independent facial expression based on improved LBP (Local Binary Pattern) and HOSVD (Higher-Order Singular Value Decomposition). The algorithm has the following contributions of facial expression recognition framework. In the stage of facial expression feature extraction, the transient features extracted by LDP(Local Directional Pattern) and the persistent features extracted by CBP(Centralized Binary Pattern) are integrated to improve the discrimination of facial expression features. Moreover, in the stage of facial expression classification and recognition, the traditional nearest neighbor classification is changed into k-nearest neighbor pre-classification, and the regional energy calculated by HOSVD is used to determine the similarity of two images for secondary classification. Finally, in the extended Cohn-Kanade dataset and Oulu-CASIA NIR&VIS facial expression database, the theoretical analysis and experimental results show that the method has better recognition effect for solving the problem of person-independent facial expression recognition.

In this paper, we investigate an efficient palmprint texture modeling method that incorporates a robust analysis based on fusing multiple information. In fact, a single descriptor alone may not achieve a high accuracy in palmprint biometric system. Hence, we propose the fusion of various information features extracted by the different descriptors, such as the fractal and the Multi-fractal techniques which produce a robustness to face the numerous challenging and variation of palmprint in unconstrained environments. To increase the performance of palmprint biometric systems, information fusion is proposed as a key phase in multi-characteristic systems. The obtained information can be combined at different levels, i.e., at the feature level, the score level or the decision level. Nevertheless, the feature level fusion is considered more effective than both the matching score and the classifier decision levels, thanks to a feature vector set which contains more and richer information about the input palmprint image. In order to improve the discriminating texture information, our proposed method extracts the fractal dimension features from the preprocessed palmprint images and fuses them with the Multi-fractal dimension features using the Canonical Correlation Analysis (CCA) incorporating the Linear Discriminant Analysis (LDA) in order to reduce the feature dimensionality for each feature set. To demonstrate the feasibility and effectiveness of our proposed method, we performed the experimental results on two benchmark datasets. These results outperform other well-known state of the art methods and produce promising recognition rates by achieving 96.02% for PolyU-Palmprint database and 97.00% for CASIA-Palmprint database.

Extraction of human emotions from facial expression has attracted significant attention in computer vision community. There are several appearance based techniques like local binary patterns (LBP), local directional patterns (LDP), local ternary patterns (LTP) and gradient local ternary patterns (GLTP). Recently, many investigations have been done to improve these feature extraction techniques. Although GLTP has achieved an improvement in robustness to noise and illumination, it encodes image gradient in four directions and two orientations only. This paper proposes to improve GLTP to directional gradient local ternary patterns (DGLTP) by encoding image gradient on eight directions and four orientations. The eight directional Kirsch mask is used to encode the image gradient followed by dimensionality reduction using linear discriminant analysis (LDA) and AVG, MAX and MIN pooling techniques are compared for fusing facial expression features. The proposed technique was experimented on JAFFE facial expression dataset with support vector machine (SVM). The experimental results show that proposed technique improved accuracy of GLTP.

Canonical correlation analysis using within-class coupling

Multimodal biometrics-based systems aim to improve the recognition accuracy of human beings using more than one physical and/or behavioral characteristics of a person. In this paper, different fusion schemes at matching score level and feature level are employed to obtain a robust recognition system using several standard feature extractors. The proposed method involves the consideration of a face–iris multimodal biometric system using score level and feature level fusion. Principal Component Analysis (PCA), subspace Linear Discriminant Analysis (LDA), subpattern-based PCA, modular PCA and Local Binary Patterns (LBP) are global and local feature extraction methods applied on face and iris images. In fact, different feature sets obtained from five local and global feature extraction methods for unimodal iris biometric system are concatenated at feature level fusion called iris feature vector fusion (iris-FVF), while for unimodal face biometric system, LBP is used to achieve efficient texture descriptors. Feature selection is performed using Particle Swarm Optimization (PSO) at feature level fusion step to reduce the dimension of feature vectors for improving the recognition performance. Our proposed method is validated by forming three datasets using ORL, BANCA, FERET face databases and CASIA, UBIRIS iris databases. The results based on recognition performance and ROC analysis demonstrate that the proposed matching score level fusion scheme using Weighted Sum rule, tanh normalization, iris-FVF and facial features extracted by LBP achieves a significant improvement over unimodal and multimodal methods. Support Vector Machine (SVM) and t-norm normalization are also used to improve the recognition performance of the proposed method.

Computer-based automated facial expression recognition has a wide variety of applications that include interactive games, data-driven animation, sociable robotics, and many other human–computer interaction systems. The recognition of facial expressions is a challenging research problem as people have different ways to show their expressions at different occasions. This paper proposes a model for the recognition of facial expressions using thermal infrared (IR) images which has two distinct stages. In the first stage, we have used the concept of snapshot ensembling to combine the features of a customized Convolutional Neural Network (CNN) at different epochs. Though CNN models can learn differently at different epochs from the inputs, there may be some redundancy if the feature maps of different epochs are combined straightforwardly. Hence, in the next stage, we have applied a nature-inspired meta-heuristic called Moth-Flame Optimization algorithm to reduce the feature dimension of the feature vector generated by the snapshot ensembling-based CNN model. The optimized feature vector is fed to the Support Vector Machine for classification of the facial images based on its expression. The proposed model is evaluated on a publicly available facial image database, called IR database, and it successfully classifies 99.63% of the images using only 45% features of the original feature vector. The model outperforms many state-of-the-art methods developed for facial expression recognition which are experimented on the same IR database considered here.

To design a face gloss classification model and to provide an automatic and quantitative approach for the diagnosis of Chinese medicine (CM) based on the face images. To classify the face gloss images into two groups (gloss and non-gloss), feature extraction methods were applied to the original images. The original images were supposed to obtain a more ideal representation in which gloss information was better revealed in four color spaces [including red, green, blue (RGB), hue, saturation, value (HSV), Gray and Lab]. Principal component analysis (PCA), 2-dimensional PCA (2DPCA), 2-directional 2-dimensional PCA [(2D)2PCA], linear discriminant analysis (LDA), 2-dimensional LDA (2DLDA), and partial least squares (PLS) were used as the feature extraction methods of face gloss. k nearest neighbor was used as the classifification method. All the six feature extraction methods were useful in extracting information of face gloss, especially LDA, which had the best prediction accuracy in the 4 color spaces. The average accuracy of LDA in the Lab was 7%-10% higher than that of PCA, 2DPCA, (2D)2PCA and 2DLDA P<0.05). The prediction accuracy of LDA reached 98% in the Lab color space and showed practical usage in clinical diagnosis. The consistent rate between the CM experts and the facial diagnosis system was 81%. A computer-assisted classifification model was designed to provide an automatic and quantitative approach for the gloss diagnosis of CM based on the face images.

Local line directional pattern for palmprint recognition