Facial Affect Recognition Using Regularized Discriminant Analysis-Based Algorithms

Regularized discriminant analysis for the small sample size problem in face recognition

Kernel quadratic discriminant analysis for small sample size problem

Linear Discriminant Analysis (LDA) has been one of the popular subspace methods for face recognition. But this method suffers from the small sample size (SSS) problem, also known as 'curse of dimensionality'. Various techniques have been proposed in literature to overcome this limitation. But it is still unclear which method provides the best solution to SSS problem. In this paper, we have investigated the performance of some popular subspace methods such as principal component analysis (PCA), PCA + LDA, LDA via QR decomposition, Null-space LDA, Exponential Discriminant Analysis (EDA), PCA+EDA etc. Extensive experiments have been performed on five publically available face datasets viz. AR, CMU-PIE, PIX, Yale and YaleB. The performance is measured in terms of average classification accuracy. Experimental results show that the performance increases with the increase in the number of images per person in training set irrespective of the datasets. There is no clear winner among the subspace methods under investigation. But, the performance of PCA+LDA and PCA+EDA is consistent in tackling SSS problem irrespective of the dataset and can also handle the illumination variation in face recognition.

Machine learning-based receiver operating characteristic (ROC) curves for crisp and fuzzy classification of DNA microarrays in cancer research

Sufficient dimension-reduction methods provide effective ways to visualize discriminant analysis problems. For example, Cook and Yin showed that the dimension-reduction method of sliced average variance estimation (SAVE) identifies variates that are equivalent to a quadratic discriminant analysis (QDA) solution. This article makes this connection explicit to motivate the use of SAVE variates in exploratory graphics for discriminant analysis. Classification can then be based on the SAVE variates using a suitable distance measure. If the chosen measure is Mahalanobis distance, then classification is identical to QDA using the original variables. Just as canonical variates provide a useful way to visualize linear discriminant analysis (LDA), so do SAVE variates help visualize QDA. This would appear to be particularly useful given the lack of graphical tools for QDA in current software. Furthermore, whereas LDA and QDA can be sensitive to nonnormality, SAVE is more robust.

People can identify emotion from facial expressions easily, but it is more difficult to do it with a computer. It is now feasible to identify feelings from images because of recent advances in computational intelligence. Emotional responses are those mental states of thoughts that develop without conscious effort and are naturally associated with facial muscles, resulting in different facial expressions such as happy, sad, angry, contempt, fear, surprise etc. Emotions play an important role in nonverbal cues that represent a person’s interior thoughts. Intimate robots are expanding in every domain, whether it is completing requirements of elderly people, addressing psychiatric patients, child rehabilitation, or even childcare, as the human–robot interface is grabbing on every day with the increased demand for automation in every industry. We evaluate and test machine learning algorithms on the FER 2013 data set to recognize human emotion from facial expressions, with some of them achieving the highest accuracy and the others failing to detect emotions. Many researchers have used various machine learning methods to identify human emotions during the last few years. In this research article, we analyze eight frequently used machine learning techniques on the FER 2013 dataset to determine which method performs best at categorizing human facial expression. After analyzing the results, it is found that the accuracy of some of the algorithms is quite satisfactory, with 37% for Logistic Regression, 33% for K-neighbors classifier, 100% for Decision Tree Classifier, 78% for Random Forests, 57% for Ada-Boost, 100% for Gaussian NB, 33% for LDA (Linear Discriminant Analysis), and 99% for QDA (Quadratic Discriminant Analysis). Furthermore, the experimental results show that the Decision Tree and Gaussian NB Classifier can correctly identify all of the emotions in the FER 2013 dataset with 100% classification accuracy, while Quadratic Discriminant Analysis can do so with 99% accuracy.KeywordsFacial expressionEmotionMachine learningKNNQDADecision treeGaussian NBRandom Forest

Many contemporary studies involve the classification of a subject into two classes based on n observations of the p variables associated with the subject. Under the assumption that the variables are normally distributed, the well-known linear discriminant analysis (LDA) assumes a common covariance matrix over the two classes while the quadratic discriminant analysis (QDA) allows different covariance matrices. When p is much smaller than n, even if they both diverge, the LDA and QDA have the smallest asymptotic misclassification rates for the cases of equal and unequal covariance matrices, respectively. However, modern statistical studies often face classification problems with the number of variables much larger than the sample size n, and the classical LDA and QDA can perform poorly. In fact, we give an example in which the QDA performs as poorly as random guessing even if we know the true covariances. Under some sparsity conditions on the unknown means and covariance matrices of the two classes, we propose a sparse QDA based on thresholding that has the smallest asymptotic misclassification rate conditional on the training data. We discuss an example of classifying normal and tumor colon tissues based on a set of p = 2; 000 genes and a sample of size n = 62, and another example of a cardiovascular study for n = 222 subjects with p = 2; 434 genes. A simulation is also conducted to check the performance of the proposed method.

The performance of seven classification methods, binary logistic (BLR), probit (PR) and cumulative probit (CPR) regression, linear (LDA) and quadratic (QDA) discriminant analysis, artificial neural networks (ANN), and naïve Bayes classification (NBC), is examined in skeletal sex estimation. These methods were tested using cranial and pelvic sexually dimorphic traits recorded on a modern documented collection, the Athens Collection. For their implementation, an R package has been written to perform cross-validated (CV) sex classification and give the discriminant function of each of the methods studied. A simple algorithm that combines two discriminant functions is also proposed. It was found that the differences in the classification performance between BLR, PR, CPR, LDA, QDA, ANN, and NBC are overall small. However, LDA is simpler and more flexible than CPR, QDA, and ANN and has a small but clear advantage over BLR, NBC, and PR. Consequently, LDA may be preferred in skeletal sex estimation. Finally, it is striking that the combination of pelvic and cranial traits via their discriminant functions, determined either by BLR or LDA, removes practically any population-specificity and yields much better predictions than the individual functions; in fact, the prediction accuracy increases above 97%.

Automated recognition of facial expression has attracted significant attention in recent years due to its potential applicability in security and surveillance, human computer interaction, social robotics, and animation. This paper presents a new facial expression recognition method that utilizes bit plane specific local image description in a weighted score level fusion. The motivation is to utilize bit plane slicing to highlight the contribution of a particular bit plane made to the holistic facial appearance, which is then used in a weighted score level fusion in order to boost the recognition performance. A new local image descriptor is proposed specifically to extract local features from bit plane representations that utilizes Fisher linear discriminant to maximize the inter-class distance, while minimizing the intra-class variance. Two well-known facial expression databases, namely the Cohn-Kanade (CK) and the Japanese female facial expression (JAFFE) database have been used to evaluate the performance of the proposed method against existing facial appearance descriptors, such as local binary pattern (LBP), local ternary pattern (LTP), local directional pattern (LDP), and linear discriminant analysis (LDA). Experiments with a total of seven prototypic facial expressions show promising results for the proposed method, as compared with the other existing methods.

The objective of this study is to concentrate on the classification method of the logistic regression and the discriminant analysis by using the simulation dataset and the liver patients as the actual data. These datasets are used the binary dependent variable depending on the correlated independent variables or called multicollinearity data. The standard classification method is logistic regression, which uses the logit function’s probability to conduct the dichotomous dependent variable. The iteration process can be solved to estimate logit function parameters and explain the relationship between a dependent binary variable and independent variables. Discriminant analysis is a powerful classification based on linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and regularized discriminant analysis (RDA). These methods consider the decision boundaries by building a classifier model on the multivariate normal distribution. LDA defines the standard covariance matrix, but QDA has an individual covariance matrix. RDA extends from QDA by setting the regularized parameter to estimate the covariance matrix. In the case of the simulation study, the independent variables are generated by defining the constant correlation on the multivariate normal distribution that made the multicollinearity problem. Then the binary response variable can be approximated from the logit function. For application to actual data, we expressed the classification of type liver and non-liver patients as the dependent variables and obtained patient personal information on the nine independent variables. The highest average percentage of accuracy determines the performance of these methods. The results have shown that the logistic regression was successful when using small independent variables, but the RDA performed when using large independent variables.

We investigated three classifiers for the task of distinguishing between benign and malignant breast lesions. Classification performance was measured in terms of area under the ROC curve (AUC value). We compared linear discriminant analysis (LDA), quadratic discriminant analysis (QDA) and a Bayesian neural net (BNN) with 5 hidden units. For each lesion, 46 image features were extracted and principal component analysis (PCA) of these features was used as classifier input. For each classifier, the optimal number of principal components was determined by performing PCA within each step of a leave-one-case-out protocol for the training dataset (1125 lesions, 14% cancer prevalence) and determining which number of components maximized the AUC value. Subsequently, each classifier was trained on the training dataset and applied ‘cold turkey’ to an independent test set from a different population (341 lesions, 30% cancer prevalence). The optimal number of principal components for LDA was 24, accounting for 97% of the variance in the image features. For QDA and BNN, these numbers were 5 (70%) and 15 (93%), respectively. The LDA, QDA and BNN obtained AUC values of 0.88, 0.85, and 0.91, respectively, in the leave-one-case-out analysis. In the independent test – with AUCs of 0.88, 0.76, and 0.82 – only LDA achieved performance identical to that for the training set (lower bound of 95% non-inferiority interval -.0067), while the others performed significantly worse (p-values << 0.05). While the more complex BNN classifier outperformed the others in leave-one-case-out of a large dataset, LDA was the robust best-performer in an independent test.

Lung cancer is a type of cancer that starts from the lungs with abnormal cell growth. Lung cancer is a type of cancer with the largest number of sufferers. In this research, Regularized Discriminant Analysis (RDA) analysis is used to classify the types of lung cancer that exist. Regularized Discriminant Analysis (RDA) is a generalization of Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA). If the alpha parameter is set to 1, this operator performs an LDA. Likewise if the alpha parameter is set to 0, this operator performs QDA. Discriminant analysis is used to determine which variables distinguish between two or more groups that occur naturally. With an accuracy value of 60%, it can be ascertained the type of lung cancer suffered by patients for ease of care and treatment.

Motor oil classification is important for quality control and the identification of oil adulteration. In this work, we propose a simple, rapid, inexpensive and nondestructive approach based on image analysis and pattern recognition techniques for the classification of nine different types of motor oils according to their corresponding color histograms. For this, we applied color histogram in different color spaces such as red green blue (RGB), grayscale, and hue saturation intensity (HSI) in order to extract features that can help with the classification procedure. These color histograms and their combinations were used as input for model development and then were statistically evaluated by using linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and support vector machine (SVM) techniques. Here, two common solutions for solving a multiclass classification problem were applied: (1) transformation to binary classification problem using a one-against-all (OAA) approach and (2) extension from binary classifiers to a single globally optimized multilabel classification model. In the OAA strategy, LDA, QDA, and SVM reached up to 97% in terms of accuracy, sensitivity, and specificity for both the training and test sets. In extension from binary case, despite good performances by the SVM classification model, QDA and LDA provided better results up to 92% for RGB-grayscale-HSI color histograms and up to 93% for the HSI color map, respectively. In order to reduce the numbers of independent variables for modeling, a principle component analysis algorithm was used. Our results suggest that the proposed method is promising for the identification and classification of different types of motor oils.

In facial expression recognition applications, the images are corrupted with random noise, and this affects the classification accuracy. This article proposes a maximum and Minimum Response-based Gabor (MMRG) that can encode the facial texture more discriminatively and eliminate random noise. Two code images are produced from the available Gabor images. Then, after dividing the code images into grids, feature vectors are formed using histograms. A technique based on the bat algorithm is proposed for the optimization of the Gabor filter banks as Bat Algorithm-based Gabor Optimization (BAGO). The MMRG increases the efficiency of Gabor filter-based features by precisely distinguishing the texture frequencies. It also helps in reducing the dimensions of feature vector which is a major problem in Gabor filter-based feature extraction. Radial Basis Function-Extreme Learning Machine (RBF-ELM) classifier is used for a faster and accurate multi-classification. The proposed approach has been evaluated with six datasets namely, Japanese Female Facial Expression (JAFFE), Cohn Kanade (CK+), Multi-media Understanding Group (MUG), Static Facial Expressions in the Wild (SFEW), Oulu-Chinese Academy of Science, Institute of Automation (Oulu-CASIA) and Man–Machine Interaction (MMI) datasets to meet a classification accuracy of 97.2, 97.4, 95.4, 35.4, 87.4 and 82.3% for seven class emotion detection, which is high when compared to other state-of –the-art methods.

This paper presents a new method for facial expression recognition using divided local binary pattern features. Motivated by Fourier’s parity decomposition, we decompose a LBP descriptor into two subdescriptors. This method reduces the amount of data of the feature extraction, reducing the data dimension effectively. Experimental results on JAFFE facial expression database show that the proposed method can get significant performance improvement in recognition rate, reduce the recognition time greatly.