The Application of a Hybrid Transfer Algorithm Based on a Convolutional Neural Network Model and an Improved Convolution Restricted Boltzmann Machine Model in Facial Expression Recognition

In the realm of computer vision, image classification is a critical issue with many applications, including multimedia content analysis, security and surveillance, and medical imaging. The accuracy of image classification algorithms has considerably increased with the development of deep learning. The discipline of image classification has dramatically benefited from the usage of pre-trained Convolutional Neural Network (CNN) models. In this study, we perform image classification on the Wang dataset composed of 1000 images separated into ten categories, using six different pre-trained CNN models. The research made use of pre-trained versions of VGG16, Densenet, Mobilenet, Inception V3, Resnet50, and Xception models. We assessed the model performances in terms of training and testing accuracy. Testing accuracies for ten unique batches of data were calculated and averaged out. Densenet outperformed state-of-the-art models like Xception by a small margin, mainly due to low data availability. The findings of this study show how pre-trained models have advanced the field of image classification and shed light on how well these models perform in diverse image classification tasks. This study can serve as a valuable reference for researchers and practitioners in the field of computer vision and deep learning and help inform their choices when selecting pre-trained models for image classification, depending on their needs, while also considering data availability and computational constraints.

Emotions expressed by humans can be identified from facial expressions, speech signals, or physiological signals. Among them, the use of physiological signals for emotion classification is a notable emerging area of research. In emotion recognition, a person’s electrocardiogram (ECG) and galvanic skin response (GSR) signals cannot be manipulated, unlike facial and voice signals. Moreover, wearables such as smartwatches and wristbands enable the detection of emotions in people’s naturalistic environment. During the COVID-19 pandemic, it was necessary to detect people’s emotions in order to ensure that appropriate actions were taken according to the prevailing situation and achieve societal balance. Experimentally, the duration of the emotion stimulus period and the social and non-social contexts of participants influence the emotion classification process. Hence, classification of emotions when participants are exposed to the elicitation process for a longer duration and taking into consideration the social context needs to be explored. This work explores the classification of emotions using five pretrained convolutional neural network (CNN) models: MobileNet, NASNetMobile, DenseNet 201, InceptionResnetV2, and EfficientNetB7. The continuous wavelet transform (CWT) coefficients were detected from ECG and GSR recordings from the AMIGOS database with suitable filtering. Scalograms of the sum of frequency coefficients versus time were obtained and converted into images. Emotions were classified using the pre-trained CNN models. The valence and arousal emotion classification accuracy obtained using ECG and GSR data were, respectively, 91.27% and 91.45% using the InceptionResnetV2 CNN classifier and 99.19% and 98.39% using the MobileNet CNN classifier. Other studies have not explored the use of scalograms to represent ECG and GSR CWT features for emotion classification using deep learning models. Additionally, this study provides a novel classification of emotions built on individual and group settings using ECG data. When the participants watched long-duration emotion elicitation videos individually and in groups, the accuracy was around 99.8%. MobileNet had the highest accuracy and shortest execution time. These subject-independent classification methods enable emotion classification independent of varying human behavior.

Comparative analysis of deep convolution neural network models on small scale datasets

Deep convolutional neural networks (CNNs) enable learning trainable, highly representative and hierarchical image feature from sufficient training data which makes rapid progress in computer vision possible. There are currently three major techniques that successfully employ CNNs to medical image classification: training the CNN from scratch, using off-the-shelf pretrained CNN features, and transfer learning , i.e., fine-tuning CNN models pretrained from natural image dataset (such as large-scale annotated natural image database: ImageNet) to medical image tasks. In this chapter, we exploit three important factors of employing deep convolutional neural networks to computer-aided detection problems. First, we exploit and evaluate several different CNN architectures including from shallower to deeper CNNs: classical CifarNet, to recent AlexNet and state-of-the-art GoogLeNet and their variants. The studied models contain five thousand to 160 million parameters and vary in the numbers of layers. Second, we explore the influence of dataset scales and spatial image context configurations on medical image classification performance. Third, when and why transfer learning from the pretrained ImageNet CNN models (via fine-tuning) can be useful for medical imaging tasks are carefully examined. We study two specific computer-aided detection (CADe) problems, namely thoracoabdominal lymph node (LN) detection and interstitial lung disease (ILD) classification. We achieve the state-of-the-art performance on the mediastinal LN detection and report the first fivefold cross-validation classification results on predicting axial CT slices with ILD categories. Our extensive quantitative evaluation, CNN model analysis, and empirical insights can be helpful to the design of high-performance CAD systems for other medical imaging tasks, without loss of generality.

Introduction: The early automatic diagnosis of the novel coronavirus (COVID-19) disease could be very helpful to reduce its spread around the world. In this study, we revisit the identification of COVID-19 from chest X-ray images using deep learning. Methods: We collected a relatively large COVID-19 dataset—compared with previous studies—containing 309 real COVID-19 chest X-ray images. We also prepared 2,000 chest X-ray images of pneumonia cases and 1,000 images of healthy controls. Deep transfer learning was used to detect abnormalities in our image dataset. We fine-tuned three, pre-trained convolutional neural network (CNN) models on a training dataset: DenseNet 121, NASNetLarge, and NASNet-Mobile. Results: The evaluation of our models on a test dataset showed that these models achieved an average sensitivity rate of approximately 99.45% and an average specificity rate of approximately 99.5%. Conclusion: A larger dataset of COVID-19 X-ray images could lead to more accurate and reliable identification of COVID-19 infections using deep transfer learning. However, the clinical diagnosis of COVID-19 disease is always necessary.

The adoption of artificial intelligence in post-earthquake inspections and reconnaissance has received considerable attention in recent years, owing to its exponential increase in computation capabilities and inherent potential in addressing disadvantages associated with manual inspections. Herein, we present the effectiveness of automated deep learning in enhancing the assessment of damage caused by the 2017 Pohang earthquake. Six classical pre-trained convolutional neural network (CNN) models are implemented through transfer learning (TL) on a small dataset, comprising 1780 manually labeled images of structural damage. Feature extraction and fine-tuning TL methods are trained on the image datasets. The performances of various CNN models are compared on a testing image dataset. Results confirm that the MobileNet fine-tuned model offers the best performance. Therefore, the model is further developed as a web-based application for classifying earthquake damage. The severity of damage is quantified by assigning damage assessment values, derived using the CNN model and gradient-weighted class activation mapping. The web-based application can effectively and automatically classify structural damage resulting from earthquakes, rendering it suitable for decision making, such as in resource allocation, policy development, and emergency response.

Recognition of emotional states using frequency effective connectivity maps through transfer learning approach from electroencephalogram signals

Convolutional neural networks (CNNs) are an effective technique for hyperspectral image classification. Deep learning models require a large number of labeled and diverse samples to properly train a CNN model. But a training set is often not large enough. Transfer learning can help to overcome the need for training sets. In this paper, six pre-trained CNN models: EfficientNetB0, EfficientNetB7, ResNet50, VGG19, DenseNet121 and DenseNet201 are fine-tuned for hyperspectral image classification. The experiments are carried out on two benchmark images Houston and Kennedy Space Center (KSC). The results show that hyperspectral images can be classified with good accuracy by fine-tuned pre-trained CNN models. As compared to training a model from scratch, fine-tuning takes a small number of epochs. Thus, alleviating the requirement for high-end computing resources. Among the tested models, VGG19 achieves the best accuracy of 95.77% for the KSC image, while EfficientNetB0 performs better than others with 90.79% accuracy for the Houston image.

Fingerprint classification helps in reducing the number of comparisons during the matching stage in automatic fingerprint identification system. In this study, a convolutional neural network model is proposed for classification of plain, rolled and latent fingerprints. We first propose a new convolutional neural network model initialised with random weights and train the model on fingerprint images. Then we fine-tune two pre-trained convolutional neural network models on fingerprint images. Finally, we compare these three models: two pre-trained models and a custom convolutional neural network model initialised with random weights. We show that pre-trained models can achieve the state-of-the-art results on other similar tasks with no or little fine-tuning. We also show that training data size and depth of the network have a serious impact on the overall performance of deep networks. Dropout is used to enhance the performance of deep networks where the labelled training data is not of sufficient size. All the three models trained on NIST DB4 fingerprint and IIIT-D latent fingerprint databases report good accuracy. By only fine-tuning the pre-trained convolutional neural network model, we get the accuracy of 99%, easily out-performing the state-of-the-art.

Prediction of drug response in major depressive disorder using ensemble of transfer learning with convolutional neural network based on EEG

Crop pest classification with a genetic algorithm-based weighted ensemble of deep convolutional neural networks

Background/Objectives: There is only limited research work is going on in the field of facial expression recognition on low resolution images. Mostly, all the images in the real world will be in low resolution and might also contain noise, so this study is to design a novel convolutional neural network model (FERConvNet), which can perform better on low resolution images. Methods: We proposed a model and then compared with state-of-art models on FER2013 dataset. There is no publicly available dataset, which contains low resolution images for facial expression recognition (Anger, Sad, Disgust, Happy, Surprise, Neutral, Fear), so we created a Low Resolution Facial Expression (LRFE) dataset, which contains more than 6000 images of seven types of facial expressions. The existing FER2013 dataset and LRFE dataset were used. These datasets were divided in the ratio 80:20 for training and testing and validation purpose. A HDM is proposed, which is a combination of Gaussian Filter, Bilateral Filter and Non local means denoising Filter. This hybrid denoising method helps us to increase the performance of the convolutional neural network. The proposed model was then compared with VGG16 and VGG19 models. Findings: The experimental results show that the proposed FERConvNet_HDM approach is effective than VGG16 and VGG19 in facial expression recognition on both FER2013 and LRFE dataset. The proposed FERConvNet_HDM approach achieved 85% accuracy on Fer2013 dataset, outperforming the VGG16 and VGG19 models, whose accuracies are 60% and 53% on Fer2013 dataset respectively. The same FERConvNet_HDM approach when applied on LRFE dataset achieved 95% accuracy. After analyzing the results, our FERConvNet_HDM approach performs better than VGG16 and VGG19 on both Fer2013 and LRFE dataset. Novelty/Applications: HDM with convolutional neural networks, helps in increasing the performance of convolutional neural networks in Facial expression recognition. Keywords: Facial expression recognition; facial emotion; convolutional neural network; deep learning; computer vision

Among the many deep learning methods, the convolutional neural network (CNN) model has an excellent performance in image recognition. Research on identifying and classifying image datasets using CNN is ongoing. Animal species recognition and classification with CNN is expected to be helpful for various applications. However, sophisticated feature recognition is essential to classify quasi-species with similar features, such as the quasi-species of parrots that have a high color similarity. The purpose of this study is to develop a vision-based mobile application to classify endangered parrot species using an advanced CNN model based on transfer learning (some parrots have quite similar colors and shapes). We acquired the images in two ways: collecting them directly from the Seoul Grand Park Zoo and crawling them using the Google search. Subsequently, we have built advanced CNN models with transfer learning and trained them using the data. Next, we converted one of the fully trained models into a file for execution on mobile devices and created the Android package files. The accuracy was measured for each of the eight CNN models. The overall accuracy for the camera of the mobile device was 94.125%. For certain species, the accuracy of recognition was 100%, with the required time of only 455 ms. Our approach helps to recognize the species in real time using the camera of the mobile device. Applications will be helpful for the prevention of smuggling of endangered species in the customs clearance area.

Land-cover and land-use classification from aerial images is a challenging problem due to high intra-class diversity and inter-class similarities of the images. To analyze the performances of deep convolutional neural network (CNN) models in this domain, we provide three pre-trained CNN models that are adapted to NWPU-RESISC45 dataset using three different training splits, i.e., 80%, 20%, and 10% ratios. The architectures of all three models are redesigned to be modest in size and their structure is kept simple, yet when tested with the NWPU-RESISC45 dataset, all three models perform comparably to the state-of-the-art models. Each of these models is then used to classify the scenes taken from five well-known datasets in this domain without any fine-tuning. We aim to assess the generalization capabilities of these models on the selected datasets. For better analysis, we considered using top-3 and top-5 accuracies of the models in addition to the best predicted category (top-1) that is usually reported by the models. This way of interpretation is very suitable in this domain, since the datasets contain a high number of fine grained categories with large semantic overlaps. We empirically show that the proposed CNN models actually learn the relevant semantic features in the aerial images better than we observed via standard measures. To the best of our knowledge, this is the first work in this domain that analyzes and presents model generalization performances the way we presented it here.

The convolutional neural network (CNN) has been widely used in image recognition field due to its good performance. This paper proposes a facial expression recognition method based on the CNN model. Regarding the complexity of the hierarchic structure of the CNN model, the activation function is its core, because the nonlinear ability of the activation function really makes the deep neural network have authentic artificial intelligence. Among common activation functions, the ReLu function is one of the best of them, but it also has some shortcomings. Since the derivative of the ReLu function is always zero when the input value is negative, it is likely to appear as the phenomenon of neuronal necrosis. In order to solve the above problem, the influence of the activation function in the CNN model is studied in this paper. According to the design principle of the activation function in CNN model, a new piecewise activation function is proposed. Five common activation functions (i.e., sigmoid, tanh, ReLu, leaky ReLus and softplus–ReLu, plus the new activation function) have been analysed and compared in facial expression recognition tasks based on the Keras framework. The Experimental results on two public facial expression databases (i.e., JAFFE and FER2013) show that the convolutional neural network based on the improved activation function has a better performance than most-of-the-art activation functions.