A Resource Efficient Integer-Arithmetic-Only FPGA-Based CNN Accelerator for Real-Time Facial Emotion Recognition

Recognition of human activity based on convolutional neural network has received the interest of researchers in recent years due to its significant improvement in accuracy. A large number of algorithms based on the deep learning approach have been proposed for activity recognition purpose. However, with the increasing advancements in technologies having limited computational resources, it needs to design an efficient deep learning-based approaches with improved utilization of computational resources. This paper presents a simple and efficient 2-Dimensional convolutional neural network (2-D CNN) architecture with very small size convolutional kernel for human activity recognition. The merit of the proposed CNN architecture over standard deep learning architectures is fewer trainable parameters and lesser memory requirement which enables it to train the proposed CNN architecture on low GPU memory-based devices and also works well with smaller as well as larger size datasets. The proposed approach consists of mainly four stages: namely (1) creation of dataset and data augmentation, (2) designing 2-D convolutional neural network (CNN) architecture, (3) the proposed 2-D CNN architecture trained from scratch up to optimum stage, and (4) evaluation of the trained 2-D CNN architecture. To illustrate the effectiveness of the proposed architecture several extensive experiments are conducted on three publicly available datasets, namely IXMAS, YouTube, and UCF101 dataset. The results of the proposed method and its comparison with other state-of-the-art methods [8-12,14,18-26,29-33] demonstrate the usefulness of the proposed method.

Automatic Optical Inspection (AOI) is any method of detecting defects during a Printed Circuit Board (PCB) manufacturing process. Early AOI methods were based on classic image processing algorithms using a reference PCB. The traditional methods require very complex and inflexible preprocessing stages. With recent advances in the field of deep learning, especially Convolutional Neural Networks (CNN), automating various computer vision tasks has been established. Limited research has been carried out in the past on using CNN for AOI. The present systems are inflexible and require a lot of preprocessing steps or a complex illumination system to improve the accuracy. This paper studies the effectiveness of using CNN to detect soldering bridge faults in a PCB assembly. The paper presents a method for designing an optimized CNN architecture to detect soldering faults in a PCBA. The proposed CNN architecture is compared with the state-of-the-art object detection architecture, namely YOLO, with respect to detection accuracy, processing time, and memory requirement. The results of our experiments show that the proposed CNN architecture has a 3.0% better average precision, has 50% less number of parameters and infers in half the time as YOLO. The experimental results prove the effectiveness of using CNN in AOI by using images of a PCB assembly without any reference image, any complex preprocessing stage, or a complex illumination system. Doi: 10.28991/HIJ-2022-03-01-01 Full Text: PDF

The detection and classifications of weld images is important for improving the quality of the joined materials during production period. In order to automate the classification of weld images in industry, this paper proposes an effective automatic method for the detection and classifications of the weld images into four different cases using deep learning methods. In this work, Convolutional Neural Network (CNN) is adopted for the weld image classification by modifying the internal architecture of the CNN architecture. This proposed ResNet CNN architecture is designed with three Convolutional layers, two numbers of pooling layers with activation layer and two numbers of Fully Connected Neural Networks (FCNN).The FCNN in proposed CNN architecture is designed with 15 internal hidden layers and each hidden layer is designed with 20 neurons which obtains high classification efficiency. The morphological activity functional methods are used on the classified weld images to detect the crack regions.

Automatic modulation classification (AMC) plays an important role in many fields to identify the modulation type of wireless signals. In this paper, we introduce deep learning to signal recognition. Based on architecture analysis of the convolutional neural network (CNN), we used real signal data generated by instruments as dataset, and proposed an improved CNN architecture to achieve compatible recognition accuracy of modulation classification. According to various conditions of signal noise ratio (SNR), we test the proposed CNN architecture with the real sampled signals. Experiments results show that the high-layer network is not necessary for modulation recognition with high SNR signals. The proposed CNN architecture has higher average classification accuracy than RESNET and is more compatible for modulation classification of signals with lower SNR.

Advanced lung tumor diagnosis using a 3D deep neural network based CAD system

Although font recognition is a fundamental issue in the field of document analysis and recognition, it was usually ignored in the past. With the development of optical character recognition (OCR), font recognition becomes more and more important. This paper proposed a well-designed convolutional neural network (CNN) architecture for traditional Mongolian font recognition by means of a single word. To be specific, the whole word image is regarded as input of CNN. Hence, the word images should be normalized into the same size before being inputted into CNN. By comparison, an appropriate aspect ratio for the traditional Mongolian word images has been determined. Experimental results demonstrate that the proposed CNN architecture outperforms three classic CNN architectures, including LeNet-5, AlexNet and GoogLeNet. Therefore, the proposed CNN is much more suitable for the task of the traditional Mongolian font recognition in the way of a single word.

Music genre classification based on visual representation has been successfully explored over the last years. Recently, there has been increasing interest in attempting convolutional neural networks (CNNs) to achieve the task. However, most of the existing methods employ the mature CNN structures proposed in image recognition without any modification, which results in the learning features that are not adequate for music genre classification. Faced with the challenge of this issue, we fully exploit the low-level information from spectrograms of audio and develop a novel CNN architecture in this paper. The proposed CNN architecture takes the multi-scale time-frequency information into considerations, which transfers more suitable semantic features for the decision-making layer to discriminate the genre of the unknown music clip. The experiments are evaluated on the benchmark datasets including GTZAN, Ballroom, and Extended Ballroom. The experimental results show that the proposed method can achieve 93.9%, 96.7%, 97.2% classification accuracies respectively, which to the best of our knowledge, are the best results on these public datasets so far. It is notable that the trained model by our proposed network possesses tiny size, only 0.18M, which can be applied in mobile phones or other devices with limited computational resources. Codes and model will be available at https://github.com/CaifengLiu/music-genre-classification .

In this paper, we study the performance of variants of well-known Convolutional Neural Network (CNN) architectures on different audio tasks. We show that tuning the Receptive Field (RF) of CNNs is crucial to their generalization. An insufficient RF limits the CNN's ability to fit the training data. In contrast, CNNs with an excessive RF tend to over-fit the training data and fail to generalize to unseen testing data. As state-of-the-art CNN architectures - in computer vision and other domains - tend to go deeper in terms of number of layers, their RF size increases and therefore they degrade in performance in several audio classification and tagging tasks. We study well-known CNN architectures and how their building blocks affect their receptive field. We propose several systematic approaches to control the RF of CNNs and systematically test the resulting architectures on different audio classification and tagging tasks and datasets. The experiments show that regularizing the RF of CNNs using our proposed approaches can drastically improve the generalization of models, out-performing complex architectures and pre-trained models on larger datasets. The proposed CNNs achieve state-of-the-art results in multiple tasks, from acoustic scene classification to emotion and theme detection in music to instrument recognition, as demonstrated by top ranks in several pertinent challenges (DCASE, MediaEval).

Every year, new Convolutional Neural Network (CNN) architectures appear to deal with different problems in the activity of image and video recognition. These architectures usually work along the ImageNet dataset for looking for the best performance of the CNNs without taking into account the video task where they are used. This can represent a problem if the task is Human Action Recognition (HAR) in video, since the CNN architectures are pre-trained with an image dataset that can practically contain any object, while HAR problem requires consecutive frames of people doing actions. To prove the idea that using CNNs pre-trained on an image dataset does not always achieve the best performance on a video dataset and that, therefore, it is worth comparing the performance of different CNNs under similar circumstances for the HAR problem, this work proposes an analysis between eight different CNN architectures. Each one of the CNN was exclusively trained with RGB images, which were extracted from the frames of the different classes of videos of HMDB51 dataset. To make the classification of an activity in video, we average the predictions taking into account the successes. We also made some ensembles with the best performance CNNs to measure the improvement in accuracy. Our results suggest that Xception is a strong baseline model that could be used by the community to make their comparisons of their proposals more robust.

In this paper, the Convolutional Neural Network (CNN) architecture, which is one of the deep learning architectures, is used to classify the basic circuit components drawn by hand. During the training and testing stages of the model, a new dataset containing images of 863 circuit components manually drawn by different people is created. The data set contains images of four different classes of circuit components such as resistor, inductor, capacitor and voltage source. All images have been fixed to the same size and converted to grayscale to increase recognition performance and reduce process complexity. In the study, training for four classes is performed with CNN architecture. Based on the CNN architecture, four new CNN models are employed with different the number of layers. The training and validation results of these models are compared separately, the model with the highest training and validation performance is observed with four layer CNN model (CNN-4). This model obtained 84.41% accuracy rate at classification task.

One of the ways humans communicate is by using facial expressions. Research on technology development in artificial intelligence uses deep learning methods in human and computer interactions as an effective system application process. One example, if someone does show and tries to recognize facial expressions when communicating. The prediction of the expression or emotion of some people who see it sometimes does not understand. In psychology, the detection of emotions or facial expressions requires analysis and assessment of decisions in predicting a person's emotions or group of people in communicating. This research proposes the design of a system that can predict and recognize the classification of facial emotions based on feature extraction using the Convolution Neural Network (CNN) algorithm in real-time with the OpenCV library, namely: TensorFlow and Keras. The research design implemented in the Raspberry Pi consists of three main processes, namely: face detection, facial feature extraction, and facial emotion classification. The prediction results of facial expressions in research with the Convolutional Neural Network (CNN) method using Facial Emotion Recognition (FER-2013) were 65.97% (sixty-five point ninety-seven percent).

Pathologists often deal with high complexity and sometimes disagreement over osteosarcoma tumor classification due to cellular heterogeneity in the dataset. Segmentation and classification of histology tissue in H&E stained tumor image datasets is a challenging task because of intra-class variations, inter-class similarity, crowded context, and noisy data. In recent years, deep learning approaches have led to encouraging results in breast cancer and prostate cancer analysis. In this article, we propose convolutional neural network (CNN) as a tool to improve efficiency and accuracy of osteosarcoma tumor classification into tumor classes (viable tumor, necrosis) versus nontumor. The proposed CNN architecture contains eight learned layers: three sets of stacked two convolutional layers interspersed with max pooling layers for feature extraction and two fully connected layers with data augmentation strategies to boost performance. The use of a neural network results in higher accuracy of average 92% for the classification. We compare the proposed architecture with three existing and proven CNN architectures for image classification: AlexNet, LeNet, and VGGNet. We also provide a pipeline to calculate percentage necrosis in a given whole slide image. We conclude that the use of neural networks can assure both high accuracy and efficiency in osteosarcoma classification.

Unmanned aerial vehicles (UAVs) are invaluable technologies concerning their remote control and monitoring capabilities. Convolutional neural networks (CNNs), known for their high pattern recognition capabilities, are appropriate for forest fire detection with UAVs. Deep convolutional neural networks show substantial performance on hardware with high processing capabilities. While these networks can be operated in unmanned aerial vehicles controlled from ground control stations equipped with GPU-supported hardware, the execution on a typical UAV’s limited computational resources necessitates the use of lightweight, small-sized networks. To overcome these impediments, this article presents a lightweight and attention-based approach for performing forest fire detection tasks using UAV vision data (images acquired by cameras mounted on UAVs). In this paper, we also present comprehensive research for different approaches such as transfer learning, deep CNNs, and lightweight CNNs. Among the experimented models, the attention-based EfficientNetB0 backboned model emerged as the most successful architecture for forest fire detection. With the test accuracy of , the F1-score of , the recall of , and the precision of have strongly reinforced the efficiency of the EfficientNetB0-based model in wildfire recognition. Moreover, the network has a less parameter size than the experimented networks. It proves the model’s suitability for wildfire detection with UAVs having limited hardware resources.

Can edges help convolution neural networks in emotion recognition?

Deep learning, in particular Convolutional Neural Network (CNN), has achieved promising results in face recognition recently. However, it remains an open question: why CNNs work well and how to design a 'good' architecture. The existing works tend to focus on reporting CNN architectures that work well for face recognition rather than investigate the reason. In this work, we conduct an extensive evaluation of CNN-based face recognition systems (CNN-FRS) on a common ground to make our work easily reproducible. Specifically, we use public database LFW (Labeled Faces in the Wild) to train CNNs, unlike most existing CNNs trained on private databases. We propose three CNN architectures which are the first reported architectures trained using LFW data. This paper quantitatively compares the architectures of CNNs and evaluate the effect of different implementation choices. We identify several useful properties of CNN-FRS. For instance, the dimensionality of the learned features can be significantly reduced without adverse effect on face recognition accuracy. In addition, traditional metric learning method exploiting CNN-learned features is evaluated. Experiments show two crucial factors to good CNN-FRS performance are the fusion of multiple CNNs and metric learning. To make our work reproducible, source code and models will be made publicly available.