Face Recognition Using Depth Images Base Convolutional Neural Network

Recently, many researches have been conducted on recognition of facial emotion using convolutional neural networks (CNNs), which show excellent performance in computer vision. To obtain a high classification accuracy, a CNN architecture with many parameters and high computational complexity is required. However, this is not suitable for embedded systems where hardware resources are limited. In this paper, we present a lightweight CNN architecture optimized for embedded systems. The proposed CNN architecture has a small memory footprint and low computational complexity. Furthermore, a novel hardware-friendly quantization method that uses only integer-arithmetic is proposed. The proposed hardware-friendly quantization method maps the scale factors to power-of-two terms and replaces multiplication and division operations using scale factors with shift operations. To improve the generalization and classification performance of the CNN, we create the FERPlus-A dataset. This is a new training dataset created using a variety of image processing algorithms. After training with FERPlus-A, quantization has been performed. The size of a quantized CNN parameter is about 0.39 MB, and the number of operations is about 28 M integer operations (IOPs). By evaluating the performance of the quantized CNN that uses only integer-arithmetic on the FERPlus test dataset, the classification accuracy is approximately 86.58%. It achieved higher accuracy than other lightweight CNNs in prior studies. The proposed CNN architecture that uses only integer-arithmetic is implemented on the Xilinx ZC706 SoC platform for real-time facial emotion recognition by applying parallelism strategies and efficient data caching strategies. The FPGA-based CNN accelerator implemented for real-time facial emotion recognition achieves about 10 frame per second (FPS) at 250 MHz and consumes 2.3 W.

The detection of abnormal activities in deep learning is of great significance for preventing the occurrence of abnormal disasters in mine production. As the underground scenes of coal mines are characterized by much noise and uneven light, the traditional manual feature extraction method has little obvious effect in the underground and low accuracy of anomaly detection. To solve the above problems, a feature extraction method combining CNN+LSTM is proposed. Secondly, the obtained features are matched by graph structure. Finally, multiple classifiers are used to classify the features before and after matching. In this paper, experiments are carried out in coal mine dataset and UCSDped1 dataset respectively, and comparisons are made with some classical algorithms. Experimental show that the algorithm achieves high recognition accuracy in different abnormal event datasets.

Signa1 modulation recognition (SMR) is an essential and challenging topic in orthogonal frequency-division multiplexing (OFDM) systems, and also it is the fundamental technique for signal detection and recovery. However, traditional feature extraction based SMR methods cannot effectively acquire the characteristics of the OFDM signals. Hence, the modulated OFD-M signal cannot be reliably identified. In this paper, we propose a deep learning (DL) based SMR method for recognizing OFDM signals, which is combined with a convolutional neural network (CNN) trained on in-phase and quadrature (IQ) samples. In the network model, the batch normalization (BN) layer and dropout layer are used to speed up model training and prevent overfitting, respectively. Three convolution layers with different convolution kernels perform well than traditional feature extraction methods in obtaining intrinsic properties of OFDM signals. The same number of multiple modulated signals are mixed and sent to the trained model for identification. Experiments are conducted to show that the method we proposed performs better than the traditional methods, mainly reflected in a higher probability of correct classification (PCC) and better consistency.

In recent years, the performances of face recognition have been improved significantly by using convolution neural networks (CNN) as the feature extractors. On the other hands, in order to avoid spreading COVID-19 virus, people would wear mask even when they want to pass the face recognition system. Thus, it is necessary to improve the performance of masked face recognition so that users could utilize face recognition methods more easily. In this paper, we propose a feature extraction backbone named ResSaNet that integrates CNN (especially Residual block) and Self-attention module into the same network. By capturing the local and global information of face area simultaneously, our proposed ResSaNet could achieve promising results on both masked and non-masked testing data.

This study describes a method for classifying electrocorticograms (ECoGs) based on motor imagery (MI) on the brain–computer interface (BCI) system. This method is different from the traditional feature extraction and classification method. In this paper, the proposed method employs the deep learning algorithm for extracting features and the traditional algorithm for classification. Specifically, we mainly use the convolution neural network (CNN) to extract the features from the training data and then classify those features by combing with the gradient boosting (GB) algorithm. The comprehensive study with CNN and GB algorithms will profoundly help us to obtain more feature information from brain activities, enabling us to obtain the classification results from human body actions. The performance of the proposed framework has been evaluated on the dataset I of BCI Competition III. Furthermore, the combination of deep learning and traditional algorithms provides some ideas for future research with the BCI systems.

Automatic Optical Inspection (AOI) is any method of detecting defects during a Printed Circuit Board (PCB) manufacturing process. Early AOI methods were based on classic image processing algorithms using a reference PCB. The traditional methods require very complex and inflexible preprocessing stages. With recent advances in the field of deep learning, especially Convolutional Neural Networks (CNN), automating various computer vision tasks has been established. Limited research has been carried out in the past on using CNN for AOI. The present systems are inflexible and require a lot of preprocessing steps or a complex illumination system to improve the accuracy. This paper studies the effectiveness of using CNN to detect soldering bridge faults in a PCB assembly. The paper presents a method for designing an optimized CNN architecture to detect soldering faults in a PCBA. The proposed CNN architecture is compared with the state-of-the-art object detection architecture, namely YOLO, with respect to detection accuracy, processing time, and memory requirement. The results of our experiments show that the proposed CNN architecture has a 3.0% better average precision, has 50% less number of parameters and infers in half the time as YOLO. The experimental results prove the effectiveness of using CNN in AOI by using images of a PCB assembly without any reference image, any complex preprocessing stage, or a complex illumination system. Doi: 10.28991/HIJ-2022-03-01-01 Full Text: PDF

Fine-tuning Pre-trained Convolutional Neural Networks for Gastric Precancerous Disease Classification on Magnification Narrow-band Imaging Images

Automatic weed detection and mapping are critical for site-specific weed control in order to reduce the cost of farming as well as the impact of herbicides on human health. In this paper, we investigate patch-based weed identification using hyperspectral images. Convolutional Neural Network (CNN) is evaluated and compared with the Histogram of Oriented Gradients (HoG) for this purpose. Suitable patch sizes are investigated. The limitation of RGB imagery is demonstrated. The experimental results indicate that the overall accuracy of the weed classification using CNN increases with the increasing number of bands used. With more bands, CNN extracts more powerful and discriminative features and leads to improved classification as compared to the traditional HoG feature extraction method. The computational load of CNN, however, is slightly increased with the increasing number of bands.

With the rapid development of Internet of Things technology, intelligent signal detection has become a highly challenging key issue. Traditional manual feature extraction methods are no longer able to cope with complex and ever-changing practical environments. In response to this issue, this study proposes an intelligent signal detection model for the Internet of Things based on deep learning. This model adopts a convolutional neural network structure, which can automatically learn local pattern features in signals, thereby improving its adaptability to complex environments. Research a large-scale wireless signal dataset constructed based on real IoT scenarios, and adopt reasonable data preprocessing and augmentation strategies. Through scientific model training and multi-dimensional evaluation index settings, a comprehensive evaluation of the proposed model was conducted on the test set. The experimental results show that the model has achieved excellent performance over traditional methods in key indicators such as accuracy, recall, and F1 score, especially in complex signal recognition scenarios.

Face recognition technology has evolved as a popular identification technique to perform verification of human identity. By using the feature extraction methods and dimensionality reduction techniques in the pattern recognition applications, a number of facial recognition systems has been produced with distinct measure of success. Various face recognition algorithms and their extensions, have been proposed in the past three decades. However, face recognition faces challenging problems in real life applications because of the variation in the illumination of the face images. In the recent years, the research is focused towards Illumination-invariant face recognition system and many approaches have been proposed. But, there are several issues in face recognition across illumination variation which still remains unsolved. This paper provides a research on an efficient illumination-invariant face recognition system using Discrete Cosine Transform (DCT) and Principal Component Analysis (PCA). For processing the illumination invariant image, low frequency components of DCT are used to normalize the illuminated image, odd and even components of DCT is used for compensation in illumination variation and PCA is used for recognition of face images. The existing approaches in illumination Invariant face recognition are comprehensively reviewed and discussed. The proposed approach is validated with Yale Face Database B. Experimental results demonstrate the effectiveness of this approach in the performance of face recognition.

Depth information has been proven useful for face recognition. However, existing depth-image-based face recognition methods still suffer from noisy depth values and varying poses and expressions. In this paper, we propose a novel method for normalizing facial depth images to frontal pose and neutral expression and extracting robust features from the normalized depth images. The method is implemented via two deep convolutional neural networks (DCNN), normalization network (\(Net_{N}\)) and feature extraction network (\(Net_{F}\)). Given a facial depth image, \(Net_{N}\) first converts it to an HHA image, from which the 3D face is reconstructed via a DCNN. \(Net_{N}\) then generates a pose-and-expression normalized (PEN) depth image from the reconstructed 3D face. The PEN depth image is finally passed to \(Net_{F}\), which extracts a robust feature representation via another DCNN for face recognition. Our preliminary evaluation results demonstrate the superiority of the proposed method in recognizing faces of arbitrary poses and expressions with depth images.

A quantitative identification method based on CWT and CNN for external and inner broken wires of steel wire ropes

Deep learning (DL) algorithms, especially the convolutional neural network (CNN), have been proven as a newly developed tool in machinery intelligent diagnosis. However, the current CNN-based fault diagnosis studies usually consider features or images extracted from a single domain as model input. This single domain information may not reflect fault patterns comprehensively, leading to low modeling accuracy and inaccurate diagnostic results. To overcome this limitation, this paper proposes a new CNN-based fault diagnosis approach using image representation considering multi-domain features of vibration signals. First, multi-domain features of vibration signals are extracted. These extracted features are then used to construct a n × n matrix, and subsequently to form images by RGB color transformations. This image transformation technique allows for capturing complementary and rich diagnostic information from multiple domains. At last, these images associated with different mechanical defects are fed into a CNN model that is improved based on the classic LeNet-5 CNN architecture for fault diagnosis and identification. Comparative experiments with the traditional feature extraction methods as well as state-of-the-art CNN-based methods are also investigated. Experimental studies on rolling bearings validate the effectiveness and superiorities of the proposed approach.

Application of Face Recognition Method Under Deep Learning Algorithm in Embedded Systems

The increasing global population has escalated the demand for fish products. This calls for a stable supply which can only be met through improved aquaculture practices. The automatic recognition of fish diseases from diseased underwater images is one of such practices that aims to control disease, improve fish production and optimize profits. However, due to lack of public fish disease dataset, there has been limited research towards fish disease recognition. Moreover, due to low quality of underwater images and complex underwater environments, traditional hand-designed feature extraction methods or convolutional neural networks (CNNs)-based classifiers cannot adequately recognize fish diseases in real underwater scenes. Therefore, this paper proposes a novel hybrid approach based on multilayer fusion, attention mechanism and online sequential extreme learning machine (OSELM) to recognize fish diseases in aquaculture. We further compare the classification performance of the proposed model with baseline, attention-based, ConvNeXt and Swin transformer models. Feature extraction is enhanced by integrating same level features and focusing on salient features for fish disease recognition. The characteristic information of fish disease is refined by using strongly discriminative features of the infected fish regions and weakening regions of low interest using convolutional block attention module (CBAM). The module is added to the multilayer fusion network to sequentially infer attention maps along the channel and spatial dimensions for every intermediate feature map. Fish disease recognition is then done using OSELM for faster learning and improved classification performance. The models are trained, validated and tested on a custom dataset with image samples collected from various internet sources. The proposed method achieves 94.28% of accuracy, precision of 92.67%, recall of 92.17% and 92.42% of F1- score on dataset with background elimination. The proposed method can be used for fish disease identification in complex underwater environments in aquaculture.