On-Device Partial Learning Technique of Convolutional Neural Network for New Classes

Aiming at the problems of slow convergence and low recognition accuracy of the existing image big data recognition methods, a convolution neural network based image big data recognition method is proposed. In the concept of machine vision, image recognition refers to the ability of software to distinguish characters, positions, objects, actions and handwriting in pictures. Computers can apply machine vision techniques, combined with artificial intelligence and cameras to image recognition. Describes the convolution layer, pool layer, full connection layer and classification layer in the model. MNIST data sets are simulated to verify the superiority of the proposed method. Convolutional neural network has achieved good results in the field of image recognition, but its network structure has a great impact on the effect and efficiency of image recognition. In order to improve the recognition performance, a new convolution neural network structure is designed and implemented by repeated use of smaller convolution kernels. The results show that compared with other similar algorithms, the false recognition rate of this method is low, and the lower average square error of classification can be realized in a short time. This paper expounds the basic flow of digital recognition system from the aspects of data extraction and preprocessing, feature extraction and selection, classifier design and so on[1].

An attribution graph-based interpretable method for CNNs

The influencing factors of community risk are complex. For the low accuracy of traditional prediction model, a multichannel convolutional neural network community risk prediction model is proposed by improving convolutional neural network of deep learning. First of all, in the community risk prediction model, the structure of multichannel input convolutional neural network is selected. Then, add it into the full connection layer. Subsequently, the DenseNet layer is added to establish connections between different network layers. Finally, the receptive field is improved, and the gradient disappearance is solved. Thus, the prediction accuracy of model is improved. Compared with the traditional model, the proposed multichannel convolutional neural network model has better prediction accuracy. In addition, it performs better on the three indicators, namely, correlation coefficient R , coefficient of determination R 2 , and mean square root error RMSE. Compared with the commonly used LSTM model and logic regression model, the proposed model also has certain advantages, which is more suitable for community risk prediction.

Age estimation is a fascinating field of study because its implementation can be utilized in a variety of contexts. Currently, the Convolutional Neural Network (CNN) technique is frequently utilized to solve the age estimation problem. However, this method has one disadvantage, namely a high computational cost that lengthens the training process. This paper proposes an alternative method for resolving age estimation issues. To overcome the shortcomings of the backpropagation method, the proposed method employs the Extreme Learning Machine (ELM) algorithm as a fully-connected CNN layer. ResNet50 and VGG16 are the CNN architectures employed in this study. Using the UTKFace dataset, the proposed method was trained and evaluated. Using ELM as a fully-connected layer on CNN provided significantly faster training time performance than the fully-connected layer trained with the backpropagation technique. In addition, the proposed combination of CNN and ELM can provide age classification performance that is competitive with state-of-the-art method for age estimation cases.

The deep “Convolutional Neural Networks (CNNs)” gained a grand success on a broad of computer vision tasks. However, CNN structures training consumes a massive computing resources amount. The researchers in this field are concerned on designing CNN structures to maximize the performance and accuracy. The main design methods are human hand-crafted fixed model structures and automatic generated models. We proposed an automatic CNN structure design approach based on genetic algorithm that concerned with generating light weight CNN structures. We also introduce a chromosome novel representation for the structure of CNN. Unlike existing approaches, the proposed methodology is designed to work on limited computing assets with achieving high accuracy. It utilizes advanced training methods to decrease the overhead on the computing resources that are involved in the process. Our experimental results denote the proposed model effectiveness over the related work methods.

Due to the irregular and sporadic nature of wildfires, continuous monitoring of large areas is required. Since geostationary satellite sensors can observe large areas with high temporal resolution, they are suitable for monitoring wildfires in real time. However, the threshold algorithm currently employed for satellite-based active fire detection has poor performance in sensors with low spatial resolution. In addition, the algorithm does not account for environmental conditions that affect wildfire detection, resulting in poor generalization performance for large areas. This study examines the viability of an adaptive active fire detection model by combining satellite and numerical model data with deep learning. A model for active fire detection was developed using commonly employed brightness temperature-related variables (key variables) and local environmental variables (sub variables). Key variables are the cross spectral and spatial differences between the MIR (central wavelength of 3.85 m) and 2 TIR (central wavelengths of 9.63 and 11.20 m) channels of the Advanced Himawari Imager (AHI). Sub variables include Solar zenith angle (SOZ) and satellite zenith angle (SAZ) of AHI, skin temperature (ST) and relative humidity (RH) of European Centre for Medium-Range Weather Forecasts (ECMWF) Reanalysis v5 (ERA5)-land data. Four processes (confidence, frequency, land cover, and continuity tests) were used to extract reference fire samples from Moderate Resolution Imaging Spectroradiometer (MODIS) and Visible Infrared Imaging Radiometer Suite (VIIRS) active fire products. To consider the different properties of key and sub variables, a 2-way convolutional neural network (CNN) structure was developed. To evaluate the influence of environmental variables, a CNN model without sub variables was adopted as a control model. The 2-way CNN (recall of 0.86, precision of 0.96, and standard deviation of recall of 0.13) was more robust at five focus sites than the control CNN (recall of 0.82, precision of 0.97, and standard deviation of recall of 0.163). Despite having a lower spatial resolution than MODIS/VIIRS, 2-way CNN outperformed other satellite-based active fire products (MODIS, VIIRS, AHI, and Advanced Meteorological Imager) in terms of detection capacity. The control CNN demonstrated poor performance under certain environmental conditions (high RH, high SAZ, and transition time between day and night), but 2-way CNN mitigates this tendency. In particular, the use of RH improved detection sensitivity, and SAZ contributed to the spatial robustness. This study demonstrated the significance of environmental conditions in active fire detection and proposed a suitable CNN structure for this intent. Based on the findings of this study, higher-level adaptive active fire monitoring under diverse environmental conditions will be possible together with explainable artificial intelligence.

The automatic extraction of road network information from satellite images is a meaningful and challenging task. Particularly, the analysis of road surface materials is very important during transport construction and maintenance. This paper proposes a method to extract road area and identify its corresponding materials. The approach is based on two different convolutional neural network structures. Firstly, we use encoder-decoder symmetric network structure to extract the candidate road area. Then the former outputs is processed by atrous convolutional network with very deep layers, in order to classify the covered substances through their representative spectral features. We also utilize the physical characteristics of road network to design morphology approach to enhance the completeness and formation of the road network. Experiential results on various satellite images show that the method can yields better accuracy and adaptability than other convolutional network based methods.

In deep convolutional neural networks (DCNNs), model size and computation complexity are two important factors governing throughput and energy efficiency when deployed to hardware for inference. Recent works on compact DCNNs as well as pruning methods are effective, yet with drawbacks. For instance, more than half the size of all MobileNet models lies in their last two layers, mainly because compact separable convolution (CONV) layers are not applicable to their last fully connected (FC) layers. Also, in pruning methods, the compression is gained at the expense of irregularity in the DCNN architecture, which necessitates additional indexing memory to address nonzero weights, thereby increasing memory footprint, decompression delays, and energy consumption. In this article, we propose cyclic sparsely connected (CSC) architectures, with memory/computation complexity of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(N\log {}N)$ </tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N$ </tex-math></inline-formula> is the number of nodes/channels given a DCNN layer that, contrary to compact depthwise separable layers, can be used as an overlay for both FC and CONV layers of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(N^{2})$ </tex-math></inline-formula> . Also, contrary to pruning methods, CSC architectures are structurally sparse and require no indexing due to their cyclic nature. We show that both standard convolution and depthwise convolution layers are special cases of the CSC layers, whose mathematical function, along with FC layers, can be unified into one single formulation and whose hardware implementation can be carried out under one arithmetic logic component. We examine the efficacy of the CSC architectures for compression of LeNet, AlexNet, and MobileNet DCNNs with precision ranging from 2 to 32 bits. More specifically, we surge upon the compact 8-bit quantized 0.5 MobileNet V1 and show that by compressing its last two layers with CSC architectures, the model is compressed by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\sim 1.5\times $ </tex-math></inline-formula> with a size of only 873 kB and little accuracy loss. Finally, we design a configurable hardware that implements all types of DCNN layers including FC, CONV, depthwise, CSC-FC, and CSC-CONV indistinguishably within a unified pipeline. We implement the hardware on a tiny Xilinx field-programmable gate array (FPGA) for total on-chip processing of the compressed MobileNet that, compared to the related work, has the highest Inference/J while utilizing the smallest FPGA.

In order to identify pests quickly and accurately in agricultural environment, a deep learning model based on convolutional neural network is proposed to locate and identify pests. In this paper, by accelerating the color attenuation of the whole image, combining with the spatial influence between the hyperpixel regions, the saliency value of each super region is calculated, and then the potential area of the pest target is provided. Finally, the pest target location and segmentation are combined with grabcut algorithm. For the segmented pest target, the optimized convolution neural network is used to express and classify, and the structure of convolution neural network is further reduced.

Convolutional Neural Network (CNN) has achieved great success in various fields of machine vision, such as image classification and recognition, image segmentation, and video analysis. In specific applications, it is often necessary to customize the network structure by altering image size, convolution kernel size, pooling size, network architecture, and the number of network layers. These customizations pose significant challenges to the architecture of CNN accelerator especially in real-time systems, and as the number of these network variables increases, the internal convolution calculations become extremely demanding. The inference speed and energy consumption of CNN are becoming more and more important, which requires the design of the accelerator to adapt to different network architecture and run efficiently. Field-programmable gate arrays (FPGAs) is an ideal choice for a CNN accelerator due to its high programmability and low power consumption. To address the aforementioned challenges, we present NRUCA, a novel network reconfigurable universal CNN accelerator based on FPGA. On one hand, we have designed a flexible architecture that can dynamically configure CNN’s network structure parameters. These parameters can be sent to the FPGA via a configuration file, enabling the same design to run different CNN networks and allowing to modify on the fly. This forms the basis of the architecture’s adaptability presented in this paper. On the other hand, we employ a multi-channel parallel and efficient pipeline matrix multiplication architecture to implement the convolution and fully-connected layers, which constitute the majority of the computational load in CNNs. By utilizing an innovative inverted matrix multiplication algorithm and an interleaving cache data method, we reduce the internal cache required for multi-channel data. Furthermore, most of the intermediate calculation data does not need to be output to DDR, significantly improving the operation efficiency of the accelerator. We also fully leverage the FPGA chip architecture to compile multiple calculation kernels, which can be flexibly scheduled and combined to cater to various application scenarios. Based on the flexible and efficient overall design architecture, the layout of various resources of FPGA is balanced, which enables the designed FPGA project to compile four acceleration kernels with the running clock up to 300MHz in Xilinx Vitis and test them on Xilinx Alveo U250. We verified the adaptability and acceleration capabilities of our FPGA project using LeNet, AlexNet, VGG11, VGG13, VGG16, and VGG19 networks. Experiments demonstrated that our FPGA architecture achieves 33X and 35X energy savings compared to the Intel Xeon 5220R CPU, and 1.08X and 1.05X energy savings compared to the Nvidia Tesla P100 GPU, when accelerating AlexNet and VGG16, respectively.

Self-attention based models such as vision transformers (ViTs) have emerged as a very competitive architecture alternative to convolutional neural networks (CNNs) in computer vision. Despite increasingly stronger variants with ever higher recognition accuracies, due to the quadratic complexity of self-attention, existing ViTs are typically demanding in computation and model size. Although several successful design choices (e.g., the convolutions and hierarchical multi-stage structure) of prior CNNs have been reintroduced into recent ViTs, they are still not sufficient to meet the limited resource requirements of mobile devices. This motivates a very recent attempt to develop light ViTs based on the state-of-the-art MobileNet-v2, but still leaves a performance gap behind. In this work, pushing further along this under-studied direction we introduce EdgeViTs, a new family of light-weight ViTs that, for the first time, enable attention based vision models to compete with the best light-weight CNNs in the tradeoff between accuracy and on-device efficiency. This is realized by introducing a highly cost-effective local-global-local (LGL) information exchange bottleneck based on optimal integration of self-attention and convolutions. For device-dedicated evaluation, rather than relying on inaccurate proxies like the number of FLOPs or parameters, we adopt a practical approach of focusing directly on on-device latency and, for the first time, energy efficiency. Extensive experiments on image classification, object detection and semantic segmentation validate high efficiency of our EdgeViTs when compared to the state-of-the-art efficient CNNs and ViTs in terms of accuracy-efficiency tradeoff on mobile hardware. Specifically, we show that our models are Pareto-optimal when both accuracy-latency and accuracy-energy tradeoffs are considered, achieving strict dominance over other ViTs in almost all cases and competing with the most efficient CNNs. Code is available at https://github.com/saic-fi/edgevit .

Convolutional neural networks (CNNs) have become a useful tool for a wide range of applications such as text classification. However, CNNs are not always sufficiently accurate to be useful in certain applications. The selection of activation functions within CNN architecture can affect the efficacy of the CNN. However, there is limited research regarding which activation functions are best for CNN text classification. This study tested sixteen activation functions across three text classification datasets and six CNN structures, to determine the effects of activation function on accuracy, iterations to convergence, and Positive Confidence Difference (PCD). PCD is a novel metric introduced to compare how activation functions affected a network’s classification confidence. Tables were presented to compare the performance of the activation functions across the different CNN architectures and datasets. Top performing activation functions across the different tests included the symmetrical multi-state activation function, sigmoid, penalised hyperbolic tangent, and generalised swish. An activation function’s PCD was the most consistent evaluation metric during activation function assessment, implying a close relationship between activation functions and network confidence that has yet to be explored.

This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 203962, “Upscaling of Realistic Discrete Fracture Simulations Using Machine Learning,” by Nikolai Andrianov, SPE, Geological Survey of Denmark and Greenland, prepared for the 2021 SPE Reservoir Simulation Conference, Galveston, Texas, 4–6 October. The paper has not been peer reviewed. Upscaling of discrete fracture networks to continuum models such as the dual-porosity/dual-permeability (DP/DP) model is an industry-standard approach in modeling fractured reservoirs. In the complete paper, the author parametrizes the fine-scale fracture geometries and assesses the accuracy of several convolutional neural networks (CNNs) to learn the mapping between this parametrization and DP/DP model closures. The accuracy of the DP/DP results with the predicted model closures was assessed by a comparison with the corresponding fine-scale discrete fracture matrix (DFM) simulation of a two-phase flow in a realistic fracture geometry. The DP/DP results matched the DFM reference solution well. The DP/DP model also was significantly faster than DFM simulation. Introduction The goal of this study was to evaluate the effect of different CNN architectures on prediction accuracy for the DP/DP model closures and on the accuracy of DP/DP simulations in comparison with fine-scale DFM simulations. As a starting point, two CNN configurations were considered that have achieved breakthrough performance in image-classification tasks. The author adopted these architectures to the problem of learning the mapping between pixelated fracture geometries and the DP/DP model closures and indicated several key features in the CNN structure that are crucial for achieving high prediction accuracy. Mapping of fracture geometries requires significant effort, which limits the possibilities for creating large training data sets with realistic fracture geometries. The author, therefore, used the synthetic random linear fractures’ data set to train the CNNs and the fracture geometry from the Lägerdorf outcrop for testing purposes. It was demonstrated that an optimal CNN configuration yielded the DP/DP model closures such that the corresponding DP/DP results matched well the two-phase DFM simulations on a subset of the Lägerdorf data. The run times for the DP/DP model were a fraction of the time needed to accomplish the DFM simulations. Problem formulation is presented in a series of equations in the complete paper.

Recent years, convolutional neural network (CNN) is a research hot spot in machine learning and has some application value in computer aided diagnosis. Firstly, this paper briefly introduces the basic principle of CNN. Secondly, it summarizes the improvement on network structure from two dimensions of model and structure optimization. In model structure, it summarizes eleven classical models about CNN in the past 60 years, and introduces its development process according to timeline. In structure optimization, the research progress is summarized from five aspects (input layer, convolution layer, down-sampling layer, full-connected layer and the whole network) of CNN. Thirdly, the learning algorithm is summarized from the optimization algorithm and fusion algorithm. In optimization algorithm, it combs the progress of the algorithm according to optimization purpose. In algorithm fusion, the improvement is summarized from five angles: input layer, convolution layer, down-sampling layer, full-connected layer and output layer. Finally, CNN is mapped into the medical image domain, and it is combined with computer aided diagnosis to explore its application in medical images. It is a good summary for CNN and has positive significance for the development of CNN.

In this article, we consider the problem of human head movement recognition (HMR) using linear frequency-modulated continuous-wave (LFMCW) radar. The multidomain fusion network is proposed to improve the HMR performance, which extracts and fuses the multidomain features from the range and time–frequency (TF) domain. Specifically, the 2-D convolutional neural network (2D-CNN) structure is applied to extract range domain features, and the 3-D convolutional neural network (3D-CNN) structure is designed to extract multitype TF representation (TFR) plots features cooperatively. After that, the learnable convolution weights are used for the adaptive fusion of multidomain features. In addition, the attention mechanism is employed to remove redundant information in the fused features. Finally, experimental results on real data verify the effectiveness of the proposed method.