MGIC: Multigrid-in-Channels Neural Network Architectures

Wildlife is an important biological resource in China. Classifying images of wildlife through computer technology can help people identify wildlife, which is of great significance to help people understand and protect wildlife. Therefore, this issue is worth studying. Traditional methods mostly use standard Convolutional Neural Networks (CNN) to classify wild animal images, but these methods have disadvantages such as slow computing speed, long time consumption and low accuracy. With an attempt to address such issues, this paper proposes a method based on transfer-learning for classifying wild animal images. By using the pre-trained model it can save a lot of training time. The experimental results on Oregon Wildlife, using a public wildlife data set, show that the method proposed in this paper achieved 99.01% accuracy and is 57.82% more accurate than the standard Convolutional Neural Networks (CNN) method. Moreover, in terms of running time, the method presented in this paper has achieved higher efficiency, and the training time is 50% shorter than the standard method, which proves the superiority of the method proposed in this paper.

Unmanned aerial vehicles (UAVs) are invaluable technologies concerning their remote control and monitoring capabilities. Convolutional neural networks (CNNs), known for their high pattern recognition capabilities, are appropriate for forest fire detection with UAVs. Deep convolutional neural networks show substantial performance on hardware with high processing capabilities. While these networks can be operated in unmanned aerial vehicles controlled from ground control stations equipped with GPU-supported hardware, the execution on a typical UAV’s limited computational resources necessitates the use of lightweight, small-sized networks. To overcome these impediments, this article presents a lightweight and attention-based approach for performing forest fire detection tasks using UAV vision data (images acquired by cameras mounted on UAVs). In this paper, we also present comprehensive research for different approaches such as transfer learning, deep CNNs, and lightweight CNNs. Among the experimented models, the attention-based EfficientNetB0 backboned model emerged as the most successful architecture for forest fire detection. With the test accuracy of , the F1-score of , the recall of , and the precision of have strongly reinforced the efficiency of the EfficientNetB0-based model in wildfire recognition. Moreover, the network has a less parameter size than the experimented networks. It proves the model’s suitability for wildfire detection with UAVs having limited hardware resources.

Breast cancer is the dominant cancer among women as it accounts for about one-quarter of all cancer cases in females. The digitized images of Hematoxylin and Eosin (H&E) stained slides of breast cancer specimens carry valuable diagnostic information. However, inspecting these slides manually is a non-trivial task prone to subjective interpretation. Digital pathology (DP) and artificial intelligence (AI) open an opportunity for objective interpretation of the image data. It is challenging to automate the segmentation process in the whole slide images due to the visual complexity of tissue appearance without the need for tedious and time-consuming fine annotations. Many algorithms classify the tissue regions into different types instead of segmenting them, as the classification algorithms require coarse annotations that are easier to acquire. In this paper, we propose a new segmentation framework that combines the simple non-iterative clustering algorithm with a standard convolutional neural network (CNN) classifier to segment whole slide images into different tissue types. In addition, a graph-based post-processing step is applied to improve the framework segmentation performance further. The results show promising improvement to the CNN classifier based coarse segmentation, which would give better feasibility to quantify and study tissues’ mutual relationships.

Texture analysis has many important applications, including material recognition, face recognition, object detection, image segmentation. Local feature descriptors were the principle approach for texture analysis in the past. Recently, Convolutional Neural Network (CNN) has provided more promising results for texture recognition and other related computer vision tasks. Standard CNN takes labeled RGB images as input. However, other encoded images were used as extra input to CNN, which have been shown to improve the performance. We propose to feed CNN with the new encoded image. The experimental results on four benchmark color texture database show the efficiency of our proposed approach. The source code of our algorithm and all simulations used for this paper are publicly available at: https://sites.google.com/view/vinhsiam/codes.

As urbanization accelerates, the prevalence of fire incidents leads to significant hazards. Enhancing the accuracy of remote fire detection systems while reducing computation complexity and power consumption in edge hardware are crucial. Therefore, this paper investigates an innovative lightweight Convolutional Spiking Neural Network (CSNN) method for fire detection based on acoustics. In this model, Poisson encoder and convolution encoder strategies are considered and compared. Additionally, the study investigates the impact of observation time steps, surrogate gradient functions, and the threshold and decay rate of membrane potential on network performance. A comparison is made between the classification metrics of the traditional Convolutional Neural Network (CNN) approaches and the proposed lightweight CSNN method. To assess the generalization performance of the proposed lightweight method, publicly available datasets are merged with our experimental data for training, which results in a high accuracy of 99.02%, a precision of 99.37%, a recall of 98.75%, and an F1 score of 99.06% on the test datasets.

Self-attention based models such as vision transformers (ViTs) have emerged as a very competitive architecture alternative to convolutional neural networks (CNNs) in computer vision. Despite increasingly stronger variants with ever higher recognition accuracies, due to the quadratic complexity of self-attention, existing ViTs are typically demanding in computation and model size. Although several successful design choices (e.g., the convolutions and hierarchical multi-stage structure) of prior CNNs have been reintroduced into recent ViTs, they are still not sufficient to meet the limited resource requirements of mobile devices. This motivates a very recent attempt to develop light ViTs based on the state-of-the-art MobileNet-v2, but still leaves a performance gap behind. In this work, pushing further along this under-studied direction we introduce EdgeViTs, a new family of light-weight ViTs that, for the first time, enable attention based vision models to compete with the best light-weight CNNs in the tradeoff between accuracy and on-device efficiency. This is realized by introducing a highly cost-effective local-global-local (LGL) information exchange bottleneck based on optimal integration of self-attention and convolutions. For device-dedicated evaluation, rather than relying on inaccurate proxies like the number of FLOPs or parameters, we adopt a practical approach of focusing directly on on-device latency and, for the first time, energy efficiency. Extensive experiments on image classification, object detection and semantic segmentation validate high efficiency of our EdgeViTs when compared to the state-of-the-art efficient CNNs and ViTs in terms of accuracy-efficiency tradeoff on mobile hardware. Specifically, we show that our models are Pareto-optimal when both accuracy-latency and accuracy-energy tradeoffs are considered, achieving strict dominance over other ViTs in almost all cases and competing with the most efficient CNNs. Code is available at https://github.com/saic-fi/edgevit .

Abstract. Crowd counting, a critical component in the management and safety planning of large gatherings and public spaces, is essential for ensuring smooth event operations and preventing potential overcrowding issues. While the standard convolutional neural network (CNN) based model performs well in head counting tasks, it has certain drawbacks when applied to complex scenarios. With the rapid development of artificial intelligence, Transformer models that rely on self-attention mechanisms, as Swin Transformer, have demonstrated exceptional performance in visual tasks, such as image classification, and segmentation in recent times. This study examines the experimental findings of Swin Transformer's head counting tasks and contrasts them with the CNN-based model. Mean Absolute Error (MAE) and Mean Square Error (MSE) evaluation indicators show that the Transformer model outperforms the classic CNN model in terms of generalization abilities when dealing with complicated scenarios. Future research work will increase the diversity of data sets and focus on optimizing model structure and improving training efficiency.

An explainable deep-learning model to stage sleep states in children and propose novel EEG-related patterns in sleep apnea

Standard convolutional neural networks (CNNs) have large amounts of data redundancy, and the same accuracy can be obtained even in lower bit weights instead of floating-point representation. Most CNNs have to be developed and executed on high-end GPU-based workstations, for which it is hard to transplant the existing implementations onto portable edge FPGAs because of the limitation of on-chip block memory storage size and battery capacity. In this paper, we present adaptive pointwise convolution and 2D convolution joint network (AP2D-Net), an ultra-low power and relatively high throughput system combined with dynamic precision weights and activation. Our system has high performance, and we make a trade-off between accuracy and power efficiency by adopting unmanned aerial vehicle (UAV) object detection scenarios. We evaluate our system on the Zynq UltraScale+ MPSoC Ultra96 mobile FPGA platform. The target board can get the real-time speed of 30 fps under 5.6 W, and the FPGA on-chip power is only 0.6 W. The power efficiency of our system is 2.8× better than the best system design on a Jetson TX2 GPU and 1.9× better than the design on a PYNQ-Z1 SoC FPGA.

Deep Learning has shown great success in many fields, however, transferring this potential to remote sensing imagery interpretation is still a challenging task due to the special data properties, e.g., low Signal-to-Noise Ratio (SNR), high variation, etc. In this work, a lifting scheme based deep model is presented for remote sensing imagery classification. The main idea underlying this scheme is that, an innovative strategy is adopted to decompose the input image into two compact and low-resolution components, and these components are then fed into a standard Convolutional Neural Network (CNN) for classification task. More precisely, (1) one decomposed component is devoted to enhancing the latent patterns and simultaneously attenuating the random variation in the input, and (2) the other component is used to capture the local structural information in the input. The experimental results show that the lifting deep model is computationally efficient and has promising potential, improving the classification accuracy by about 5.7% and obtaining 2.69 x speed-up compared with the counterpart CNN.

We introduce a Convolutional Neural Network (CNN) to post-process recordings obtained by means of Background Oriented Schlieren (BOS), a popular technique to visualize compressible/convective flows. To reconstruct BOS image deformation, we devised a lightweight network (LIMA) that has comparatively fewer parameters to train, allowing the deployment of the network on embedded GPU hardware. To train the CNN, we introduce a novel strategy based on the generation of synthetic images with random, irrotational displacement field that mimic those provided by real BOS recording. This allows us to generate a large number of training examples at minimal computational cost. To assess the accuracy of the reconstructed displacement, we consider test cases consisting of synthetic images with sinusoidal displacement as well as images obtained in a real experimental study of flow past and inside a heated hollow hemisphere. By comparing the prediction of the CNN with conventional post-processing techniques such as Direct Image Correlation (DIC) and Particle Image Velocimetry (PIV) cross-correlation, we show that LIMA gives more accurate and robust results for the synthetic example. When applied to the recordings from the real experiment, all methods provide consistent deformation fields. As they offer similar or better accuracy at a fraction of the computational costs, properly designed CNNs offer a valuable alternative to conventional post-processing techniques for BOS experiments.

Deep convolutional neural networks (CNNs) have seen significant growth in medical image classification applications due to their ability to automate feature extraction, leverage hierarchical learning, and deliver high classification accuracy. However, Deep CNNs require substantial computational power and memory, particularly for large datasets and complex architectures. Additionally, optimising the hyperparameters of deep CNNs, although critical for enhancing model performance, is challenging due to the high computational costs involved, making it difficult without access to high-performance computing resources. To address these limitations, this study presents a fast and efficient model that aims to achieve superior classification performance compared to popular Deep CNNs by developing lightweight CNNs combined with the Nonlinear Lévy chaotic moth flame optimiser (NLCMFO) for automatic hyperparameter optimisation. NLCMFO integrates the Lévy flight, chaotic parameters, and nonlinear control mechanisms to enhance the exploration capabilities of the Moth Flame Optimiser during the search phase while also leveraging the Lévy flight theorem to improve the exploitation phase. To assess the efficiency of the proposed model, empirical analyses were performed using a dataset of 2314 brain tumour detection images (1245 images of brain tumours and 1069 normal brain images). The evaluation results indicate that the CNN_NLCMFO outperformed a non-optimised CNN by 5% (92.40% accuracy) and surpassed established models such as DarkNet19 (96.41%), EfficientNetB0 (96.32%), Xception (96.41%), ResNet101 (92.15%), and InceptionResNetV2 (95.63%) by margins ranging from 1 to 5.25%. The findings demonstrate that the lightweight CNN combined with NLCMFO provides a computationally efficient yet highly accurate solution for medical image classification, addressing the challenges associated with traditional deep CNNs.

Recently, many researches have been conducted on recognition of facial emotion using convolutional neural networks (CNNs), which show excellent performance in computer vision. To obtain a high classification accuracy, a CNN architecture with many parameters and high computational complexity is required. However, this is not suitable for embedded systems where hardware resources are limited. In this paper, we present a lightweight CNN architecture optimized for embedded systems. The proposed CNN architecture has a small memory footprint and low computational complexity. Furthermore, a novel hardware-friendly quantization method that uses only integer-arithmetic is proposed. The proposed hardware-friendly quantization method maps the scale factors to power-of-two terms and replaces multiplication and division operations using scale factors with shift operations. To improve the generalization and classification performance of the CNN, we create the FERPlus-A dataset. This is a new training dataset created using a variety of image processing algorithms. After training with FERPlus-A, quantization has been performed. The size of a quantized CNN parameter is about 0.39 MB, and the number of operations is about 28 M integer operations (IOPs). By evaluating the performance of the quantized CNN that uses only integer-arithmetic on the FERPlus test dataset, the classification accuracy is approximately 86.58%. It achieved higher accuracy than other lightweight CNNs in prior studies. The proposed CNN architecture that uses only integer-arithmetic is implemented on the Xilinx ZC706 SoC platform for real-time facial emotion recognition by applying parallelism strategies and efficient data caching strategies. The FPGA-based CNN accelerator implemented for real-time facial emotion recognition achieves about 10 frame per second (FPS) at 250 MHz and consumes 2.3 W.

The key to preventing the COVID-19 is to diagnose patients quickly and accurately. Studies have shown that using Convolutional Neural Networks (CNN) to analyze chest Computed Tomography (CT) images is helpful for timely COVID-19 diagnosis. However, personal privacy issues, public chest CT data sets are relatively few, which has limited CNN's application to COVID-19 diagnosis. Also, many CNNs have complex structures and massive parameters. Even if equipped with the dedicated Graphics Processing Unit (GPU) for acceleration, it still takes a long time, which is not conductive to widespread application. To solve above problems, this paper proposes a lightweight CNN classification model based on transfer learning. Use the lightweight CNN MobileNetV2 as the backbone of the model to solve the shortage of hardware resources and computing power. In order to alleviate the problem of model overfitting caused by insufficient data set, transfer learning is used to train the model. The study first exploits the weight parameters trained on the ImageNet database to initialize the MobileNetV2 network, and then retrain the model based on the CT image data set provided by Kaggle. Experimental results on a computer equipped only with the Central Processing Unit (CPU) show that it consumes only 1.06 s on average to diagnose a chest CT image. Compared to other lightweight models, the proposed model has a higher classification accuracy and reliability while having a lightweight architecture and few parameters, which can be easily applied to computers without GPU acceleration. Code:github.com/ZhouJie-520/paper-codes.

Convolutional neural networks (CNNs) have achieved significant accuracy improvement in many intelligent applications at the cost of intensive convolution operations and massive data movements. To efficiently deploy CNNs on low power embedded platforms in real time, the depthwise separable convolution has been proposed to replace the standard convolution, especially in lightweight CNNs, which remarkably reduces the computation complexity and model size. However, it is difficult for a general convolution engine to obtain the theoretical performance improvement as the decreased data dependency of depthwise convolution significantly reduces the data reuse opportunity. To address this issue, a flexible and highperformance accelerator based on FPGA is proposed to efficiently process the inference of both large-scale and lightweight CNNs. Firstly, by sharing the activation dataflow between the depthwise convolution and pooling layers, the control logic and data bus of the two layers are reused to maximize the data utilization and minimize the logic overhead. Furthermore, these two layers can be processed either directly after standard convolutions to eliminate the external memory accesses or independently to gain better flexibility. Thirdly, a performance model is proposed to automatically explore the optimal design options of the accelerator. The proposed hardware accelerator is evaluated on Intel Arria 10 SoC FPGA and demonstrates state-of-the-art performance on both large-scale CNNs, e.g., VGG, and lightweight ones, e.g., MobileNet.