Efficient densely connected convolutional neural networks

Over the last years, Convolutional Neural Networks (CNNs) have been widely used in remote sensing applications, such as marine surveillance, traffic management or road networks detection. However, since CNNs have extremely high computation, bandwith and memory requirements, the hardware implementation of a CNN on space-grade devices like FPGAs for the on-board processing of the acquired images has brought many challenges. In fact, FPGAs show great advantages in terms of reconfigurability and performance-power ratio, whichmakes them a suitable election to deploy CNNs, although the computational capabilities of this type of hardware devices available on-board is limited, so implementations have to be carefully planned. In this paper, the authors present their work towards the implementation of an efficient CNN onto a spacegrade FPGA in order to achieve the on-board processing of very-high resolution remotely sensed images as soon as the data are provided by the sensor. All this work has been conducted within the EU-funded VIDEO project. As it will be presented in this paper, the work includes the introduction of a methodology based on the project constraints, the evaluation of different state-of-the-art CNN architectures by means of a new efficiency measurement also proposed in this work, the introduction of a new efficient CNN architecture, and finally, its optimized hardware implementation by means of high-level synthesis tools.

Self-attention based models such as vision transformers (ViTs) have emerged as a very competitive architecture alternative to convolutional neural networks (CNNs) in computer vision. Despite increasingly stronger variants with ever higher recognition accuracies, due to the quadratic complexity of self-attention, existing ViTs are typically demanding in computation and model size. Although several successful design choices (e.g., the convolutions and hierarchical multi-stage structure) of prior CNNs have been reintroduced into recent ViTs, they are still not sufficient to meet the limited resource requirements of mobile devices. This motivates a very recent attempt to develop light ViTs based on the state-of-the-art MobileNet-v2, but still leaves a performance gap behind. In this work, pushing further along this under-studied direction we introduce EdgeViTs, a new family of light-weight ViTs that, for the first time, enable attention based vision models to compete with the best light-weight CNNs in the tradeoff between accuracy and on-device efficiency. This is realized by introducing a highly cost-effective local-global-local (LGL) information exchange bottleneck based on optimal integration of self-attention and convolutions. For device-dedicated evaluation, rather than relying on inaccurate proxies like the number of FLOPs or parameters, we adopt a practical approach of focusing directly on on-device latency and, for the first time, energy efficiency. Extensive experiments on image classification, object detection and semantic segmentation validate high efficiency of our EdgeViTs when compared to the state-of-the-art efficient CNNs and ViTs in terms of accuracy-efficiency tradeoff on mobile hardware. Specifically, we show that our models are Pareto-optimal when both accuracy-latency and accuracy-energy tradeoffs are considered, achieving strict dominance over other ViTs in almost all cases and competing with the most efficient CNNs. Code is available at https://github.com/saic-fi/edgevit .

Over the past decade, deep-learning-based representations have demonstrated remarkable performance in academia and industry. The learning capability of convolutional neural networks (CNNs) originates from a combination of various feature extraction layers that fully utilize a large amount of data. However, they often require substantial computation and memory resources while replacing traditional hand-engineered features in existing systems. In this review, to improve the efficiency of deep learning research, we focus on three aspects: quantized/binarized models, optimized architectures, and resource-constrained systems. Recent advances in light-weight deep learning models and network architecture search (NAS) algorithms are reviewed, starting with simplified layers and efficient convolution and including new architectural design and optimization. In addition, several practical applications of efficient CNNs have been investigated using various types of hardware architectures and platforms.

Semantic segmentation is increasingly being applied on mobile devices due to advancements in mobile chipsets, particularly in low-power consumption scenarios. However, the lightweight design of mobile devices poses limitations on the receptive field, which is crucial for dense prediction problems. Existing approaches have attempted to balance lightweight designs and high accuracy by downsampling features in the backbone. However, this downsampling may result in the loss of local details at each network stage. To address this challenge, this paper presents a novel solution in the form of a compact and efficient convolutional neural network (CNN) for real-time applications: our proposed model, local spatial perception convolution (LSPConv). Furthermore, the effectiveness of our architecture is demonstrated on the Cityscapes dataset. The results show that our model achieves an impressive balance between accuracy and inference speed. Specifically, our LightSeg, which does not rely on ImageNet pretraining, achieves an mIoU of 76.1 at a speed of 61 FPS on the Cityscapes validation set, utilizing an RTX 2080Ti GPU with mixed precision. Additionally, it achieves a speed of 115.7 FPS on the Jetson NX with int8 precision.

Intrusion detection plays a critical role in safeguarding computer networks from malicious activities and unauthorized access. Traditional machine learning (ML) models have been widely employed for intrusion detection, but their effectiveness can be limited due to the complex and evolving nature of network attacks. An efficient Convolutional Neural Network (CNN) that utilizes one-dimensional convolutions, batch normalization, and max pooling layers is designed to capture the attack patterns in network traffic data. By leveraging the inherent feature extraction capabilities of CNNs, the proposed architecture demonstrates improved performance in identifying network intrusions compared to traditional ML models. The effectiveness of the novel CNN architecture is evaluated by conducted comprehensive experiments on two benchmark intrusion detection datasets. The designed CNN performance was compared with four other ML models such as logistic regression, k-nearest neighbors, decision trees, and random forest. The experimental results revealed that the proposed CNN architecture outperformed the traditional ML models with highest accuracy of 97.68% and 97.39% and highest F1-score of 97.68% and 97.40% for. UNSW-NB15 and CICIDS2017 datasets, respectively. The proposed CNN architecture serves as a promising alternative to traditional ML models, offering improved performance in identifying and mitigating network attacks.

Convolutional neural networks have demonstrated state-of-the-art performance in image classification and various other computer vision tasks. Plant disease detection is an important area of deep learning which has been addressed by many recent methods. However, there is a dire need to optimize these solutions for resource-constrained portable devices such as smartphones. This is a challenging problem because deep learning models are resource extensive in nature. This paper proposes an efficient method to systematically classify plant disease symptoms using convolutional neural networks. These networks are memory efficient and when coupled with the proposed training configuration it enables rapid development of industrial applications by reducing the training times. Another critical problem arises with the improper distribution of samples among classes known as the class imbalance problem, which is addressed by employing a simple statistical methodology. Transfer learning is a known technique for training small datasets which transfers pre-trained weights learned on a large dataset. However, during transfer learning, negative transfer learning is a common problem. Therefore, a stepwise transfer learning approach is proposed which can help in fast convergence, while reducing overfitting and preventing negative transfer learning during knowledge transfer across domains. The system is trained and evaluated on two plant disease datasets i.e., PlantVillage (a publicly available dataset) and pepper disease dataset provided by the National Institute of Horticultural and Herbal Science, Republic of Korea. The pepper dataset is particularly challenging as it contains images from different parts of the plant such as the leaf, pulp, and stem. The proposed system has dominated the previous works on the PlantVillage dataset and achieved 99% and 99.69% accuracy on the Pepper dataset and PlantVillage datasets, respectively.

Character recognition and classification is considered one of the most important parts of current LPR systems. Because of low recognition quality and poor robustness of traditional character recognition techniques, those techniques were gradually replaced by powerful deep learning modules such as convolutional neural networks. Convolutional neural networks (CNNs) show satisfying ability in character recognition and outperform most of other available models. Since Chinese license plates contain Chinese characters in addition to ordinary alphanumeric characters, a robust, powerful, and efficient CNN is needed to accomplish character recognition task efficiently. In this paper, we have proposed an efficient CNN model based on Darknet architecture to perform character recognition. Through convolutional and max pooling layers, features of input character images will be extracted and then sent to softmax layer for classification. To avoid overfitting problem in the training process, the dropout regularization technique is adopted. We have used a dataset of 84,000 character images for training and testing our model. The experimental result shows satisfactory outputs and eventually achieves test accuracy of 99.69%.

Color in any image playing a vital role for better understanding and real image visualization. In past when there was no concept of colored images each black and white image need to highlight its own merits to explain an image in a better way, but when the concept of colored images started the things were totally changed to describe and highlight key features and attributes of an image in a more better and effective manner. The key issue is that the old images and videos are color less and in some cases it needs to be colorized. There are a number of applications are uses to colorized a gray-scale image. The authors of this paper also presented an efficient deep learning based image colorization technique. The proposed system maps all the gray contrast pixels of the image into its corresponding colored pixels to produce a colored image using Convolutional Neural Network (CNN) technique. The proposed system is totally an automated system and it avoid any manual work or user hand-define rules. The system first read the gray contrast of each pixel of image; matches function then create the corresponding color contrast from the color map and finally convert the gray contrast image into color image. The system tested around 100 images and found accuracy about 95%.

This study is done to enable real-time diagnosis of retinal diseases from fundus images using a hardwarecompatible and efficient convolutional neural network (CNN) architecture. A light-weighted convolutional neural network (CNN) is embedded in the proposed system for on-chip inference and fixed-point computation based on concepts from very largescale integration (VLSI). The model minimizes latency and energy consumption through memory-efficient buffering, activation layer quantization, and depth wise separable convolutions. The system classifies objects at 96.21 % accuracy while taking up just <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.92 ~\text{mm}^{2}$</tex> of silicon area and only 78.4 mW of power, after training and testing using the APTOS 2019 Blindness Detection dataset. The suggested architecture performs better than traditional lightweight CNNs in comparison studies measuring inference accuracy and hardware efficiency. A portable and power-friendly early-stage retinal disease diagnostic instrument has become a reality because of the deploy ability of the entire architecture on FPGA platforms.

In recent years, convolutional neural networks (CNNs) have played a dominant role in facial expression recognition. While CNN-based methods have achieved remarkable success, they are notorious for having an excessive number of parameters, and they rely on a large amount of manually annotated data. To address this challenge, we expand the number of training samples by learning expressions from a face recognition dataset to reduce the impact of a small number of samples on the network training. In the proposed deep joint learning framework, the deep features of the face recognition dataset are clustered, and simultaneously, the parameters of an efficient CNN are learned, thereby marking the data for network training automatically and efficiently. Specifically, first, we develop a new efficient CNN based on the proposed affinity convolution module with much lower computational overhead for deep feature learning and expression classification. Then, we develop an expression-guided deep facial clustering approach to cluster the deep features and generate abundant expression labels from the face recognition dataset. Finally, the AC-based CNN is fine-tuned using an updated training set and a combined loss function. Our framework is evaluated on several challenging facial expression recognition datasets as well as a self-collected dataset. In the context of facial expression recognition applied to the field of education, our proposed method achieved an impressive accuracy of 95.87% on the self-collected dataset, surpassing other existing methods.

In recent years, Convolution Neural Network (CNN) gained great success in many applications, especially in computer vision. Now adapting CNN inference on edge devices has become an active research area in embedded vision and hot topic in Edge AI. The primary design hurdles for implementing CNN inference on embedded systems are limited computational resource, memory resource, and power budget. This study presents a novel architecture for SqueezeNet [1] like CNN models, which can be extended to support any CNN model as well. We address two approaches to mitigate resource constraints. First, we use a custom floating point(12 bit for computation and 8bit for storing). Second is slicing the model into a repetitive block called computation blocks. Computation block can be configured dynamically by the host processor to operate in a different mode. We have implemented SqueezeNet v1.1 for Image-Net [2] for large-scale classification which achieved around 9 FPS at 100MHz. SqueezeNet, mapped for our architecture achieves top-1 accuracy of 51% for the ImageNet dataset. Unlike other implementations which use FPGA boards with a large amount of resources, our experiments are done in DE10 Nano which mimics actual embedded system like environment.

Due to the simple design pipeline, end-to-end (E2E) neural models for speech enhancement (SE) have attracted great interest. In order to improve the performance of the E2E model, the local and sequential properties of speech should be efficiently taken into account when modelling. However, in most current E2E models for SE, these properties are either not fully considered or are too complex to be realized. In this letter, we propose an efficient E2E SE model, termed WaveCRN. Compared with models based on convolutional neural networks (CNN) or long short-term memory (LSTM), WaveCRN uses a CNN module to capture the speech locality features and a stacked simple recurrent units (SRU) module to model the sequential property of the locality features. Different from conventional recurrent neural networks and LSTM, SRU can be efficiently parallelized in calculation, with even fewer model parameters. In order to more effectively suppress noise components in the noisy speech, we derive a novel restricted feature masking approach, which performs enhancement on the feature maps in the hidden layers; this is different from the approaches that apply the estimated ratio mask to the noisy spectral features, which is commonly used in speech separation methods. Experimental results on speech denoising and compressed speech restoration tasks confirm that with the SRU and the restricted feature map, WaveCRN performs comparably to other state-of-the-art approaches with notably reduced model complexity and inference time.

In modern manufacturing industry featured with automation and flexibility, the intelligent tool management for Computer Numeric Control (CNC) machine plays an essential role in manufacturing automation. The automatic tool recognition in terms of geometric shapes, materials and usage functions could facilitate the seamless integration with downstream process planning and scheduling processes. In this paper, a intelligent tool recognition system is proposed with a novel hybrid framework of multi-channel deep learning network with non-iterative and fast feedforward neural network to meet high efficiency and accuracy requirement in intelligent manufacturing. The combination of the fine-tuning Convolutional Neural Networks (CNNs) with the random parameter assignment mechanism of Extreme Learning Machines (ELMs) reach a balance in accurate feature extraction and fast recognition. In the proposed hybrid framework, features extracted from efficient CNNs are aggregated into robust ELM auto-encoders (ELM-AEs) to generate the compact but rich feature information, which are then feed to the subsequent single layer ELM network for tool recognition. The performance of proposed framework is verified on several standardized 3D shape retrieval and classification dataset, as well as on a self-constructed multi-view 3D data represented tool library database. Numerical experiments reveal a promising application perspective of proposed intelligent recognition system on manufacturing automation.

Mild cognitive impairment (MCI) represents the intermediate stage between normal cerebral aging and dementia associated with Alzheimer's disease (AD). Early diagnosis of MCI and AD through artificial intelligence has captured considerable scholarly interest; researchers hope to develop therapies capable of slowing or halting these processes. We developed a state-of-the-art deep learning algorithm based on an optimized convolutional neural network (CNN) topology called MCADNNet that simultaneously recognizes MCI, AD, and normally aging brains in adults over the age of 75 years, using structural and functional magnetic resonance imaging (fMRI) data. Following highly detailed preprocessing, four-dimensional (4D) fMRI and 3D MRI were decomposed to create 2D images using a lossless transformation, which enables maximum preservation of data details. The samples were shuffled and subject-level training and testing datasets were completely independent. The optimized MCADNNet was trained and extracted invariant and hierarchical features through convolutional layers followed by multi-classification in the last layer using a softmax layer. A decision-making algorithm was also designed to stabilize the outcome of the trained models. To measure the performance of classification, the accuracy rates for various pipelines were calculated before and after applying the decision-making algorithm. Accuracy rates of 99.77% 0.36% and 97.5% 1.16% were achieved for MRI and fMRI pipelines, respectively, after applying the decision-making algorithm. In conclusion, a cutting-edge and optimized topology called MCADNNet was designed and preceded a preprocessing pipeline; this was followed by a decision-making step that yielded the highest performance achieved for simultaneous classification of the three cohorts examined.

The Hyperspectral Images (HSI) are now being widely popular due to the evolution of satellite imagery and camera technology. Remote sensing has also gained popularity and it is also closely related to HSI. HSI possesses a wide variety of spatial and spectral features. However, HSI also has a consider-able amount of useless or redundant data. This redundant data causes a lot of trouble during classifications as it possesses a huge range in contrast to RGB. Traditional classification techniques do not apply efficiently to HSI. Even if somehow the traditional techniques are applied to it, the results produced are inefficient and undesirable. The Convolutional Neural Network (CNN), which are widely famous for the classification of images, have their fair share of trouble when dealing with HSI. 2D CNNs is not very efficient and 3D CNNs increases the computational complexity. To overcome these issues a new hybrid CNN approach is used which uses sigmoid activation function at the output layer, using a 2D CNN with 3D CNN to generate the desired output. Here, we are using HSI classification using hybrid CNN i.e., 2D and 3D. The dataset used is the Indian pines dataset sigmoid classifier for classification. And we gain the Overall accuracy 99.34 %, average accuracy 99.27%, kappa 99.25%.