LCDet: Low-Complexity Fully-Convolutional Neural Networks for Object Detection in Embedded Systems

Object detection is a fundamental task in computer vision, which is usually based on convolutional neural networks (CNNs). While it is difficult to be deployed in embedded devices due to the huge storage and computing consumptions, binary neural networks (BNNs) can execute object detection with limited resources. However, the extreme quantification in BNN causes diversity of feature representation loss, which eventually influences the object detection performance. In this paper, we propose a method balancing Information Retention and Deviation Control to achieve effective object detection, named IR-DC Net. On the one hand, we introduce the KL-Divergence to compose multiple entropy for maximizing the available information. On the other hand, we design a lightweight convolutional module to generate scale factors dynamically for minimizing the deviation between binary and real convolution. The experiments on PASCAL VOC, COCO2014, KITTI, and VisDrone datasets show that our method improved the accuracy in comparison with previous binary neural networks.

In the recent generations, object detection (OD) has played an important role ranging from navigation of autonomous vehicles to video data compression. In the area of OD, deep learning (DL) has played a vital role in detecting the objects automatically. Breakthroughs have been achieved in the field of object detection with the simultaneous advancements in the field of DL. In this article, the author proposes a region-based neural network for detecting desired object in which it overcomes the computational complexity acquired by convolutional neural networks and provides a better accuracy. For training larger datasets and reusability, the deep learning principles are also proposed.

Detecting objects from images is a challenging problem in the domain of computer vision and plays a very crucial role for wide range of real-time applications. The ever-increasing growth of deep learning due to availability of large training data and powerful GPUs helped computer vision community to build commercial products and services which were not possible a decade ago. Deep learning architectures especially convolutional neural networks have achieved state-of-the-art performance on worldwide competitions for visual recognition like ILSVRC, PASCAL VOC. Deep learning techniques alleviate the need of human expertise from designing the handcrafted features and automatically learn the features. This resulted into use of deep architectures in many domains like computer vision (image classification, visual recognition) and natural language processing (language modeling, speech recognition). Object detection is one such promising area immensely needed to be used in automated applications like self-driving cars, robotics, drone image analysis. This paper analytically reviews state-of-the-art deep learning techniques based on convolutional neural networks for object detection.

The detection of rotated objects is a meaningful and challenging research work. Although the state-of-the-art deep learning models have feature invariance, especially convolutional neural networks (CNNs), their architectures did not specifically design for rotation invariance. They only slightly compensate for this feature through pooling layers. In this study, we propose a novel network, named LPNet, to solve the problem of object rotation. LPNet improves the detection accuracy by combining retina-like log-polar transformation. Furthermore, LPNet is a plug-and-play architecture for object detection and recognition. It consists of two parts, which we name as encoder and decoder. An encoder extracts images which feature in log-polar coordinates while a decoder eliminates image noise in cartesian coordinates. Moreover, according to the movement of center points, LPNet has stable and sliding modes. LPNet takes the single-shot multibox detector (SSD) network as the baseline network and the visual geometry group (VGG16) as the feature extraction backbone network. The experiment results show that, compared with conventional SSD networks, the mean average precision (mAP) of LPNet increased by 3.4% for regular objects and by 17.6% for rotated objects.

This paper proposes an enhancement of convolutional neural networks for object detection in resource-constrained robotics through a geometric input transformation called Visual Mesh. It uses object geometry to create a graph in vision space, reducing computational complexity by normalizing the pixel and feature density of objects. The experiments compare the Visual Mesh with several other fast convolutional neural networks. The results demonstrate execution times sixteen times quicker than the fastest competitor tested, while achieving outstanding accuracy.

Unlike standard Convolutional Neural Networks (CNNs) with fully-connected layers, Fully Convolutional Neural Networks (FCN) are prevalent in computer vision applications such as object detection, semantic/image segmentation, and the most popular generative tasks based on Generative Adversarial Networks (GAN). In an FCN, traditional convolutional layers and deconvolutional layers contribute to the majority of the computation complexity. However, prior deep learning accelerator designs mostly focus on CNN optimization. They either use independent compute-resources to handle deconvolution or convert deconvolutional layers (Deconv) into general convolution operations, which arouses considerable overhead. To address this problem, we propose a unified fully convolutional accelerator aiming to handle both the deconvolutional and convolutional layers with a single processing element (PE) array. We re-optimize the conventional CNN accelerator architecture of regular 2D processing elements array, to enable it more efficiently support the data flow of deconvolutional layer inference. By exploiting the locality in deconvolutional filters, this architecture reduces the consumption of on-chip memory communication from 24.79 GB to 6.56 GB and improves the power efficiency significantly. Compared to prior baseline deconvolution acceleration scheme, the proposed accelerator achieves 1.3X -- 44.9X speedup and reduces the energy consumption by 14.6%-97.6% on a set of representative benchmark applications. Meanwhile, it keeps similar CNN inference performance to that of an optimized CNN-only accelerator with negligible power consumption and chip area overhead.

Application of Deep Learning for Object Detection

The effectiveness of object detection largely depends on the availability of large annotated datasets to train the deep network successfully; however, obtaining a large-scale dataset is expensive and remains a challenge. In this work, we explore two different GAN-based approaches for data augmentation of agricultural images in a camouflaged environment. Camouflage is the property of an object which makes it hard to detect because of its similarity to its environment. We leverage paired and unpaired image-to-image translation to create synthetic images based on custom segmentation masks. We evaluate the quality of synthetic images by applying these to the object detection task as additional training samples. The experiments demonstrate that adversarial-based data augmentation significantly improves the accuracy of region-based convolutional neural network for object detection. Our findings show that when evaluated on the testing dataset, data augmentation achieves detection performance improvement of 3.97%. Given the difficulty of object detection task in camouflaged images, the result suggests that combining adversarial-based data augmentation with the original data can theoretically be synergistic in enhancing deep neural network efficiency to address the open problem of detecting objects in camouflaged environments.KeywordsGenerative adversarial networksData augmentationCamouflaged images

Object detection is a core computer vision task that aims to localize and classify categories for various objects in an image. With the development of convolutional neural networks, deep learning methods have been widely used in the object detection task, achieving promising performance compared to traditional methods. However, designing a well-performing detection network is inefficient. It consumes too much hardware resources and time to trial, and it also heavily relies on expert knowledge. To efficiently design the neural network architecture, there has been a growing interest in automatically designing neural network architecture by Neural Architecture Search (NAS). In this paper, we propose a Memory-Efficient Multi-Agent Neural Architecture Search (MEMA-NAS) framework in end-to-end object detection neural network. Specifically, we introduce the multi-agent learning to search holistic architecture of the detection network. In this way, a lot of GPU memory is saved, allowing us to search each module’s architecture of the detection network simultaneously. To find a better tradeoff between the precision and computational costs, we add the resource constraint in our method. Search experiments on multiple datasets show that MEMA-NAS achieves state-of-the-art results in search efficiency and precision.

Convolutional neural network based methods have dominated object detection in recent years, which can be divided into the one-stage approach and the two-stage approach. In general, the two-stage approach (e.g., Faster R-CNN) achieves high accuracy, while the one-stage approach (e.g., SSD) has the advantage of high efficiency. To inherit the merits of both while overcoming their disadvantages, we propose a novel single-shot based detector, namely RefineDet++, which achieves better accuracy than two-stage methods and maintains comparable efficiency of one-stage methods. The proposed RefineDet++ consists of two inter-connected modules: the anchor refinement module and the alignment detection module. Specifically, the former module aims to (1) filter out negative anchors to reduce search space for the subsequent classifier, and (2) coarsely adjust the locations and sizes of anchors to provide better initialization for the subsequent regressor. The latter module takes (1) the refined anchors as the input from the former module with (2) a newly designed alignment convolution operation to further improve the regression accuracy and predict multi-class label. Meanwhile, we design a transfer connection block to transfer the features in the anchor refinement module to predict locations, sizes and class labels of objects in the object detection module. The multi-task loss function enables us to train the whole network in an end-to-end way. Extensive experiments on PASCAL VOC and MS COCO demonstrate that RefineDet++ achieves state-of-the-art detection accuracy with high efficiency.

Air quality and reduction of emissions in the transport sector are determinant factors in achieving a sustainable global climate. The monitoring of emissions in traffic routes can help to improve route planning and to design strategies that may make the pollution levels to be reduced. In this work, a method which detects the pollution levels of transport vehicles from the images of IP cameras by means of computer vision techniques and neural networks is proposed. Specifically, for each sequence of images, a homography is calculated to correct the camera perspective and determine the real distance for each pixel. Subsequently, the trajectory of each vehicle is computed by applying convolutional neural networks for object detection and tracking algorithms. Finally, the speed in each frame and the pollution emitted by each vehicle are determined. Experimental results on several datasets available in the literature support the feasibility and scalability of the system as an emission control strategy.

Deep Learning Models or Neural Networks are loosely modeled based on the Human brain. Neural Networks consist of thousands of simple processing units that are densely interconnected, and are a means of performing machine learning. They aim to recognize patterns in data that enable a machine to perform some tasks based on analyzing several examples. One popular field of Deep Learning is Computer Vision that aims to simulate the complex visual system of human beings. An object detection model for instance, might be fed on thousands of images of objects to find visual patterns in the images that consistently correlate with particular labels. However, human visual perception is based on both visual and non-visual modalities of data. This paper aims to propose a Neural Network for Object detection that use various modalities of data, specifically Images and Text, to perform the object detection task

This paper focuses on the design and comparison of different deep neural networks for the real-time prediction of locomotor intentions by using data from inertial measurement units. The deep neural network architectures are convolutional neural networks, recurrent neural networks, and convolutional recurrent neural networks. The input to the architectures are features in the time domain, which have been derived either from one inertial measurement unit placed on the upper right leg of ten healthy subjects, or two inertial measurement units placed on both the upper and lower right leg of ten healthy subjects. The study shows that a WaveNet, i.e., a full convolutional neural network, achieves a peak F1-score of 87.17% in the case of one IMU, and a peak of 97.88% in the case of two IMUs, with a 5-fold cross-validation.

With the success of convolutional neural network (CNN) in the field of image processing and image recognition, an improved fully convolutional neural network (FCNN) based on CNN is proposed. Although FCNN has better performance than CNN, the number of parameters of FCNN has also increased significantly compared to CNN. Consequently, this paper proposes a multi‐feature fusion decision convolutional neural network (MFFD‐CNN), which use two kinds down‐sample methods in the pooling layer, and chooses the concatenate or add operation as the way that consolidates the feature of the convolutional layer with a stride of 2 and the max‐pooling layer to execute fusion operations. Meanwhile replaces the traditional softmax loss with an additive margin Softmax (AM‐Softmax) loss. The structure, which is based on the multi‐target classification experiment results of the MSTAR data set, shows can not only obtain an average correct recognition rate 1.2% higher than FCNN, but also better stability without augmenting the training samples of MSTAR data.

Spiking Neural Networks (SNNs), as a biologically inspired computational model, have garnered significant attention in object detection and image classification due to their event-driven mechanism and low-power characteristics. However, in object detection tasks, the residual structures in conventional networks introduce nonspiking operations, posing a critical challenge for SNNs. To address this issue, we propose a multi-layer feature cascade fusion SNN (MFCF-SNN) for object detection. During feature extraction, our novel multi-level cascaded feature extraction module replaces residual connections with cascade operations, eliminating nonspiking computations while enhancing gradient propagation to deeper layers. For downsampling, we introduce a pooling-convolution module that combines max-pooling and spiking convolution, effectively preserving feature information and improving gradient flow. These two modules collectively ensure pure spike-based computation while facilitating deep network training, thereby enhancing detection accuracy. Experimental results on the PASCAL VOC 2012 and SSDD datasets demonstrate state-of-the-art performance, validating the effectiveness of our approach in advancing SNN-based object detection.