Multi-flow Sub-network and Multiple Connections for Single Shot Detection

The study of remote sensing image object detection has excellent research value in environmental protection and public safety. However, the performance of the detectors is unsatisfactory due to the large variability of object size and complex background noise in remote sensing images. Therefore, it is essential to improve the detection performance of the detectors. Inspired by the idea of "divide and conquer", we proposed a Multiple Receptive Field Attention(MRFA) to solve these problems and which is a plug-and-play attention method. First, we use the method of multiple receptive field feature map generation to convert the input feature map into four feature maps with different receptive fields. In this way, the small, medium, large, and immense objects in the input feature maps are "seen" in these feature maps, respectively. Then, we used the multiple attention map fusion method to focus objects of different sizes separately, which can effectively suppress noise in the background of remote sensing images. Experiments on remote sensing object detection datasets DIOR and HRRSD demonstrate that the performance of our method is better than other state-of-the-art attention modules. At the same time, the experiments on remote sensing image semantic segmentation dataset WHDLD and classification dataset AID prove the generalization and superiority of our method.

Convolutional neural network based methods have dominated object detection in recent years, which can be divided into the one-stage approach and the two-stage approach. In general, the two-stage approach (e.g., Faster R-CNN) achieves high accuracy, while the one-stage approach (e.g., SSD) has the advantage of high efficiency. To inherit the merits of both while overcoming their disadvantages, we propose a novel single-shot based detector, namely RefineDet++, which achieves better accuracy than two-stage methods and maintains comparable efficiency of one-stage methods. The proposed RefineDet++ consists of two inter-connected modules: the anchor refinement module and the alignment detection module. Specifically, the former module aims to (1) filter out negative anchors to reduce search space for the subsequent classifier, and (2) coarsely adjust the locations and sizes of anchors to provide better initialization for the subsequent regressor. The latter module takes (1) the refined anchors as the input from the former module with (2) a newly designed alignment convolution operation to further improve the regression accuracy and predict multi-class label. Meanwhile, we design a transfer connection block to transfer the features in the anchor refinement module to predict locations, sizes and class labels of objects in the object detection module. The multi-task loss function enables us to train the whole network in an end-to-end way. Extensive experiments on PASCAL VOC and MS COCO demonstrate that RefineDet++ achieves state-of-the-art detection accuracy with high efficiency.

Traffic sign detection significantly reduces traffic accidents, but in real-world scenarios, the considerable distance between traffic signs and in-vehicle cameras means only a small proportion of traffic signs are captured in the images. This makes far-off traffic sign detection a small object detection problem, and with fewer details in small sign objects, detection becomes increasingly challenging. In this paper, we specifically address poor localization, low accuracy, and missed detections when using You Only Look Once Version 5 (YOLOv5) for detecting small traffic sign objects. Firstly, we incorporate a decoupled head into YOLOv5’s detection algorithm, which serves to improve detection accuracy and accelerate network convergence. Then, to handle low-resolution targets better, we substitute the network’s original convolution layers with Space-to-Depth Convolution (SPD-Conv) modules. This modification enhances the model’s capacity to extract features from low-resolution traffic sign objects. Lastly, we integrate the Context Augmentation Module (CAM) into YOLOv5 by employing variable rate extended convolution. This module extracts context information from multiple receptive fields, thus providing essential supplementary information and significantly enhancing detection accuracy. Empirical results demonstrate the efficacy of our algorithm, shown by a substantial increase in object detection precision rate to 95.0%, a recall rate of 91.6%, and an average precision of 95.4%. These results represent improvements of 2.1%, 4.8% and 3.7%, respectively, when compared to the original YOLOv5 algorithm. Furthermore, when tested against other state-of-the-art methods, our proposed methodology shows superior performance.

Object region proposal has a wide application on various domains, such as object detection, object tracking, robot navigation, and anomaly detection. Widely used region proposal methods are based on either grouping superpixels or sliding windows. Previous studies have been done by grouping either convolution features, superpixels, or windows. Moreover, histogram of gradients (HoGs) of lower-level features (pixels) can be used for the region proposal task. In this study, both the concepts of HoGs and convolution features are used for proposing object(s) region(s) in an image. Region proposal is done by HoGs of convolution features and statistical application on the gradient data of different feature maps. First, a sized image is passed through one convolution layer to generate reduced feature maps. Then, a window slides over each feature map and HoG is applied over each window. Thereafter, the histogram is analysed using the concept of Coefficient of Variation (CV) and angle (slope) applied on histogram data values. A window is considered as the object region if the CV and angle of its histogram data exceed some thresholds. For all feature maps, we have obtained several widows that represent approximated object regions. These windows are grouped to obtain the possible region(s) proposal. The effectiveness of the proposed method is demonstrated over one of the benchmark datasets, called `PASCAL VOC'. Our method is found superior to two other state-of-the-art, namely region-based segmentation (RBS), and combining efficient object localization and image classification (CEOLIC) in terms of recall.

This paper proposes a coupled region based convolutional neural networks (R-CNN) to automatically detect vehicles in aerial images. Traditional methods are mostly based on sliding-window search, and use handcrafted or shallow-learning based features. They have limited description ability and heavy computational costs. Recently, a series of R-CNN based methods have achieved great success in general object detection. Inspired by the previous work, we propose a coupled R-CNN to detect small size vehicles in large-scale aerial images. First, a vehicle proposal network (VPN) is proposed to generate candidate vehicle-like regions, using a hyper feature map combined by feature maps of different layers. Then, a vehicle classification network (VCN) is developed to further verify the candidate regions and classify vehicles in eight directions. In this study, our method is tested on a challenge Munich vehicle dataset and the collected vehicle dataset, with improvements in accuracy and speed compared to existing methods.

Object detection and tracking is one of the key applications of wireless sensor networks (WSNs). The key issues associated with this application include network lifetime, object detection and localization accuracy. To ensure the high quality of the service, there should be a trade-off between energy efficiency and detection accuracy, which is challenging in a resource-constrained WSN. Most researchers have enhanced the application lifetime while achieving target detection accuracy at the cost of high node density. They neither considered the system cost nor the object localization accuracy. Some researchers focused on object detection accuracy while achieving energy efficiency by limiting the detection to a predefined target trajectory. In particular, some researchers only focused on node clustering and node scheduling for energy efficiency. In this study, we proposed a mobile object detection and tracking framework named the Energy Efficient Object Detection and Tracking Framework (EEODTF) for heterogeneous WSNs, which minimizes energy consumption during tracking while not affecting the object detection and localization accuracy. It focuses on achieving energy efficiency via node optimization, mobile node trajectory optimization, node clustering, data reporting optimization and detection optimization. We compared the performance of the EEODTF with the Energy Efficient Tracking and Localization of Object (EETLO) model and the Particle-Swarm-Optimization-based Energy Efficient Target Tracking Model (PSOEETTM). It was found that the EEODTF is more energy efficient than the EETLO and PSOEETTM models.

There have been two streams in the 3D detection from point clouds: single-stage methods and two-stage methods. While the former is more computationally efficient, the latter usually provides better detection accuracy. By carefully examining the two-stage approaches, we have found that if appropriately designed, the first stage can produce accurate box regression. In this scenario, the second stage mainly rescores the boxes such that the boxes with better localization get selected. From this observation, we have devised a single-stage anchor-free network that can fulfill these requirements. This network, named AFDetV2, extends the previous work by incorporating a self-calibrated convolution block in the backbone, a keypoint auxiliary supervision, and an IoU prediction branch in the multi-task head. We take a simple product of the predicted IoU score with the classification heatmap to form the final classification confidence. The enhanced backbone strengthens the box localization capability, and the rescoring approach effectively joins the object presence confidence and the box regression accuracy. As a result, the detection accuracy is drastically boosted in the single-stage. To evaluate our approach, we have conducted extensive experiments on the Waymo Open Dataset and the nuScenes Dataset. We have observed that our AFDetV2 achieves the state-of-the-art results on these two datasets, superior to all the prior arts, including both the single-stage and the two-stage 3D detectors. AFDetV2 won the 1st place in the Real-Time 3D Detection of the Waymo Open Dataset Challenge 2021. In addition, a variant of our model AFDetV2-Base was entitled the "Most Efficient Model" by the Challenge Sponsor, showing a superior computational efficiency. To demonstrate the generality of this single-stage method, we have also applied it to the first stage of the two-stage networks. Without exception, the results show that with the strengthened backbone and the rescoring approach, the second stage refinement is no longer needed.

Over the last few years, researchers have focused on computer‐aided polyp detection in gastroscopy. Deep learning (DL) has shown great promise for polyps' identification. The most exceptional contribution of DL methods in gastroenterology is their ability to identify polyps quickly and accurately using convolution neural network. Nonetheless, despite significant advancements, automatic detection of small polyps remains a challenging and complex task. Furthermore, due to multiple pooling operations, the features of small polyps are lost, resulting in low detection accuracy. This paper proposes an efficient object detection method for polyp detection using gastric images to address this issue. A single‐shot multi‐box detector (SSD) was combined with the feature extractor VGG‐16, and the Refined Map Block (RMB) was integrated into SSD's high‐resolution feature maps to get more semantic information. The RMB output was used as the input to the successive layers. The RMB comprises of attention cascade and feature map concatenation cascade. The attention cascade improved the localization accuracy, while the feature map concatenation cascade improved the classification accuracy. Using the former, the proposed attention‐based SSD for gastric polyps (ASSD‐GPNet) model focused on the specific information, a polyp, rather than the background. Furthermore, the feature map concatenation cascade adds semantic information while reducing computational complexity. The output of these two cascades was combined to produce a refined feature map that enhances the detection of small polyps. The model was trained and tested on 1970 gastric images and Pascal VOC07 + 12. Image augmentation was applied to increase the training data of gastric images to reduce overfitting and skip connections were used to overcome the vanishing gradient problem. Overall, the experimental results demonstrated that the proposed model outperformed than compared models in both medical and natural images. The ASSD‐GPNet obtained mean average precision (mAP) of 94.2% on gastric images and 76.9% on Pascal VOC.

A CNN object detection method based on the structural reparameterisation technique using branch matching is proposed to address the problem of balancing accuracy and speed in object detection techniques. By the structural reparameterisation of the convolutional layer in the object detection network, the amount of computation and the number of parameters in the network inference are reduced, the memory overhead is lowered, and the use of the branch-matching method to improve the structural reparameterisation model improves the computational efficiency and speed of the network while maintaining the detection accuracy. Optimisation is also carried out in terms of target screening and loss function, and a new CPC NMS screening strategy was introduced to further improve the performance of the model. The experimental results show that the proposed method achieves competitive results on the PASCAL VOC2012 and MS COCO2017 datasets compared to the traditional object detection methods and the current mainstream models, achieving a better balance between the detection accuracy and detection speed.

This paper solves the speed bottleneck of deformable part model (DPM), while maintaining the accuracy in detection on challenging datasets. Three prohibitive steps in cascade version of DPM are accelerated, including 2D correlation between root filter and feature map, cascade part pruning and HOG feature extraction. For 2D correlation, the root filter is constrained to be low rank, so that 2D correlation can be calculated by more efficient linear combination of 1D correlations. A proximal gradient algorithm is adopted to progressively learn the low rank filter in a discriminative manner. For cascade part pruning, neighborhood aware cascade is proposed to capture the dependence in neighborhood regions for aggressive pruning. Instead of explicit computation of part scores, hypotheses can be pruned by scores of neighborhoods under the first order approximation. For HOG feature extraction, look-up tables are constructed to replace expensive calculations of orientation partition and magnitude with simpler matrix index operations. Extensive experiments show that (a) the proposed method is 4 times faster than the current fastest DPM method with similar accuracy on Pascal VOC, (b) the proposed method achieves state-of-the-art accuracy on pedestrian and face detection task with frame-rate speed.

Accurate object detection requires correct classification and high-quality localization. Currently, most of the single shot detectors (SSDs) conduct simultaneous classification and regression using a fully convolutional network. Despite high efficiency, this structure has some inappropriate designs for accurate object detection. The first one is the mismatch of bounding box classification, where the classification results of the default bounding boxes are improperly treated as the results of the regressed bounding boxes during the inference. The second one is that only one-time regression is not good enough for high-quality object localization. To solve the problem of classification mismatch, we propose a novel reg-offset-cls (ROC) module including three hierarchical steps: the regression of the default bounding box, the prediction of new feature sampling locations, and the classification of the regressed bounding box with more accurate features. For high-quality localization, we stack two ROC modules together. The input of the second ROC module is the output of the first ROC module. In addition, we inject a feature enhanced (FE) module between two stacked ROC modules to extract more contextual information. The experiments on three different datasets (i.e., MS COCO, PASCAL VOC, and UAVDT) are performed to demonstrate the effectiveness and superiority of our method. Without any bells or whistles, our proposed method outperforms state-of-the-art one-stage methods at a real-time speed. The source code is available at https://github.com/JialeCao001/HSD.

Object detection is a challenging task in the field of remote sensing applications due to the complex backgrounds and uncertain orientation of targets. Compared with the horizontal bounding box, the oriented bounding box can provide orientation information while retaining the true size. Most existing oriented object detection methods are based on Faster-RCNN and the other one-stage methods that can achieve real-time speed but have shortcomings in localization and detection accuracy. To further enhance the performance of one-stage methods, we propose an oriented object detection framework that is based on the single shot detector, namely, single shot anchor refinement network (S²ARN). The S²ARN obtains the accurate detection results by performing two consecutive regressions. More precisely, the multilevel features of the backbone are used to regress the coordinate offsets between the predefined rotated anchors and the ground-truth boxes to generate the refined anchors. The classification and regression subnetworks assigned to the output features are used to perform the second regression to determine the class labels and further adjust the location of the refined anchors. In addition, receptive field amplification modules (RFAMs) are inserted to enlarge the receptive field and extract more discriminative features. Furthermore, in the anchor matching step, angle-related Intersection over Union (ArIoU) is used to calculate the Intersection over Union (IoU) score instead of the traditional method. Benefiting from the multiple regressions and the insensitivity of the ArIoU score to the angle deviation, the angle sampling interval of the rotated anchor can be reduced. The experimental results for the two public datasets, HRSC2016 and UCAS-AOD, demonstrate the effectiveness of the proposed network.

Recently, it has been recognized that a single airway sensory unit may contain multiple receptive fields and that each field houses at least one encoder. Since some units respond to both lung inflation and deflation, we hypothesized that these units contain heterogeneous encoders for sensing inflation and deflation, respectively. Single unit activities were recorded from the cervical vagus nerve in anesthetized, open chest, and mechanically ventilated rabbits. Fifty-two airway sensory units with multiple receptive fields that responded to both lung inflation and deflation were identified. Among them, 13 units had separate receptive fields for inflation and deflation, where one of the fields could be blocked by local injection of 2% lidocaine (10 μl). In 8 of the 13 units, the deflation response was blocked without affecting the unit's response to inflation, whereas in the remaining five units, the inflation response was blocked without affecting the deflation response. Our results support the hypothesis that a single mechanosensory unit may contain heterogeneous encoders that can respond to either inflation or deflation.

Brain–computer interface (BCI) based on motor imagery (MI) supplies a communication method between humans and machines independent of external stimulation. MI is based on the imagination quality of the subjects, which resulted in large variability and unreliability of the signals. Improved MI paradigms and data analysis methods would be helpful to reduce the side effects of the individual specificity and temporal variation, and increase the classification performance of MI-based BCI. Visual guidance could improve the MI quality, but there is a lack of a systematic analysis of the forms of visual guidance and their effects on MI. In this article, we conducted a comparative study to evaluate the MI paradigms with different visual guidance manners. Five paradigms were designed, i.e., pure MI, MI with synchronous pursuit (SP), asynchronous pursuit (AP), synchronous saccade (SS), and asynchronous saccade (AS). Furthermore, a convolutional neural network (CNN) architecture with multiple receptive fields and attention module was proposed. Twenty subjects accomplished the experiments. The SP paradigm induced the most significant event-related desynchronization (ERD) phenomenon and the highest classification accuracy for electroencephalograph (EEG) signals. The proposed network achieved an average classification accuracy of 91.89% and standard deviation of 5.55%, which outperformed the compared methods. To test the applicability of the paradigm and the method, six subjects with different performance in the offline experiment then participated in an online experiment and a simulated brain-controlled trajectory tracking flight of the unmanned aerial vehicle (UAV). All the subjects could accomplish the task, and their performance was positively correlated with the classification accuracy and negatively correlated with the complexity of the tracking path. In general, the SP visual guidance effectively helped the subjects modulate their brain activity, leading to increased MI quality. The convolutional network with multiple receptive fields and an attention module showed the improved classification performance and robustness.

In the process of remote sensing data processing, cloud data will have a considerable negative impact on data processing.Therefore, a matrix optimization cloud detection algorithm based on multiple receptive fields is proposed in this paper.First, the algorithm adopts a block reshaping model to improve the efficiency of the algorithm, while reducing the need for hardware configuration.Second, the cloud region adaptive sparse attention matrix is used to improve the characteristics of cloud regions with different concentrations.Finally, the multi-receptive field scaling module is used to improve the ability to segment cloud regions at different scales.The experimental results show that the accuracy of the proposed algorithm is 85.5%, and the recall rate is 86.4%.The proposed algorithm can basically accurately screen the location of a cloud region and provide technical support for the subsequent image application processing.