SCRDet++: Detecting Small, Cluttered and Rotated Objects via Instance-Level Feature Denoising and Rotation Loss Smoothing.

A Real-World Size Organization of Object Responses in Occipitotemporal Cortex

With the development of deep convolutional neural networks (CNNs), the object detection accuracy has been greatly improved. But the performance of small object detection is still far from satisfactory, mainly because small objects are so tiny that the information contained in the feature map is limited. Existing methods focus on improving classification accuracy but still suffer from the limitation of bounding box prediction. To solve this issue, we propose a detection framework by generative and discriminative learning. First, a reconstruction generator network is designed to reconstruct the mapping from low frequency to high frequency for anchor box prediction. Then, a detector module extracts the regions of interest (ROIs) from generated results and implements a RoI-Head to predict object category and refine bounding box. In order to guide the reconstructed image related to the corresponding one, a discriminator module is adopted to tell from the generated result and the original image. Extensive evaluations on the challenging MS-COCO dataset demonstrate that our model outperforms most state-of-the-art models in detecting small objects, especially the reconstruction module improves the average precision for small object (APs) by 7.7%.

In recent years, in the field of remote sensing image target detection, oriented target detection has received extensive attention. Traditional detectors use horizontal bounding box (HBB) to locate targets, which may cause inaccurate detection when detecting targets with arbitrary orientation angles, dense distribution and large aspect ratios. The Oriented Bounding Box (OBB) adds different rotation angles to the horizontal bounding box, which can better deal with the above problems. However, with the introduction of oriented bounding boxes in the detector, new problems have emerged, such as the increase in the number of anchor points with different scales, angles, and aspect ratios, and the imbalance of IoU matching of positive and negative anchor boxes, which leads to classification confidence and Inconsistency between positioning accuracy. In order to overcome these shortcomings while using the oriented bounding box, this paper proposes a novel oriented target detector and redesigns a matching strategy for evaluating the position of the anchor frame. By selecting a higher quality anchor frame to achieve more accurate target detection, thereby reducing the instability of the angle in the matching process. Experiments conducted on public remote sensing data sets such as HRSC2016 and UCAS-AOD show that the proposed method achieves advanced detection accuracy.

Range-spread target (RST) detection is an important issue for high-resolution radar (HRR). Traditional detectors relying on manually designed detection statistics have their performance limitations. Therefore, in this work, two deep learning-based detectors are proposed for RST detection using HRRPs, i.e., an NLS detector and DFCW detector. The NLS detector leverages domain knowledge from the traditional detector, treating the input HRRP as a low-level feature vector for target detection. An interpretable NLS module is designed to perform noise reduction for the input HRRP. The DFCW detector takes advantage of the extracted high-level feature map of the input HRRP to improve detection performance. It incorporates a feature cross-weighting module for element-wise feature weighting within the feature map, considering the channel and spatial information jointly. Additionally, a nonlinear accumulation module is proposed to replace the conventional noncoherent accumulation operation in the double-HRRP detection scenario. Considering the influence of the target spread characteristic on detector performance, signal sparseness is introduced as a measure and used to assist in generating two datasets, i.e., a simulated dataset and measured dataset incorporating real target echoes. Experiments based on the two datasets are conducted to confirm the contribution of the designed modules to detector performance. The effectiveness of the two proposed detectors is verified through performance comparison with traditional and deep learning-based detectors.

With the advancement of object detection, oriented object detection is gradually replacing horizontal object detection as the preferred solution in remote sensing images. However, most existing datasets are annotated with horizontal bounding boxes, while re-annotating these datasets is both costly and inefficient. Recently, horizontal box supervised oriented object detection methods using merely horizontal box annotations have emerged as a solution. Nonetheless, these methods are relatively poor in predicting angles because of lacking angle supervision. Additionally, these methods overlook edge features, which can help capture and represent the oriented boundaries between objects and backgrounds, especially for objects with distinct edges or in blank backgrounds. To address these issues, we propose an edge feature self-attention and multi-task synthetic pattern enhanced horizontal box supervised oriented object detector named EAMSDet, aiming to achieve high precision based on horizontal box annotations. Specifically, we use horizontal and vertical Sobel operators to extract edge information and a self-attention module to fuse the edge information with the feature maps. Furthermore, to enrich the supervision of aspect ratio and angles, we add a multi-task synthetic branch and an extra regression loss to make full use of our introduced synthetic visual patterns. Based on H2RBox-v2, we conducted extensive experiments on remote sensing datasets such as DOTA-v1.0, DIOR, and HRSC2016, demonstrating the effectiveness and generalizability of our EAMSDet.

Compared with general object detection with horizontal bounding boxes in natural images, oriented object detection in remote sensing images is an active and challenging research topic as objects are usually displayed in arbitrary orientations. To model the variant orientations of oriented objects, general CNN-based methods usually adopt more parameters or well-designed modules, which are often complex and inefficient. To address this issue, the detector requires two key components to deal with: (i) generating oriented proposals in a light-weight network to achieve effective representation of arbitrarily oriented objects; (ii) extracting the rotation-invariant feature map in both spatial and orientation dimensions. In this paper, we propose a novel, lightweight rotated region proposal network to produce arbitrary-oriented proposals by sliding two vertexes only on adjacent sides and adopt a simple yet effective representation to describe oriented objects. This may decrease the complexity of modeling orientation information. Meanwhile, we adopt the rotation-equivariant backbone to generate the feature map with explicit orientation channel information and utilize the spatial and orientation modules to obtain completely rotation-invariant features in both dimensions. Without tricks, extensive experiments performed on three challenging datasets DOTA-v1.0, DOTA-v1.5 and HRSC2016 demonstrate that our proposed method can reach state-of-the-art accuracy while reducing the model size by 40% in comparison with the previous best method.

Airport runways, as the core part of airports, belong to vital national infrastructure, and the target detection and segmentation of airport runways in remote sensing images using deep learning methods have significant research value. Most of the existing airport target detection methods based on deep learning rely on horizontal bounding boxes for localization, which often contain irrelevant background information. Moreover, when detecting multiple intersecting airport runways in a single remote sensing image, issues such as false positives and false negatives are apt to occur. To address these challenges, this study proposes an end-to-end remote sensing image airport runway detection and segmentation method based on an improved Mask RCNN (CR-Mask RCNN). The proposed method uses a rotated region generation network instead of a non-rotated region generation network, allowing it to generate rotated bounding boxes that fit the shape of the airport runway more closely, thus avoiding the interference of a large amount of invalid background information brought about by horizontal bounding boxes. Furthermore, the method incorporates an attention mechanism into the backbone feature extraction network to allocate attention to different airport runway feature map scales, which enhances the extraction of local feature information, captures detailed information more effectively, and reduces issues of false positives and false negatives when detecting airport runway targets. The results indicate that, when comparing horizontal bounding boxes with rotated bounding boxes for detecting and segmenting airport runways, the latter are more precise for complex backgrounds. Furthermore, incorporating an attention mechanism enhances the accuracy of airport runway recognition, making it highly effective and practical.

With the wide application of remote sensing images (RSIs) in military and civil fields, remote sensing object detection (RSOD) has gradually become a hot research direction. However, we observe two main challenges for remote sensing object detection, namely the complicated background and the small objects issues. Given the different appearances of generic objects and remote sensing objects, the detection algorithms designed for the former usually cannot perform well for the latter. We propose a novel global to local scale-aware detection network (GLSANet) for remote sensing object detection, aiming to solve the above mentioned two challenges. Firstly, we design a global semantic information interaction module (GSIIM) to excavate and reinforce the high-level semantic information in the deep feature map, which alleviates the obstacles of complex background on foreground objects. Secondly, we optimize the feature pyramid network to improve the performance of multiscale object detection in RSIs. Finally, a local attention pyramid (LAP) is introduced to highlight the feature representation of small objects gradually while suppressing the background and noise in the shallower feature maps. Extensive experiments on three public datasets demonstrate that the proposed method achieves superior performance compared with the state-of-the-art detectors, especially on small object detection dataset. Specifically, our algorithm reaches 94.57% mAP on NWPU VHR-10 dataset, 95.93% mAP on RSOD dataset and 77.9% mAP on DIOR dataset, respectively.

DeLiVoTr: Deep and light-weight voxel transformer for 3D object detection

Recent improvements in object detection are driven by the success of convolutional neural networks (CNN). They are able to learn rich features outperforming hand-crafted features. So far, research in traffic light detection mainly focused on hand-crafted features, such as color, shape or brightness of the traffic light bulb. This paper presents a deep learning approach for accurate traffic light detection in adapting a single shot detection (SSD) approach. SSD performs object proposals creation and classification using a single CNN. The original SSD struggles in detecting very small objects, which is essential for traffic light detection. By our adaptations it is possible to detect objects much smaller than ten pixels without increasing the input image size. We present an extensive evaluation on the DriveU Traffic Light Dataset (DTLD). We reach both, high accuracy and low false positive rates. The trained model is real-time capable with ten frames per second on a Nvidia Titan Xp. Code has been made available at https://github.com/julimueller/tl_ssd.

Use of autonomous vehicles aims to eventually reduce the number of motor vehicle fatalities caused by humans. Deep learning plays an important role in making this possible because it can leverage the huge amount of training data that comes from autonomous car sensors. Automatic recognition of traffic light and vehicle signal is a perception module critical to autonomous vehicles because a deadly car accident could happen if a vehicle fails to follow traffic lights or vehicle signals. A practical Traffic Light Recognition (TLR) or Vehicle Signal Recognition (VSR) faces some challenges, including varying illumination conditions, false positives and long computation time. In this chapter, we propose a novel approach to recognize Traffic Light (TL) and Vehicle Signal (VS) with high dynamic range imaging and deep learning in real-time. Different from existing approaches which use only bright images, we use both high exposure/bright and low exposure/dark images provided by a high dynamic range camera. TL candidates can be detected robustly from low exposure/dark frames because they have a clean dark background. The TL candidates on the consecutive high exposure/bright frames are then classified accurately using a convolutional neural network. The dual-channel mechanism can achieve promising results because it uses undistorted color and shape information of low exposure/dark frames as well as rich texture of high exposure/bright frames. Furthermore, the TLR performance is boosted by incorporating a temporal trajectory tracking method. To speed up the process, a region of interest is generated to reduce the search regions for the TL candidates. The experimental results on a large dual-channel database have shown that our dual-channel approach outperforms the state of the art which uses only bright images. Encouraged by the promising performance of the TLR, we extend the dual-channel approach to vehicle signal recognition. The algorithm reported in this chapter has been integrated into our autonomous vehicle via Data Distribute Service (DDS) and works robustly in real roads.

In view of the fact that the feature layers of different scales in the traditional single shot multibox detector (SSD) are independent of each other, resulting in poor detection performance for small objects. We propose an improved SSD network for small object detection based on dilated convolution and feature fusion, which is called DFSSD. Specifically, by introducing dilated convolution, we enhance the receptive field of the third-level feature map in the network, which enables the feature map to obtain more global information. At the same time, we designed feature fusion module to fuse low-level feature map with detailed information and high-level feature map with rich semantic information. We adjust the prediction box scale of the DFSSD network prediction layer. Our proposed network obtains 78.9% mAP on PASCAL VOC2007 test at 40 FPS and 74.7% AP on difficult objects in car class of KITTI dataset. The results outperform the original SSD model by 1.4 and 1.2 points respectively.

Small object detection is a hard task to be sovled in object detection because the small object usually contain a few effective informations, which makes it very difficult to extract the discriminative features of small scale object. In addition, a key challenge in deep network for small object detectors is the large scale variation, where the performance of small object is poor due to the imbalanced optimization on various scale. In this paper, we propose a scale adaptive balance mechanism based on anchor-free framework for small object detection. Specifically, we design a new loss function to adapt the detection model to various object scale, which is able to alleviate the imbalance of object scale. Moreover, in feature extractor, the shallow feature maps of different receptive fields are added to the DLA-34 backbone for better extracting multi-scale features. Besides, the image block is used as a detection unit during inference, and the scale mismatch is thus overcome. Finally, the proposed method is verified on the Visdrone2019 dataset and the experimental results validate the effectiveness to detect small objects in the video image from drone monitoring.

Traffic light recognition using high-definition map features

IntroductionWith advances in artificial intelligence, the use of computer-aided detection and diagnosis in clinical imaging is gaining traction. Typically, very large datasets are required to train machine-learning models, potentially limiting use of this technology when only small datasets are available. This study investigated whether pretraining of fracture detection models on large, existing datasets could improve the performance of the model when locating and classifying wrist fractures in a small X-ray image dataset. This concept is termed “transfer learning”.MethodFirstly, three detection models, namely, the faster region-based convolutional neural network (faster R-CNN), you only look once version eight (YOLOv8), and RetinaNet, were pretrained using the large, freely available dataset, common objects in context (COCO) (330000 images). Secondly, these models were pretrained using an open-source wrist X-ray dataset called “Graz Paediatric Wrist Digital X-rays” (GRAZPEDWRI-DX) on a (1) fracture detection dataset (20327 images) and (2) fracture location and classification dataset (14390 images). An orthopaedic surgeon classified the small available dataset of 776 distal radius X-rays (Arbeidsgmeischaft für Osteosynthesefragen Foundation / Orthopaedic Trauma Association; AO/OTA), on which the models were tested.ResultDetection models without pre-training on the large datasets were the least precise when tested on the small distal radius dataset. The model with the best accuracy to detect and classify wrist fractures was the YOLOv8 model pretrained on the GRAZPEDWRI-DX fracture detection dataset (mean average precision at intersection over union of 50=59.7%). This model showed up to 33.6% improved detection precision compared to the same models with no pre-training.ConclusionOptimisation of machine-learning models can be challenging when only relatively small datasets are available. The findings of this study support the potential of transfer learning from large datasets to improve model performance in smaller datasets. This is encouraging for wider application of machine-learning technology in medical imaging evaluation, including less common orthopaedic pathologies.