MCS-YOLO: A Multiscale Object Detection Method for Autonomous Driving Road Environment Recognition

Unmanned aerial vehicles (UAVs) have been widely used in post-disaster search and rescue operations, object tracking, and other tasks. Therefore, the autonomous perception of UAVs based on computer vision has become a research hotspot in recent years. However, UAV images include dense objects, small objects, and arbitrary object directions, which bring about significant challenges to existing object detection methods. To alleviate these issues, we propose a global-local feature enhanced network (GLF-Net). Considering the difficulty of processing UAV images with complex scenes and dense objects, we designed a backbone based on an involution and self-attention that can extract effective features from complex objects. A multiscale feature fusion module is also proposed to address the presence of numerous small objects in UAV images through multiscale object detection and feature fusion. To accurately detect rotated objects, a rotated regional proposal network was designed based on the midpoint offset representation, which can apply a rotated box to determine the real direction and contour of an object. GLF-Net achieves a state-of-the-art detection accuracy (86.52% mAP) on our created RO-UAV dataset, while achieving 96.95% and 97% mAP on the public datasets HRSC2016 and UCAS-AOU, respectively. The experimental results demonstrate that our method achieves a high detection accuracy and generalization, which can meet the practical requirements of UAVs under various complex scenarios.

Multi-scale object detection in natural scenes is still challenging. To enhance the multi-scale perception capability, some algorithms combine the lower-level and higher-level information via multi-scale feature fusion strategies. However, the inherent spatial properties among instances and relations between foreground and background are ignored. In addition, the human-defined “center-based” regression quality evaluation strategy, predicting a high-to-low score based on a linear relationship with the distance to the center of ground-truth box, is not robust to scale-variant objects. In this work, we propose a Depth-Guided Progressive Network (DGPNet) for multi-scale object detection. Specifically, besides the prediction of classification and localization, the depth is estimated and used to guide the image features in a weighted manner to obtain a better spatial representation. Therefore, depth estimation and 2D object detection are simultaneously learned via a unified network, where the depth features are merged as auxiliary information into the detection branch to enhance the discrimination among multi-scale objects. Moreover, to overcome the difficulty of empirically fitting the localization quality function, high-quality predicted boxes on scale-variant objects are more adaptively obtained by an IoU-aware progressive sampling strategy. We divide the sampling process into two stages, i.e., “statistical-aware” and “IoU-aware”. The former selects thresholds for positive samples based on statistical characteristics of multi-scale instances, and the latter further selects high-quality samples by IoU on the basis of the former. Therefore, the final ranking scores better reflect the quality of localization. Experiments verify that our method outperforms state-of-the-art methods on the KINS and Cityscapes dataset.

In this paper, we are interested in the object detection problem with high accuracy and fast prediction. To achieve this, we have modified the existing object detection algorithm, we design a new feature extraction network, predict detection target based on multiple feature maps and improve the anchor box. Particularly, since DenseNet can combine the output of previous convolutional layers by using skip connection operation to make prediction more accurate, we introduce DenseNet to improve the YOLOv3-Tiny network. Furthermore, we add a prediction layer to the improved network to detect more true targets. Finally, we clustered the ground truth of specific dataset to get the anchor box for training to improve the detection results of the improved network. We demonstrate the effectiveness of our network by using Pascal VOC datasets. The experimental results show that the improved network increases the detection accuracy by 14% and the detection speed is 63 frames per second in the video detection. The detection accuracy is improved to a higher level and the detection speed still meeting the real-time detection requirement, which shows that the proposed method is robust and effective. Anchor box, DenseNet, Video detection

Compared with the traditional object detection algorithm, the object detection algorithm based on deep learning has stronger robustness to complex scenarios, which is the hot direction of current research. According to the process characteristics of the object detection algorithm based on deep learning, it is divided into two-stage object detection algorithm and single-stage object detection algorithm, focusing on the problems solved by some classical algorithms and their advantages and disadvantages. In view of the problem of object detection, especially small object detection, the commonly used data sets and performance evaluation indicators are summarized; the characteristics, advantages, and detection difficulties of various common data sets are compared; the challenges faced by commonly used object detection methods and small object detection are systematically summarized; the latest work of small object detection methods based on deep learning is sorted out; and the small object detection methods based on multiscale and small object detection methods based on super-resolution are introduced. At the same time, the lightweight strategy for target detection methods and the performance of some lightweight models are introduced; the characteristics, advantages, and limitations of various methods are summarized; and the future development direction of small object detection methods based on deep learning is prospected.

Thangka cultural elements detection aims to locate and identify instances in Thangka. However, as a unique form of pictorial art, Thangka exhibits distinct spatial structures that deviate significantly from general images in scale and density. Therefore, it is challenging for most state-of-the-art detectors designed for natural scenes to handle Thangka cultural elements detection effectively. To overcome this issue, we propose a multi-scale and dense object detector referred as MDDet. It embeds a multi-scale receptive field fusion module (MRF) that enlarges the receptive field while capturing the spatial and channel relationships at different scales, which significantly enriches the multi-scale features extracted from the backbone. In addition, we introduce a threshold-slicing aided hyper inference (T-SAHI) scheme, which adaptively slices images in dense scenarios to aid with dense object detection in the test time. We thoroughly evaluate our method, and MDDet outperforms the prior art by a clear margin on the Thangka dataset, achieving an absolute improvement of 1.9% in average precision (AP). For the challenging medium and small objects in Thangka, MDDet obtains wide margins of 12% and 3.7% in accuracy improvement, respectively. It also shows strong generalization ability when evaluated on general scenarios, e.g., Pascal VOC 2007 and MS COCO, validating the role of MDDet in object detection.

Traditional feature pyramid structures have limitations in cross-level semantic information fusion, leading to inconsistent detection results across different scales. This paper proposes a Mutual Information-based Feature Pyramid Enhancement mechanism (MI-FPE) to address the lack of consensus between feature pyramid levels in real-time object detection. The aim is to improve the performance of Real-Time Detection Transformer (RT-DETR) models in multi-scale object detection. MI-FPE uses a scale alignment module to unify feature map sizes. It employs a mutual information maximization strategy to extract common semantic information across levels. An adaptive fusion mechanism then flexibly injects this common information into each original feature layer. Experimental results on the MS-COCO dataset show significant improvements in detection performance, especially for small objects. With ResNet-50 as the backbone network, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$A P_{\text{val }}$</tex> reaches 53.2%, a 0.1 percentage point increase over the baseline RT-DETR. The small object detection metric <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$A P_{S}^{\text{val }}$</tex> improves by 1.8 percentage points to 36.6%.

ABSTRACTObject detection has played an important role in the remote sensing (RS) image understanding domain, but it still suffers from challenges, including the complexity of the image background, the variety of objects and differences of target scales. Although many studies applying deep learning methods to object detection have been presented, effective methods of multiscale object detection in RS images are still lacking. In this paper, we propose a novel object detection approach based on the Multiscale-SELU-DenseNet (MSE-DenseNet) and the dynamic anchor assignment (DAA) strategy. We fuse a highly modified DenseNet and a feature pyramid network into the MSE-DenseNet to extract multi-level feature descriptions of RS images, and then we use them to detect objects on various scales. To further improve the detection performance, we propose a DAA strategy for assigning prior anchor boxes. The experimental results show that our proposed method outperforms state-of-the-art detection methods on the VHR-10 dataset and maintains a real-time detection speed.

SummaryAs various algorithms and models spring up, object detection‐based deep learning for image analysis has become an established technology in the field of computer vision. Furthermore, object detection for high‐resolution videos derived from ubiquitous surveillance cameras draws many researchers' attention due to its practical significance and the challenge on detecting accuracy and speed. Existing object detection methods mainly resize and compress each frame in the video and then apply the detection algorithm, but with the method, the detection for small objects in dense scenes is far from satisfactory in terms of accuracy because small objects are reduced to pixel‐level sizes in compressed frame images. In order to fix this issue, we propose AROD, a parallel detection framework based on adaptive image cropping. Unlike the traditional first‐compression‐and‐then‐detection methods, AROD adopts adaptive image cropping for distributed parallel detection. In this way, AROD significantly improves the accuracy of small object detection in dense scenes with acceptable overhead. In our experimental evaluation, YOLOv2‐tiny model equipped with AROD reaches 29.29 FPS along with high detection accuracy. Experimental results verify that AROD ensures the high throughput of real‐time video analytics while maintaining high detection accuracy.

High tracking frame rates have been achieved based on traditional tracking methods which however would fail due to drifts of the object template or model, especially when the object disappears from the camera’s field of view. To deal with it, tracking-and-detection-combination has become more and more popular for long-term unknown object tracking, whose detector almost does not drift and can regain the disappeared object when it comes back. However, for online machine learning and multiscale object detection, expensive computing resources and time are required. So it is not a good idea to combine tracking and detection sequentially like Tracking-Learning-Detection algorithm. Inspired from parallel tracking and mapping, this article proposes a framework of parallel tracking and detection for unknown object tracking. The object tracking algorithm is split into two separate tasks—tracking and detection which can be processed in two different threads, respectively. One thread is used to deal with the tracking between consecutive frames with a high processing speed. The other thread runs online learning algorithms to construct a discriminative model for object detection. Using our proposed framework, high tracking frame rates and the ability of correcting and recovering the failed tracker can be combined effectively. Furthermore, our framework provides open interfaces to integrate state-of-the-art object tracking and detection algorithms. We carry out an evaluation of several popular tracking and detection algorithms using the proposed framework. The experimental results show that different tracking and detection algorithms can be integrated and compared effectively by our proposed framework, and robust and fast long-term object tracking can be realized.

<p indent=0mm>Existing object detection algorithms only use a fixed size convolution kernel when extracting features, ignoring the difference in the receptive field of different scale features, which affects the detection effect of different scale objects. To solve this problem, a multi-scale object detection network based on multi-branch parallel dilated convolution is proposed. Firstly, the basic network VGG-16 is used to extract the features of the image. Secondly, a multi-branch parallel dilated convolution is designed to extract multi-scale features to improve object detection ability of the network. Then, a non-local block is employed to integrate the global spatial information and enhance the context information. Finally, the object detection and location tasks are performed on feature maps with different scales. Experimental results on PASCAL VOC and MS COCO datasets demonstrate that the proposed method can effectively improve the detection accuracy of different scale objects and clearly improve the detection accuracy of small objects.

The synthetic aperture radar (SAR) has a special ability to detect objects in any climate and weather conditions. Consequently, SAR images are widely used in maritime transportation safety and fishery law enforcement for maritime object detection. Currently, deep-learning models are being extensively used for the detection of objects from images. Among them, the feature pyramid network (FPN) uses pyramids for representing semantic information regardless of the scale and has an improved accuracy of object detection. It is also suitable for the detection of multiple small ship objects in SAR images. This study aims to resolve the problems associated with small-object and multi-object ship detection in complex scenarios e.g., when a ship nears the port, by proposing a detection method based on an optimized FPN model. The feature pyramid model is first embedded in a traditional region proposal network (RPN) and mapped into a new feature space for object identification. Subsequently, the k-means clustering algorithm based on the shape similar distance (SSD) measure is used to optimize the FPN. Initial anchor boxes and tests are created using the SAR ship dataset. Experimental results show that the proposed algorithm for object detection shows an accuracy of 98.62%. Compared with Yolo, the RPN based on VGG/ResNet, FPN based on VGG/ResNet, and other models in complex scenarios, the proposed model shows a higher accuracy rate and better overall performance.

Small object detection on drone-captured images is a recently popular and challenging task. From the drone’s perspective, the object scale varies significantly, and tiny objects lack distinguishable appearance information in complex backgrounds, which leads to undesired effects. To solve the issue mentioned above, we propose U-YOLO based on the original YOLOv5 model. We first extend the multi-scale feature fusion network and add a detection head for tiny objects. Secondly, we integrate the convolutional block attention model (CBAM) in the detection head to focus on the critical region of the feature map. Lastly, an attention feature fusion module based on contextual information is designed to combine local and global contextual details of small objects and attention mechanisms to enhance the multi-scale feature fusion capability. Experiments on dataset VisDrone2021-DET show that U-YOLO not only improves the detection performance on drone-captured scenarios but also has a real-time detection speed. Compared to baseline model (YOLOv5s), the mAP result of U-YOLO is increased from 19.07 to 24.1%, and the detection speed is 39FPS. It provides a good balance between detection accuracy and speed, promoting the progress of small object detection algorithms on UAV platforms.

Object detection is an important yet challenging task in video understanding & analysis, where one major challenge lies in the proper balance between two contradictive factors: detection accuracy and detection speed. In this paper, we propose a new adaptive patch-of-interest composition approach for boosting both the accuracy and speed for object detection. The proposed approach first extracts patches in a video frame which have the potential to include objects-of-interest. Then, an adaptive composition process is introduced to compose the extracted patches into an optimal number of sub-frames for object detection. With this process, we are able to maintain the resolution of the original frame during object detection (for guaranteeing the accuracy), while minimizing the number of inputs in detection (for boosting the speed). Experimental results on various datasets demonstrate the effectiveness of the proposed approach.

Object detection has become the cornerstone of various real-world applications, ranging from autonomous driving where it helps detect road objects to medical imaging for early disease prediction and gesture recognition systems. Due to their ability to operate continuously and cost-effectively, computer vision models are increasingly being employed in surveillance and monitoring tasks. Unlike humans, who can intuitively recognize and understand objects within images, machines require advanced algorithms to mimic such capabilities. A key challenge in computer vision lies in recognizing and tracking objects in real-time with accuracy and reliability. Human visual systems perform these tasks naturally and swiftly, enabling complex activities like driving. For computers to match such performance, efficient object detection models along with additional hardware such as sensors are essential. This study presents a comparative analysis of two popular deep learning-based object detection algorithms You Only Look Once (YOLO) and Faster Region-based Convolution Neural Network (Faster RCNN). Both models are evaluated based on their detection accuracy, speed, and performance in real-time scenarios. The paper aims to highlight the strengths and limitations of each model, offering insights into their suitability for different applications. The findings suggest that while both algorithms have their merits, the choice between them depends on the specific requirements of a task, such as the trade-off between detection speed and precision. Comparative study shows that Faster RCNN outperforms its accuracy compared with YOLO.

Deep learning-based detection from the perspective of small or tiny objects: A survey