ODC-YOLO: an optimized YOLOv5 method for detecting objects in remote sensing images

Addressing the challenge of efficiently detecting objects in ultra-high-resolution images during object detection tasks, this paper proposes a novel method called MegaDetectNet, which leverages foreground image for large-scale resolution image object detection. MegaDetectNet utilizes a foreground extraction network to generate a foreground image that highlights target regions, thus avoiding the computationally intensive process of dividing the image into multiple sub-images for detection, and significantly improving the efficiency of object detection. The foreground extraction network in MegaDetectNet is built upon the YOLOv5 model with modifications: the large object detection head and classifier are removed, and the PConv convolution is introduced to reconstruct the C3 module, thereby accelerating the convolution process and enhancing foreground extraction efficiency. Furthermore, a Res2Rep convolutional structure is developed to enlarge the receptive field and improve the accuracy of foreground extraction. Finally, a foreground image construction method is proposed, fusing and stitching foreground target regions into a unified foreground image. This approach replaces multiple divided sub-images with a single foreground image for detection, reducing overhead time. The proposed MegaDetectNet method’s effectiveness for detecting objects in ultra-high-resolution images is validated using the publicly available DOTA dataset. Experimental results demonstrate that MegaDetectNet achieves an average time reduction of 83.8% compared to the sub-image division method among various commonly used object detectors, with only a marginal 8.7% decrease in mAP (mean Average Precision). This validates the practicality and efficacy of the MegaDetectNet method for object detection in ultra-high-resolution images.

It is challenging to accurately detect flexible objects with arbitrary orientation from monitoring images in power grid maintenance and inspection sites. This is because these images exhibit a significant imbalance between the foreground and background, which can lead to low detection accuracy when using a horizontal bounding box (HBB) as the detector in general object detection algorithms. Existing multi-oriented detection algorithms that use irregular polygons as the detector can improve accuracy to some extent, but their accuracy is limited due to boundary problems during the training process. This paper proposes a rotation-adaptive YOLOv5 (R_YOLOv5) with a rotated bounding box (RBB) to detect flexible objects with arbitrary orientation, effectively addressing the above issues and achieving high accuracy. Firstly, a long-side representation method is used to add the degree of freedom (DOF) for bounding boxes, enabling accurate detection of flexible objects with large spans, deformable shapes, and small foreground-to-background ratios. Furthermore, the further boundary problem induced by the proposed bounding box strategy is overcome by using classification discretization and symmetric function mapping methods. Finally, the loss function is optimized to ensure training convergence for the new bounding box. To meet various practical requirements, we propose four models with different scales based on YOLOv5, namely R_YOLOv5s, R_YOLOv5m, R_YOLOv5l, and R_YOLOv5x. Experimental results demonstrate that these four models achieve mean average precision (mAP) values of 0.712, 0.731, 0.736, and 0.745 on the DOTA-v1.5 dataset and 0.579, 0.629, 0.689, and 0.713 on our self-built FO dataset, exhibiting higher recognition accuracy and a stronger generalization ability. Among them, R_YOLOv5x achieves a mAP that is about 6.84% higher than ReDet on the DOTAv-1.5 dataset and at least 2% higher than the original YOLOv5 model on the FO dataset.

Detecting oriented objects in optical remote sensing images has been consistently challenging due to difficulties in bounding boxes localization. The cascaded regression framework, widely employed for high-quality bounding box refinement, has demonstrated effectiveness in this domain. However, our experiments reveal a discontinuity issue in bounding box optimization in cascaded regression framework. As a result, performance gain is not guaranteed across all stages in this framework. In this paper, we propose a Distribution Discriminative Detector(DDDet) to address the above issues and enhance the optimization of bounding boxes in oriented object detection. Specifically, a novel Conditional Anchor Refinement Framework(CARF) is designed to improve cascaded regression structure. CARF distinguishes bounding boxes with different distributions, adaptively optimizing them within the well-assigned regressors. Subsequently, the Aligned Convolution Module(ACM) is integrated into each regressor, facilitating the continuous alignment between features and refined anchors. Furthermore, the Geometry-guided Training Sample Selection(GTSS) method is incorporated into CARF to assign labels based on object shape priors. Experimental results show that DDDet obtains state-of-the-art performance on mainstream datasets for oriented object detection in remote sensing image, which demonstrates the effectiveness of the proposed method. Our method surpasses many current single-stage detectors, two-stage detectors, and refine-stage detectors, achieving the mAP of 79.41% on DOTA dataset, and 44.15% on FAIR1M dataset.

Forest fires have continually endangered personal safety and social property. To reduce the occurrences of forest fires, it is essential to detect forest fire smoke accurately and quickly. Traditional forest fire smoke detection based on convolutional neural networks (CNNs) needs many hand-designed components and shows poor ability to detect small and inconspicuous smoke in complex forest scenes. Therefore, we propose an improved early forest fire smoke detection model based on deformable transformer for end-to-end object detection (deformable DETR). We use deformable DETR as a baseline containing the best sparse spatial sampling for smoke with deformable convolution and relation modeling capability of the transformer. We integrate a Multi-scale Context Contrasted Local Feature module (MCCL) and a Dense Pyramid Pooling module (DPPM) into the feature extraction module for perceiving features of small or inconspicuous smoke. To improve detection accuracy and reduce false and missed detections, we propose an iterative bounding box combination method to generate precise bounding boxes which can cover the entire smoke object. In addition, we evaluate the proposed approach using a quantitative and qualitative self-made forest fire smoke dataset, which includes forest fire smoke images of different scales. Extensive experiments show that our improved model’s forest fire smoke detection accuracy is significantly higher than that of the mainstream models. Compared with deformable DETR, our model shows better performance with improvement of mAP (mean average precision) by 4.2%, APS (AP for small objects) by 5.1%, and other metrics by 2% to 3%. Our model is adequate for early forest fire smoke detection with high detection accuracy of different-scale smoke objects.

High-precision remote sensing image object detection has broad application prospects in military defense, disaster emergency, urban planning, and other fields. However, the arbitrary orientation, dense arrangement, and small size of objects in remote sensing images lead to poor detection accuracy of existing methods. To achieve accurate detection, this paper proposes an arbitrary directional remote sensing object detection method, called FANet, based on feature fusion and angle classification. Initially, the angle prediction branch is introduced, and the circular smooth label method is used to transform the angle regression problem into a classification problem, which solves the difficult problem of abrupt changes in the boundaries of the rotating frame while realizing the object frame rotation. Subsequently, to extract robust remote sensing objects, innovative introduce pure convolutional model as a backbone network, while Conv is replaced by GSConv to reduce the number of parameters in the model along with ensuring detection accuracy. Finally, the strengthen connection feature pyramid network (SC-FPN) is proposed to redesign the lateral connection part for deep and shallow layer feature fusion, and add jump connections between the input and output of the same level feature map to enrich the feature semantic information. In addition, add a variable parameter to the original localization loss function to satisfy the bounding box regression accuracy under different IoU thresholds, and thus obtain more accurate object detection. The comprehensive experimental results on two public datasets for rotated object detection DOTA and HRSC2016 demonstrate the effectiveness of our method.

Object detection has a great significance to remote sensing image recognition. Conventional object detection methods by using horizontal bounding box have shown good performance in general images. However, because of looking down perspective of remote sensing images, rotation bounding box is more suitable and precise for object detection. Although rotation bounding box has been researched by some works for remote sensing images, most of them are still two stages. In this paper, by improvement of YOLO architecture, we propose an one-stage object detection method to predict rotation bounding box for remote sensing images. Firstly, in the data preprocessing stage, an image cropping operation is utilized instead of zooming operation to generate suitable input data for network. Meanwhile, in order to avoid an object from being divided, an overlapping band is set during cropping. Secondly, a novel rotation bounding box representation method is introduced, and a corresponding loss function is designed in training process. Experiments on the Dota dataset demonstrate that our method outperforms state-of-the-art rotation objects detection methods, in terms of mAP, our method achieves 73.50%mAP, which is higher than methods based on rotation bounding box by 2.0%.

Withcontinuous advancements in remote sensing technology, object detection in remote sensing images has emerged as a critical research direction in maritime surveillance, port management, and national defense. Among these applications, ship detection is a key task. Due to the fact that ships in images typically exhibit arbitrary rotations, multi-scale distributions, and complex backgrounds, conventional detection methods based on horizontal or rotated bounding boxes often fail to adequately capture the fine-grained information of the targets, thereby compromising detection accuracy. This paper proposes the Dense Representative Points-Guided Rotated-Ship Detection (DenseRRSD) method. The proposed approach represents ship objects using dense representative points (RepPoints) to effectively capture local semantic information, thereby avoiding the background noise issues associated with traditional rectangular bounding box representations. To further enhance detection accuracy, an edge region sampling strategy is devised to uniformly sample RepPoints from critical ship parts, and a Weighted Residual Feature Pyramid Network (WRFPN) is introduced to efficiently fuse the multi-scale features through residual connections and learnable weights. In addition, a Weighted Chamfer Loss (WCL) combined with a staged localization loss strategy is employed to progressively refine localization from coarse to fine stages. Experimental results on both the HRSC2016 dataset and the newly constructed DOTA-SHIP dataset demonstrate that DenseRRSD achieves state-of-the-art detection accuracy, with mean Average Precision (mAP) scores of 91.2% and 83.2%, respectively, significantly outperforming existing methods. These results verify the effectiveness and robustness of the proposed approach in rotated-ship detection under diverse conditions.

Visual sensors such as optical camera possess fatal shortcomings like vulnerable to the weather condition and light intensity, which may cause catastrophic disasters. Fortunately, the prevalent 3D sensors like Light Detection and Ranging (Lidar) can overcome these drawbacks. Fusing both monocular camera and 3D Lidar data can often achieve a better performance than solely using one of them. In this paper, we propose an object detection approach by fusing monocular camera data and Lidar data in decision level based on Dempster-Shafer evidence theory. First, the 3D point cloud collected from Lidar is projected onto the image plane and the upsampling method is adopted to receive dense depth maps. Then, the RGB images and dense depth maps are separately fed into YOLOv3 detection framework to extract the object features. Next, the final detection attributes, the class confidence and bounding box, are generated by using Dempster's combination rule and extracting intersection of the associated bounding boxes. Finally, the mean Average Precision (mAP) performance of the proposed method is improved by 0.5% and 3.1% compared with sigle camera method and Lidar method, respectively.

Object detection is a fundamental task in computer vision. Bounding box regression (BBR) losses are critical to detector performance. However, evaluation measures that rely on the Intersection over Union (IoU) between the predicted and ground truth boxes are highly sensitive to positional deviations, which can hinder optimization. To alleviate this issue, we propose ATDIoU, a novel arctangent-differential loss for bounding-box regression. ATDIoU computes distance similarity between a predicted and a ground truth box by modeling the distances between their corresponding vertices as a two-dimensional arctangent differential distribution (ATD). This arctangent differential-based design mitigates bounding box drift and reduces sensitivity to localization errors. As a result, it guides the model to learn target positions more effectively. We evaluate ATDIoU by integrating it into YOLOv6 and conducting experiments on PASCAL VOC and VisDrone2019. The results demonstrate that ATDIoU yields improvements of 1.4% and 0.7% in mean average precision (mAP) relative to MPDIoU.

Automatic detection of weapons is significant for improving security and well being of individuals, nonetheless, it is a difficult task due to large variety of size, shape and appearance of weapons. View point variations and occlusion also are reasons which makes this task more difficult. Further, the current object detection algorithms process rectangular areas, however a slender and long rifle may really cover just a little portion of area and the rest may contain unessential details. To overcome these problem, we propose a CNN architecture for Orientation Aware Weapons Detection, which provides oriented bounding box with improved weapons detection performance. The proposed model provides orientation not only using angle as classification problem by dividing angle into eight classes but also angle as regression problem. For training our model for weapon detection a new dataset comprising of total 7801 weapons images is gathered from the web and then manually annotated with position oriented bounding boxes. Our dataset provides not only oriented bounding box as ground truth but also horizontal bounding box. We also provide our dataset in multiple formats of modern object detectors for further research in this area. The proposed model is evaluated on this dataset, and the comparative analysis with off-the shelf object detectors yields superior performance of proposed model, measured with standard evaluation strategies. The dataset and the model implementation are made publicly available at this link: https://bit.ly/2TyZICF .

Background and Objective: Ensuring safety in high-risk environments like industrial zones and construction zones is a basic requirement. Here, the use of helmets is a mandatory protocol to protect the workers from head injuries. But even if strict rules have been complied with in these industrial areas, real-world compliance is inconsistent due to negligence and ignorance. Traditional methods of surveillance-based monitoring are both resource-intensive as well as can be easily prone to human oversight. This research proposes a transformer-based deep learning solution using DETR (DEtection TRansformer) for automated helmet detection. This aims to modernise and automate safety compliance monitoring systems. Methods: The proposed system leverages DETR, a novel object detection architecture that combines convolutional neural networks with Vision Transformers to eliminate the need for region proposal networks and non-maximum suppression. We calibrate DETR on a publicly available Kaggle dataset consisting of 764 images annotated in PASCAL VOC format across two classes: With Helmet and Without Helmet. The model was trained using the PyTorch framework, with strategic data augmentation applied to enhance generalisation. Performance was evaluated using mean Average Precision (mAP), precision, recall, and inference time. Results: The DETR-based model achieved promising results in distinguishing helmet usage, delivering high accuracy and reliable localisation even with a relatively small dataset. The proposed attention-based model achieved competitive results in hard cases, where it performed better than the traditional CNN-based approaches. Conclusion: The study demonstrates that transformer-based object detection systems are successful in meeting real-world safety compliance tasks. The DETR architecture provides a scalable intelligence-based system for real-time helmet monitoring and can be used in automated surveillance systems in various industrial scenarios.

Object localization and detection is the emerging area of computer vision and image processing that finds out the location of the object in the video frames or in the digital images. Object localization and detection have many challenges like occlusion, scale variations, intraclass similarities, illumination conditions, pose variations, etc. Multi-view object detection focuses on different views of objects like the top, bottom, side view, etc. We aim to discuss the different Object detection models based on Machine learning and Deep neural network approaches in a multi-view environment. We also discuss different datasets used for object detections and their applications. We apply some object detection models like SSD and Faster R-CNN to specific object categories of Open Image Dataset V6 and compare the results based on mAP (Mean Average Precision).KeywordsObject detectionConvolution neural networksR-CNNFast R-CNNFaster R-CNNEfficientDetCenterNet

Supervised object detection models are trained to recognize certain objects. These models are classified into two types: single-stage detectors and two-stage detectors. The single-stage detectors just need one pass through the model to anticipate all the bounding boxes, whereas the two-stage detectors require to first estimate the image portions where the object could be located. Due to their speed and simplicity, single-stage anchor-based models are used in many industrial settings. Training such models require bounding boxes that describe the spatial location of an object, which are usually drawn by an expert. However, the question remains, how much area should be considered when drawing the bounding boxes? In this paper, we demonstrate the effects that the size and placement of a rectangular bounding box can have on the performance of the anchor-based models. For this, we first perform experiments on a synthetically generated binary dataset and then on a real-world object detection dataset. Our results show that fixing the size of the bounding boxes can help in improving the performance of the model in the case of single class object detection (approximately 50% improvement in mAP@[.5:.95] for real world dataset). Furthermore, we also demonstrate how freely available tools can be combined for obtaining the best possible semi automated object labeling pipeline.

Current mainstream remote sensing target detection algorithms mostly estimate the rotation angle of targets by designing different bounding box descriptions and loss functions. However, they fail to consider the symmetry–asymmetry duality anisotropy in the distribution of key features required for target localization. Moreover, the equivalent feature extraction mode of shared convolutional kernels may lead to difficulties in accurately predicting parameters with different attributes, thereby reducing the performance of the detector. In this paper, we propose the Feature Equalization and Hierarchical Decoupling Network (FEHD-Net), which comprises three core components: a Symmetry-Enhanced Parallel Interleaved Convolution Module (PICM), a Parameter Decoupling Module (PDM), and a Critical Feature Matching Loss Function (CFM-Loss). PICM captures diverse spatial features over long distances by integrating square convolution and multi-branch continuous orthogonal large kernel strip convolution sequences, thereby enhancing the network’s capability in processing long-distance spatial information. PDM decomposes feature maps with different properties and assigns them to different regression branches to estimate the parameters of the target’s rotating bounding box. Finally, to stabilize the training of anchors with different qualities that have captured the key features required for detection, CFM-Loss utilizes the intersection ratio between anchors and true value labels, as well as the uncertainty of convolutional regression during training, and designs an alignment criterion (symmetry-aware alignment) to evaluate the regression ability of different anchors. This enables the network to fine-tune the processing of templates with different qualities, achieving stable training of the network. A large number of experiments demonstrate that compared with existing methods, FEHD-Net can achieve state-of-the-art performance on DOTA, HRSC2016, and UCAS-AOD datasets.

Real-time video surveillance, especially CCTV systems, requires fast and accurate face detection. Object detection models with slow inference times are ineffective in real-time. This study addresses this challenge by improving the inference speed of the YOLOv8 model, a leading object detection framework known for its accuracy and speed. We focus on pruning the model's architecture, particularly the P5 head section, which detects larger objects. According to Bochkovskiy's 2020 research, this modification enhances the model's performance specifically for medium and small objects in CCTV footage. The standard YOLOv8 model and its modified version were compared for inference time, mean Average Precision (mAP), and model weight. The pruned YOLOv8 model cuts inference time by 15.56%, from 4.5 ms to 3.8 ms, and reduces model weight. The advantages mentioned above are offset by a 1.6% decrease in mean average precision. This research advances object detection technology by demonstrating architectural modifications' efficacy. These changes make the model faster and lighter, making it suitable for real-time surveillance. The accuracy trade-off is slight. The implications of these findings are crucial for implementing efficient object detection systems in CCTV surveillance. These findings also lay the groundwork for future research to improve such systems' speed-accuracy trade-off.