DeLiVoTr: Deep and light-weight voxel transformer for 3D object detection

Existing DEtection TRansformer-based (DETR) object detection methods have been widely applied to standard object detection tasks, but still face numerous challenges in detecting small objects. These methods frequently miss the fine details of small objects and fail to preserve global context, particularly under scale variation or occlusion. The resulting feature maps lack sufficient spatial and structural information. Moreover, some DETR-based models specifically designed for small object detection often have poor generalization capabilities and are difficult to adapt to datasets with diverse object scales and complex backgrounds. To address these issues, this paper proposes a novel object detection model—small object detection with efficient multi-scale collaborative attention and depth feature fusion based on DETR (ED-DETR)—which consists of three core modules: an efficient multi-scale collaborative attention mechanism (EMCA), DepthPro, a zero-shot metric monocular depth estimation model, and an adaptive feature fusion module for depth maps and feature maps. Specifically, EMCA extends the single-space attention mechanism in efficient multi-scale attention (EMA) to a composite structure of parallel spatial and channel attention, enhancing ED-DETR’s ability to express features collaboratively in both spatial and channel dimensions. DepthPro generates depth maps to extract depth information. The adaptive feature fusion module integrates depth information with RGB visual features, improving ED-DETR’s ability to perceive object position, scale, and occlusion. The experimental results show that ED-DETR achieves the current best 33.6% mAP on the AI-TOD-V2 dataset, which predominantly contains tiny objects, outperforming previous CNN-based and DETR-based methods, and shows excellent generalization performance on the VisDrone and COCO datasets.

Small object detection is a hard task to be sovled in object detection because the small object usually contain a few effective informations, which makes it very difficult to extract the discriminative features of small scale object. In addition, a key challenge in deep network for small object detectors is the large scale variation, where the performance of small object is poor due to the imbalanced optimization on various scale. In this paper, we propose a scale adaptive balance mechanism based on anchor-free framework for small object detection. Specifically, we design a new loss function to adapt the detection model to various object scale, which is able to alleviate the imbalance of object scale. Moreover, in feature extractor, the shallow feature maps of different receptive fields are added to the DLA-34 backbone for better extracting multi-scale features. Besides, the image block is used as a detection unit during inference, and the scale mismatch is thus overcome. Finally, the proposed method is verified on the Visdrone2019 dataset and the experimental results validate the effectiveness to detect small objects in the video image from drone monitoring.

Advanced driver assistance systems (ADAS) mainly consist of three components: environmental perception, decision planning, and motion control. As a fundamental component of the ADAS environmental perception system, 3D object detection enables vehicles to avoid obstacles and ensure driving safety only through accurate and real‐time prediction and localization of three‐dimensional targets such as vehicles and pedestrians in road scenes. Therefore, to improve both the real‐time performance and accuracy of 3D object detection, we propose a lightweight depth prediction‐based 3D object detection model with multi‐scale fusion—SPWS‐Transformer. First, to enhance the model's accuracy, we propose a feature extraction network incorporating multi‐scale feature fusion and depth prediction. By designing a multi‐scale feature fusion module, we effectively combine multi‐scale semantic and fine‐grained information from feature maps of different scales to enhance the network's feature extraction capability. To capture spatial information from the feature maps, we apply convolution, group normalization, and nonlinear activation operations on the fused feature maps to generate depth feature maps. Both the fused feature maps and depth feature maps serve as inputs for subsequent network stages. To further improve accuracy, we leverage the long‐range modelling advantages of Transformers by designing a feature enhancement encoder to strengthen the representation capability of depth feature maps. We incorporate a dilated encoder to perform positional encoding on depth feature maps and utilize multi‐head self‐attention mechanisms to capture contextual relationships within the input scene, thereby enhancing the detection capability of the 3D object detection network. Then, to improve real‐time performance, we design a decoder structure with scale‐aware attention. By predefining masks of different scales, we adaptively learn a scale‐aware filter using depth and visual features to enhance object queries. Finally, on the KITTI dataset, the improved algorithm achieves an AP of 24.66% for the car category, with more significant improvements in detection accuracy under the ‘hard’ difficulty level. The model achieves an inference time of 24 ms.

Small object detection is always a hot research direction in the field of computer vision, and widely used in traffic, production, industry, and other fields. A neural network for detecting small objects based on original Cascade RCNN is proposed. First, we modify the traditional feature pyramid network and introduce multi-branch dilated convolutions to enhance the feature information of the small target. Second, the feature extraction networks in original Cascade RCNN are replaced with multi-layer deformable convolution networks, which can better adapt to the geometric variations of detection target and huge size span of the objects in the same scene. Finally, Soft non-maximum suppression is also integrated into the network to avoid problems in dense object detection. To verify the practicability of our proposed network, we apply it to the garbage detection system. The experimental results show that compared with original Cascade RCNN and other commonly used target detection networks, our proposed method, with a recall rate >0.98 as well as mean average precision up to 0.964, performs better not only in small object detection but also in industrial applications.

Small object detection has important application value in the fields of autonomous driving and drone scene analysis. As one of the most advanced object detection algorithms, YOLOv3 suffers some challenges when detecting small objects, such as the problem of detection failure of small objects and occluded objects. To solve these problems, an improved YOLOv3 algorithm for small object detection is proposed. In the proposed method, the dilated convolutions mish (DCM) module is introduced into the backbone network of YOLOv3 to improve the feature expression ability by fusing the feature maps of different receptive fields. In the neck network of YOLOv3, the convolutional block attention module (CBAM) and multi-scale fusion module are introduced to select the important information for small object detection in the shallow network, suppress the uncritical information, and use the fusion module to fuse the feature maps of different scales, so as to improve the detection accuracy of the algorithm. In addition, the Soft-NMS and Complete-IOU (ClOU) strategies are applied to candidate frame screening, which improves the accuracy of the algorithm for the detection of occluded objects. The experimental results on MS COCO2017, VOC2007, VOC2012 datasets and the ablation experiments on MS COCO2017 datasets demonstrate the effectiveness of the proposed method.The experimental results show that the proposed method achieves better accuracy in small object detection than the original YOLOv3 model.

Accurate object detection in unmanned aerial vehicle (UAV) images plays a crucial role in fields such as aviation, transportation, and agriculture. However, UAV images often contain a high proportion of small objects, and the limited resources of UAV platforms make it challenging for existing small object detection algorithms to balance detection performance and resource consumption. To address these issues, a lightweight small object detection algorithm called SRP-YOLO is proposed. Compared to the YOLOv8 network, the following improvements are made: first, a high-resolution detection head is designed; second, a receptive field attention convolution module tailored for small UAV objects is introduced to replace the standard convolution module, enhancing the detection capability for small objects and significantly improving their feature representation; Third, partial convolution is combined with C2f to replace some of the standard convolutions, fully leveraging the high-resolution information from shallow features. Finally, A normalized Gaussian Wasserstein Distance (NWD) metric is introduced to reduce the sensitivity of Intersection over Union (IoU) to minor positional deviations of small objects. On VisDrone-DET2019, SRP-YOLO reduces parameters while improving mAP 0.5 by 7.9%. Generalization tests on TinypersonV2 and RAIVD confirm significant gains in small object detection.

Video object detection is essential for airport surface surveillance, but the objects on the scene are mostly small objects with low resolution, they have no obvious feature information. Due to the scale differences of the objects and the fixed receptive field on the feature maps, detectors cannot model multi‐scale context information and cover all objects. In addition, although the video detection algorithm can be used as a method to solve the problem of small object detection, the temporal feature fusion method of current video detection is very dependent on the quality of a single feature map. Therefore, this paper aims to enhance the features of small objects of a single image. First, an attentional multi‐scale feature fusion enhancement (A‐MSFFE) network is built on the memory‐enhanced global‐local aggregation (MEGA) to supplement semantic and spatial information of small objects. Then, a context feature enhancement (CFE) module is designed for obtaining different receptive fields through different dilated convolutions. Meanwhile, a video detection dataset about the airport is established. Finally, the experimental results show that the proposed method can improve the detection accuracies of small objects and outperform other state‐of‐the‐art video object detection algorithms in self‐built airport dataset.

Accurate object detection is a fundamental requirement for autonomous systems to operate in dynamic urban environments. Considering the intricacy of the environment and the occlusion of objects, various point cloud based three-dimensional (3D) object detection methods have been proposed, such as point-based or voxel-based methods. In this paper, a feature selection mechanism is proposed in a voxel-based method to generate bird’s eye view (BEV) feature maps from the original point cloud. For the 3D Backbone, the U-shaped structure and single scale feature selection module are combined. After combining high-level semantics and low-level fine-grained features, the optimized features after the BEV feature maps are applied to region of interest (RoI) refinement, so that the voxel features can better serve the subsequent 3D object detection. The experimental results on KITTI dataset show higher 3D object detection accuracy compared to the state-of-the-art 3D detection methods, which reflects the effectiveness of the proposed architecture.

As a task in object detection, small object detection mainly focuses on detecting objects of small size, which is more complex than general object detection. It is pivotal in various applications, e.g., small tumor detection, national defense and security, and traffic surveillance. Small objects have low pixels, few effective features, and a large influence of background noise, making small object detection extremely challenging. Currently, most object detection algorithms fail to take advantage of global context information to improve accuracy. Moreover, the conventional bounding-box proposal generation method will cause the missed detection of small targets since the target features are few and challenging to locate. To address the above problems, this paper uses the position enhancement method to improve the proposals’ generation to improve the recall rate and accuracy rate of small object detection. First, the location enhancement module adds additional keypoint location supervision to obtain target latent keypoints. Besides, we propose a keypoint expansion method to get more accurate keypoint locations. A global contextual attention mechanism is further introduced, enabling the detector to learn fine-grained features with contextual location information better. The experimental results on two datasets show that the proposed method can significantly improve the accuracy and recall of small target detection.

Transformer-based methodologies in object detection have recently piqued considerable interest and have produced impressive results. DETR, an end-to-end object detection framework, ingeniously integrates the Transformer architecture, traditionally used in NLP, into computer vision for sequence-to-sequence prediction. Its enhanced variant, DINO, featuring improved denoising anchor boxes, has showcased remarkable performance on the COCO val2017 dataset. However, it often encounters challenges when applied to scenarios involving small object detection. Thus, we propose an innovative method for feature enhancement tailored to recursive prediction tasks, with a particular emphasis on augmenting small object detection performance. It primarily involves three enhancements: refining the backbone to favor feature maps that are more sensitive to small targets, incrementally augmenting the number of queries for small objects, and advancing the loss function for better performance. Specifically, The study incorporated the Switchable Atrous Convolution (SAC) mechanism, which features adaptable dilated convolutions, to increment the receptive field and thus elevate the innate feature extraction capabilities of the primary network concerning diminutive objects. Subsequently, a Recursive Small Object Prediction (RSP) module was designed to enhance the feature extraction of the prediction head for more precise network operations. Finally, the loss function was augmented with the Normalized Wasserstein Distance (NWD) metric, tailoring the loss function to suit small object detection better. The efficacy of the proposed model is empirically confirmed via testing on the VISDRONE2019 dataset. The comprehensive array of experiments indicates that our proposed model outperforms the extant DINO model in terms of average precision (AP) small object detection.

In the recent 3D object detection methods for point clouds, the combination of point-based methods and voxel-based methods is gradually becoming a trend. Point-based methods retain the accurate position and pose information in the raw points and voxel-based methods get multi-scale structure information through the 3D backbone. However, because of the sparsity and irregularity of point clouds, both representations ignore the context information, which is important for the detection of sparse and small objects. To solve this problem, we propose a multi-stream feature aggregation network to extract features from three representations of the point cloud for object detection. Specifically, we exploit multi-stream features extracted from point, voxel, and perspective view (PV) respectively on a parallel way, where the complementary information between different perspectives can be used to enrich the feature representations, especially for the perspective view containing rich semantic context information. Secondly, to eliminate redundant information and better exploit the correlation between different feature representations, we design an attention-based multi-stream feature fusion module (MSFF) to combine features from three information streams. Besides, we introduce a new voxel RoI pooling with the self-attention in the second refinement stage, which can further strengthen the connection between local features in the proposal to obtain accurate classification and localization predictions. Our method achieves progressive results on the KITTI dataset, especially in the cyclist category, which improves the baseline significantly by 5.56%, 4.73%, 5.16% AP in the test set for easy, moderate, and hard levels respectively. Code will be available at https://github.com/june2678/MR F.

A case of dominant multiplicative noise typical for radar images is considered. The filtering of such images corrupted by speckle, and possibly spikes, can be done both without and with the use of homomorphic transformations. The application of locally adaptive filtering is explored for both cases. It is shown that the use of homomorphic transformations is reasonable for solving some particular tasks, for example, better detection of small size objects with negative contrasts with respect to surrounding background and their preservation while processing the images. Other kinds of homomorphic filtering techniques can be useful for providing better noise suppression.

With the development of remote sensing technology, remote sensing object detection has been widely applied in various fields, but it still faces some thorny challenges, such as the following: 1) the complexity of object scale changes in remote sensing images makes it difficult to improve the performance of small object detection and 2) remote sensing images have complex backgrounds and densely arranged small and weak objects, which pose a serious problem of feature interference. To alleviate these challenges, we propose an end-to-end adaptive feature separation network called AFSNet, which includes a scale-aware module (SAM) and a class-aware module (CAM). The SAM mainly enables feature maps of different resolutions to detect objects of different scales. Shallow feature maps mainly suppress the features of large objects they contain to focus on small object detection, while deep feature maps increase the detailed features of large objects they contain to focus on large object detection. The CAM is mainly used to distinguish the features in the feature map by category, separating the features of different categories into different channels, thus mitigating the problem of inter class feature interference, and blocking background interference. The effectiveness of this article has been proven on the NWPU VHR-10, IPIU-M, DIOR, and DOTA2.0 datasets. It can be widely applied in civilian, military, and other fields. Through experimental verification, our AFSNet achieved 97.70% mAP on the NWPU VHR-10 dataset, 78.9% mAP on the DIOR dataset, and 58.22% mAP on the DOTA2.0 dataset. Our code is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Xidian-AIGroup190726/AFSNet</uri>.

Abandoned object detection is a critical task in the field of public safety. However, existing methods perform poorly when detecting small and occluded objects, leading to high false detection and missed detection rates. To address this issue, this paper proposes an abandoned object detection method that integrates an adaptive dual-background model with SAO-YOLO (Small Abandoned Object YOLO). The goal is to reduce false and missed detection rates for small and occluded objects, thereby improving overall detection accuracy. First, the paper introduces an adaptive dual-background model that adjusts according to scene changes, reducing noise interference in the background model. When combined with an improved PFSM (Pixel-based Finite State Machine) model, this enhances detection accuracy and robustness. Next, a network model called SAO-YOLO is designed. Key improvements within this model include the SAO-FPN (Small Abandoned Object FPN) feature extraction network, which fully extracts features of small objects, and a lightweight decoupled head, SODHead (Small Object Detection Head), which precisely extracts local features and enhances detection accuracy through multi-scale feature fusion. Finally, experimental results show that SAO-YOLO increases mAP@0.5 and mAP@0.5:0.95 by 9.0% and 5.1%, respectively, over the baseline model. It outperforms other advanced detection models. Ultimately, after a series of experiments on the ABODA, PETS2006, and AVSS2007 datasets, the proposed method achieved an average detection precious of 91.1%, surpassing other advanced methods. It significantly outperforms other advanced detection methods. This approach notably reduces false and missed detections, especially for small and occluded objects.

As one type of object detection, small object detection has been widely used in daily-life-related applications with many real-time requirements, such as autopilot and navigation. Although deep-learning-based object detection methods have achieved great success in recent years, they are not effective in small object detection and most of them cannot achieve real-time processing. Therefore, this paper proposes a single-stage small object detection network (SODNet) that integrates the specialized feature extraction and information fusion techniques. An adaptively spatial parallel convolution module (ASPConv) is proposed to alleviate the lack of spatial information for target objects and adaptively obtain the corresponding spatial information through multi-scale receptive fields, thereby improving the feature extraction ability. Additionally, a split-fusion sub-module (SF) is proposed to effectively reduce the time complexity of ASPConv. A fast multi-scale fusion module (FMF) is proposed to alleviate the insufficient fusion of both semantic and spatial information. FMF uses two fast upsampling operators to first unify the resolution of the multi-scale feature maps extracted by the network and then fuse them, thereby effectively improving the small object detection ability. Comparative experimental results prove that the proposed method considerably improves the accuracy of small object detection on multiple benchmark datasets and achieves a high real-time performance.