Multi-scale Feature Fusion UAV Image Object Detection Method Based on Dilated Convolution and Attention Mechanism

The performance of deep neural networks in object detection is impressive, however, direct use of existingalgorithms may not obtain satisfactory resultsdue to the characteristics of underwater images. In this paper, a multi-scale feature fusion Single Shot Multibox Detector (MFFSSD) based on the SSD algorithm is proposed to improve the accuracy of underwater object detection. There are two main improvements: introducing attention module and perform multiscale feature fusion. By introducing attention module, the features of the input image obtained through convolution are weighted in both channel and space aspect, so as to highlight the useful informationto overcomes the problem of small and blur objects which are common in underwater images. By extracting a lower level feature map forward, more details of the lower level image can be obtained to adapt to the underwater object with simple shape, so that low-level features have a higher contribution to the recognition object; at the same time, the high-level feature map is enlarged by using deconvolution, and then fused with the low-level feature map to introduce more contextual information. The fusion of the features extracted from the shallow and high level networks ensures the efficiency of the features to the greatest extent. Experiments on URPC show that MFFSSD significantly improved the underwater object recognitionaccuracy compared to the original algorithm.

In view of the fact that the feature layers of different scales in the traditional single shot multibox detector (SSD) are independent of each other, resulting in poor detection performance for small objects. We propose an improved SSD network for small object detection based on dilated convolution and feature fusion, which is called DFSSD. Specifically, by introducing dilated convolution, we enhance the receptive field of the third-level feature map in the network, which enables the feature map to obtain more global information. At the same time, we designed feature fusion module to fuse low-level feature map with detailed information and high-level feature map with rich semantic information. We adjust the prediction box scale of the DFSSD network prediction layer. Our proposed network obtains 78.9% mAP on PASCAL VOC2007 test at 40 FPS and 74.7% AP on difficult objects in car class of KITTI dataset. The results outperform the original SSD model by 1.4 and 1.2 points respectively.

With special consideration for complex scenes and densely distributed small objects, this frequently leads to serious false and missed detections for unmanned aerial vehicle (UAV) images in small object detection scenarios. Consequently, we propose a UAV image small object detection algorithm, termed SMA-YOLO. Firstly, a parameter-free simple slicing convolution (SSC) module is integrated in the backbone network to slice the feature maps and enhance the features so as to effectively retain the features of small objects. Subsequently, to enhance the information exchange between upper and lower layers, we design a special multi-cross-scale feature pyramid network (M-FPN). The C2f-Hierarchical-Phantom Convolution (C2f-HPC) module in the network effectively reduces information loss by fine-grained multi-scale feature fusion. Ultimately, adaptive spatial feature fusion detection Head (ASFFDHead) introduces an additional P2 detection head to enhance the resolution of feature maps to better locate small objects. Moreover, the ASFF mechanism is employed to optimize the detection process by filtering out information conflicts during multi-scale feature fusion, thereby significantly optimizing small object detection capability. Using YOLOv8n as the baseline, SMA-YOLO is evaluated on the VisDrone2019 dataset, achieving a 7.4% improvement in mAP@0.5 and a 13.3% reduction in model parameters, and we also verified its generalization ability on VAUDT and RSOD datasets, which demonstrates the effectiveness of our approach.

Unmanned aerial vehicles (UAVs) have been widely used in post-disaster search and rescue operations, object tracking, and other tasks. Therefore, the autonomous perception of UAVs based on computer vision has become a research hotspot in recent years. However, UAV images include dense objects, small objects, and arbitrary object directions, which bring about significant challenges to existing object detection methods. To alleviate these issues, we propose a global-local feature enhanced network (GLF-Net). Considering the difficulty of processing UAV images with complex scenes and dense objects, we designed a backbone based on an involution and self-attention that can extract effective features from complex objects. A multiscale feature fusion module is also proposed to address the presence of numerous small objects in UAV images through multiscale object detection and feature fusion. To accurately detect rotated objects, a rotated regional proposal network was designed based on the midpoint offset representation, which can apply a rotated box to determine the real direction and contour of an object. GLF-Net achieves a state-of-the-art detection accuracy (86.52% mAP) on our created RO-UAV dataset, while achieving 96.95% and 97% mAP on the public datasets HRSC2016 and UCAS-AOU, respectively. The experimental results demonstrate that our method achieves a high detection accuracy and generalization, which can meet the practical requirements of UAVs under various complex scenarios.

Existing DEtection TRansformer-based (DETR) object detection methods have been widely applied to standard object detection tasks, but still face numerous challenges in detecting small objects. These methods frequently miss the fine details of small objects and fail to preserve global context, particularly under scale variation or occlusion. The resulting feature maps lack sufficient spatial and structural information. Moreover, some DETR-based models specifically designed for small object detection often have poor generalization capabilities and are difficult to adapt to datasets with diverse object scales and complex backgrounds. To address these issues, this paper proposes a novel object detection model—small object detection with efficient multi-scale collaborative attention and depth feature fusion based on DETR (ED-DETR)—which consists of three core modules: an efficient multi-scale collaborative attention mechanism (EMCA), DepthPro, a zero-shot metric monocular depth estimation model, and an adaptive feature fusion module for depth maps and feature maps. Specifically, EMCA extends the single-space attention mechanism in efficient multi-scale attention (EMA) to a composite structure of parallel spatial and channel attention, enhancing ED-DETR’s ability to express features collaboratively in both spatial and channel dimensions. DepthPro generates depth maps to extract depth information. The adaptive feature fusion module integrates depth information with RGB visual features, improving ED-DETR’s ability to perceive object position, scale, and occlusion. The experimental results show that ED-DETR achieves the current best 33.6% mAP on the AI-TOD-V2 dataset, which predominantly contains tiny objects, outperforming previous CNN-based and DETR-based methods, and shows excellent generalization performance on the VisDrone and COCO datasets.

Small object detection on drone-captured images is a recently popular and challenging task. From the drone’s perspective, the object scale varies significantly, and tiny objects lack distinguishable appearance information in complex backgrounds, which leads to undesired effects. To solve the issue mentioned above, we propose U-YOLO based on the original YOLOv5 model. We first extend the multi-scale feature fusion network and add a detection head for tiny objects. Secondly, we integrate the convolutional block attention model (CBAM) in the detection head to focus on the critical region of the feature map. Lastly, an attention feature fusion module based on contextual information is designed to combine local and global contextual details of small objects and attention mechanisms to enhance the multi-scale feature fusion capability. Experiments on dataset VisDrone2021-DET show that U-YOLO not only improves the detection performance on drone-captured scenarios but also has a real-time detection speed. Compared to baseline model (YOLOv5s), the mAP result of U-YOLO is increased from 19.07 to 24.1%, and the detection speed is 39FPS. It provides a good balance between detection accuracy and speed, promoting the progress of small object detection algorithms on UAV platforms.

With the diversification of network traffic and the popularization of encrypted communication, traditional traffic classification methods have been difficult to cope with the challenges of new encryption protocols and hidden traffic. Deep learning, as a powerful feature extraction and classification tool, has been widely used in network traffic classification. In this paper, a deep network traffic classification model based on multi-scale feature fusion and attention mechanism (MSF-AttNet) is proposed. Features of different time scales are extracted through multi-scale convolution, and the adaptive weighting of channel attention and temporal self-attention mechanisms is combined to significantly improve the classification accuracy and interpretability of the model. Experiments show that MSF-AttNet outperforms traditional methods and other deep learning models on the ISCX VPN-nonVPN dataset.

Scene classification is an important tool for remote sensing image interpretation, and it has fundamental applications in research and industry. However, given complex backgrounds and scale variations, remote sensing images have large intraclass diversity and interclass similarity, which bring challenges to accurate classification of remote sensing images. We proposed a scene classification method using joint learning and multiscale attention to alleviate the aforementioned problems. To fully utilize the multiscale information of the image and improve the adaptability of the proposed method to objects with various sizes, different from general methods that fuse different scales of features for classification, joint learning using multiscale features is developed to optimize the whole network. Specifically, we leverage a pretrained deep convolutional neural network as the feature extractor to extract low-level, medium-level, and high-level feature maps from the images. Then, due to the poor semantics of low-level and medium-level feature maps compared with the high-level feature maps, we design a multiscale attention module to enhance the semantic information and suppress the noise information. Finally, the global mean pooling is used to obtain the feature vectors and different classifiers are used for different feature vectors. And the decision-level fusion is adopted to obtain more reliable predictions. The experimental results on the AID and NWPU-RESISC45 datasets show that the proposed method makes a significant improvement in terms of overall accuracies compared with the baselines. And the overall accuracies of our method on the two datasets are 97.49% and 95.20%, respectively, which achieves state-of-the-art performance. The code will be public at a Github repository available at https://github.com/Cbanyungong/JLMSAF.

Abandoned object detection is a critical task in the field of public safety. However, existing methods perform poorly when detecting small and occluded objects, leading to high false detection and missed detection rates. To address this issue, this paper proposes an abandoned object detection method that integrates an adaptive dual-background model with SAO-YOLO (Small Abandoned Object YOLO). The goal is to reduce false and missed detection rates for small and occluded objects, thereby improving overall detection accuracy. First, the paper introduces an adaptive dual-background model that adjusts according to scene changes, reducing noise interference in the background model. When combined with an improved PFSM (Pixel-based Finite State Machine) model, this enhances detection accuracy and robustness. Next, a network model called SAO-YOLO is designed. Key improvements within this model include the SAO-FPN (Small Abandoned Object FPN) feature extraction network, which fully extracts features of small objects, and a lightweight decoupled head, SODHead (Small Object Detection Head), which precisely extracts local features and enhances detection accuracy through multi-scale feature fusion. Finally, experimental results show that SAO-YOLO increases mAP@0.5 and mAP@0.5:0.95 by 9.0% and 5.1%, respectively, over the baseline model. It outperforms other advanced detection models. Ultimately, after a series of experiments on the ABODA, PETS2006, and AVSS2007 datasets, the proposed method achieved an average detection precious of 91.1%, surpassing other advanced methods. It significantly outperforms other advanced detection methods. This approach notably reduces false and missed detections, especially for small and occluded objects.

Video object detection is essential for airport surface surveillance, but the objects on the scene are mostly small objects with low resolution, they have no obvious feature information. Due to the scale differences of the objects and the fixed receptive field on the feature maps, detectors cannot model multi‐scale context information and cover all objects. In addition, although the video detection algorithm can be used as a method to solve the problem of small object detection, the temporal feature fusion method of current video detection is very dependent on the quality of a single feature map. Therefore, this paper aims to enhance the features of small objects of a single image. First, an attentional multi‐scale feature fusion enhancement (A‐MSFFE) network is built on the memory‐enhanced global‐local aggregation (MEGA) to supplement semantic and spatial information of small objects. Then, a context feature enhancement (CFE) module is designed for obtaining different receptive fields through different dilated convolutions. Meanwhile, a video detection dataset about the airport is established. Finally, the experimental results show that the proposed method can improve the detection accuracies of small objects and outperform other state‐of‐the‐art video object detection algorithms in self‐built airport dataset.

DeLiVoTr: Deep and light-weight voxel transformer for 3D object detection

Rodent infestation is a critical factor contributing to grassland degradation, which significantly negatively affects grassland ecosystems. To assess rodent infestation on the plateau, there is an urgent need for a scientifically sound and effective method to detect the distribution of rodent burrows. In response, this study proposes a semantic segmentation approach utilizing the SegFormer model to detect rodent infestation in highland areas. First, we used an unmanned aerial vehicle to collect video data from the plateau and constructed a rodent burrows dataset after processing and precise labeling. Second, to address the issue of SegFormer’s suboptimal performance in segmenting small targets within complex backgrounds and among similar objects, we implemented targeted modifications to enhance its effectiveness for this task. Incorporating the efficient multi-scale attention (EMA) mechanism into SegFormer’s encoder improves the model’s capacity to capture global contextual information. Meanwhile, integrating the multi-kernel convolution feed-forward network (MCFN) into the decoder optimizes the problem of detail recovery and fusion of multi-scale features. We name this method EM-SegFormer (Efficient Multi-scale SegFormer). The experimental results demonstrate that the method achieves relatively good performance on the rodent burrows dataset. This study introduces a novel approach for plateau rodent infestation detection and offers reliable technical support for grassland restoration and management.

The traditional convolutional neural network (CNN) only focuses on the features of the last layer of the network, ignoring those of other layers. The detailed information of the shallow network can improve the accuracy of classification to some extent. With the deepening of the network, the problems such as gradient disappearance and explosion are obvious. In order to alleviate this problem, a method for residual network image classification with multi-scale feature fusion is proposed in the paper. First of all, the Random Image Cropping and Patching (RICAP), a data augmentation method, is adopted to cut and splice the new training samples from the training set, based on which feature maps of different sizes are obtained by residual modules. Secondly, after reducing the dimension of high-level feature maps, all the feature maps with the same dimension are processed by the multi-scale fusion. Finally, these feature vectors are input into the Softmax classifier for training and classification, and the method of learning rate decay is used in the training process. The experiment results indicate that this method has a good performance on image classification based on the public datasets of MNIST and CIFAR-10.KeywordsMulti-scale feature fusionResidual networkData augmentationImage classification

With the development of deep convolutional neural networks (CNNs), the object detection accuracy has been greatly improved. But the performance of small object detection is still far from satisfactory, mainly because small objects are so tiny that the information contained in the feature map is limited. Existing methods focus on improving classification accuracy but still suffer from the limitation of bounding box prediction. To solve this issue, we propose a detection framework by generative and discriminative learning. First, a reconstruction generator network is designed to reconstruct the mapping from low frequency to high frequency for anchor box prediction. Then, a detector module extracts the regions of interest (ROIs) from generated results and implements a RoI-Head to predict object category and refine bounding box. In order to guide the reconstructed image related to the corresponding one, a discriminator module is adopted to tell from the generated result and the original image. Extensive evaluations on the challenging MS-COCO dataset demonstrate that our model outperforms most state-of-the-art models in detecting small objects, especially the reconstruction module improves the average precision for small object (APs) by 7.7%.

To accurately identify the abnormal door-closing noise, we propose a method to recognize the time-frequency image of door closing sound based on a multi-scale feature fusion network model. The door-closing sound signal is transformed into a time-frequency image through wavelet analysis, and a classification model based on multi-scale feature fusion is designed. The model introduces multi-scale filters and dilated convolution and adds two improved inception modules to keep the model lightweight. At the same time, richer spatial features can be obtained. The features of different scales are spliced and input to the fully connected layer, and a dropout layer is added to the fully connected layer to suppress overfitting. By comparing the loss and accuracy rate, adjusting different hyperparameters, the optimal model is obtained. The experimental results show that the multi-scale feature fusion network model has a higher accuracy rate than the transfer learning model. Test accuracy rate is 86% and can effectively recognize abnormal door-closing noise. It provides a feasible theoretical basis for the direction of abnormal door noise recognition.