Object detection algorithm based on feature enhancement

Accurate segmentation of abdominal organs on MRI is crucial for computer-aided surgery and computer-aided diagnosis. Most state-of-the-art methods for MRI segmentation employ an encoder-decoder structure, with skip connections concatenating shallow features from the encoder and deep features from the decoder. In this work, we noticed that simply concatenating shallow and deep features was insufficient for segmentation due to the feature gap between shallow features and deep features. To mitigate this problem, we quantified the feature gap from spatial and semantic aspects and proposed a spatial loss and a semantic loss to bridge the feature gap. The spatial loss enhanced spatial details in deep features, and the semantic loss introduced semantic information into shallow features. The proposed method successfully aggregated the complementary information between shallow and deep features by formulating and bridging the feature gap. Experiments on two abdominal MRI datasets demonstrated the effectiveness of the proposed method, which improved the segmentation performance over a baseline with nearly zero additional parameters. Particularly, the proposed method has advantages for segmenting organs with blurred boundaries or in a small scale, achieving superior performance than state-of-the-art methods.

Asymmetric double-winged multi-view clustering network for exploring diverse and consistent information

In plane wave medical ultrasound imaging, the analog echo signals are received via the transducer, and sampled as digital radio frequency data which is further processed by the beamforming algorithm to obtain the output ultrasound image. The sampling frequency of radio frequency data has a great impact on the image quality. The radio frequency data with higher sampling frequency is closer to the true distribution of the analog echo signals, leading to better image quality. However, high-frequency sampling chips are usually expensive. Thus, improving the plane wave imaging quality for radio frequency data with low sampling frequency has important practical significance. The ultrasound imaging quality is mainly affected by shallow features such as texture and color. However, low-sampling-frequency data has poor shallow features while high-sampling-frequency data has better shallow features. In this work, we propose an adversarial learning method to replace the shallow features of low-frequency data with the shallow features of high-frequency data, but retaining the deep features of low-frequency data. The experiment results demonstrate that the proposed method improves the plane wave imaging quality for low-sampling-frequency data by fusing shallow and deep features.

Predicting the wafer material removal rate (MRR) is an important step in semiconductor manufacturing for total quality control. This work proposes a deep learning model called a fusion network to predict the MRR, in which we consider separating features into shallow and deep features and use the characteristics of deep learning to perform a fusion of these two kinds of features. In the proposed model, the deep features go through a sequence of nonlinear transformations and the goal is to learn the complex interactions among the features to obtain the deep feature embeddings. Additionally, the proposed method is flexible and can incorporate domain knowledge into the model by encoding the knowledge as shallow features. Once the learning of deep features is completed, the proposed model uses the shallow features and the learned deep feature embeddings to obtain new features for the subsequent layers. This work performs experiments on a dataset from the 2016 Prognostics and Health Management Data Challenge. The experimental results show that the proposed model outperforms the competition winner and three ensemble learning methods. The proposed method is a single model, whereas the comparison methods are ensemble models. Besides the experimental results, we conduct extensive experiments to analyze the proposed method.

Most existing small object detection methods rely on residual blocks to process deep feature maps. However, these residual blocks, composed of multiple large-kernel convolution layers, incur high computational costs and contain redundant information, which makes it difficult to improve detection performance for small objects. To address this, we designed an improved feature pyramid network called L Feature Pyramid Network (L-FPN), which optimizes the allocation of computational resources for small object detection by reconstructing the original FPN structure. Based on L-FPN, we further proposed a small object detector named BPD-YOLO. We introduce a Dual-phase Asymptotic Feature Fusion mechanism (DAFF), where the shallow and deep semantic features extracted from the backbone network are initially fused in parallel to mitigate the semantic gap. Subsequently, the intermediate semantic layers are progressively integrated, enabling effective fusion of both shallow and deep feature representations. Additionally, we designed the Deep Spatial Pyramid Fusion module (DSPF), which generates multi-scale feature representations as an alternative to conventional residual block stacking, thereby reducing computational overhead. In the shallow feature extraction stage, DSPF focuses on semantic integration and enhances the extraction of small object features. This strategy, which adaptively selects different modules based on the resolution of the feature maps, is referred to as the Decoupled feature Extraction-semantic Integration mechanism (DEI). Finally, we conducted extensive experiments and thorough evaluations on both the VisDrone and TinyPerson datasets. The results demonstrate that, on the VisDrone dataset, compared to the baseline model YOLOv8n + p2, our BPD-YOLO model with L-FPN achieves a 2.8% improvement in mAP50 and a 1.4% increase in mAP50-95. On the TinyPerson dataset, BPD-YOLO further demonstrates its superiority in high-resolution feature extraction, effectively enhancing detection accuracy while significantly reducing computational costs.

As the spatial resolution of remote sensing images continues to increase, the complexity of the information they carry also grows. Remote sensing images are characterized by large imaging areas, scattered distributions of similar objects, intricate boundary shapes, and a high density of small objects, all of which pose significant challenges for semantic segmentation tasks. To address these challenges, we propose a Remote Sensing Image Segmentation Network that Integrates Global–Local Multi-Scale Information with Deep and Shallow Features (GLDSFNet). To better handle the wide variations in object sizes and complex boundary shapes, we design a Global–Local Multi-Scale Feature Fusion Module (GLMFM) that enhances segmentation performance by fully leveraging multi-scale information and global context. Additionally, to improve the segmentation of small objects, we propose a Shallow–Deep Feature Fusion Module (SDFFM), which effectively integrates deep semantic information with shallow spatial features through mutual guidance, retaining the advantages of both. Extensive ablation and comparative experiments conducted on two public remote sensing datasets, ISPRS Vaihingen and Potsdam, demonstrate that our proposed GLDSFNet outperforms state-of-the-art methods.

Infrared small target detection is widely applied in military and civilian fields. Due to the small size of infrared targets, textural detail is missing. Common target detection methods extract semantic feature by narrowing down the feature map several times, which may lead to the small targets lost in deep layers and are not effective for infrared small target detection. To solve this problem, we propose a novel network called deep asymmetric extraction and aggregation. The network mainly consists of two processes - the vertical feature extraction and the horizontal feature aggregation, both of which are enhanced by an asymmetric attention mechanism. In the vertical process, the use of asymmetric attention mechanism combined with the reduction of down-sampling makes the small target better retained in the deep layers. Then through the horizontal process, shallow spatial feature and deep semantic feature are aggregated to further highlight the small targets while suppressing background noise. Experiments on the public datasets NUAA-SISRT, NUDT-SISRT and MDvsFA-cGan show that our proposed network outperforms the state-of-the-art methods in terms of detection accuracy and parameter efficiency.

Image aesthetic assessment (IAA) with neural attention has made significant progress due to its effectiveness in object recognition. Current studies have shown that the features learned by convolutional neural networks (CNN) at different learning stages indicate meaningful information. The shallow feature contains the low-level information of images, and the deep feature perceives the image semantics and themes. Inspired by this, we propose a visual enhancement network with feature fusion (FF-VEN). It consists of two sub-modules, the visual enhancement module (VE module) and the shallow and deep feature fusion module (SDFF module). The former uses an adaptive filter in the spatial domain to simulate human eyes according to the region of interest (ROI) extracted by neural feedback. The latter not only extracts the shallow feature and the deep feature via transverse connection, but also uses a feature fusion unit (FFU) to fuse the pooled features together with the aim of information contribution maximization. Experiments on standard AVA dataset and Photo.net dataset show the effectiveness of FF-VEN.

Accurate object detection in unmanned aerial vehicle (UAV) images plays a crucial role in fields such as aviation, transportation, and agriculture. However, UAV images often contain a high proportion of small objects, and the limited resources of UAV platforms make it challenging for existing small object detection algorithms to balance detection performance and resource consumption. To address these issues, a lightweight small object detection algorithm called SRP-YOLO is proposed. Compared to the YOLOv8 network, the following improvements are made: first, a high-resolution detection head is designed; second, a receptive field attention convolution module tailored for small UAV objects is introduced to replace the standard convolution module, enhancing the detection capability for small objects and significantly improving their feature representation; Third, partial convolution is combined with C2f to replace some of the standard convolutions, fully leveraging the high-resolution information from shallow features. Finally, A normalized Gaussian Wasserstein Distance (NWD) metric is introduced to reduce the sensitivity of Intersection over Union (IoU) to minor positional deviations of small objects. On VisDrone-DET2019, SRP-YOLO reduces parameters while improving mAP 0.5 by 7.9%. Generalization tests on TinypersonV2 and RAIVD confirm significant gains in small object detection.

Abstract. Building change detection from remote sensing images is vital for many applications such as urban planning and dynamic monitoring, smart city construction, and geographical information census. In recent years, with the improvement of artificial intelligence and computer vision techniques, deep learning algorithms, especially convolutional neural networks (CNN), provide automatic detection and extraction methods. Unlike traditional approaches relying on shallow manual features, CNN can generate the deep semantic features by fusing spatial and spectral information, which is conducive to identifying building change regions. However, most deep CNN models directly fuse different level features and recover spatial details, which probably introduce redundant background information and noise from shallow layers. Considering the building’s multiscale, convolution operation with fixed receptive fields cannot obtain a strong global feature response to changed regions. To address the above problems, we develop a CNN framework for automatic building region change detection using dual-temporal high-resolution remote sensing images. To refine the shallow features, self-attention knowledge distillation strategies are introduced to FCN. Furthermore, we propose the multiscale feature change-aware module to integrate the globally changed information in different decoders. Finally, the model aggregates the multi-scale regional change information and outputs the prediction map. The results of comparative visual analysis and quantitative evaluation in two public datasets demonstrate that the proposed network model can improve the accuracy and efficiency of the automatic building change detection (85.69 IoU, 97.56 OA on the WHU dataset, and 83.72 IoU, 97.64 OA on the LEVIR-CD dataset).

Infrared imaging exploits the thermal radiation characteristics of objects to transform infrared radiation into images. The scattering of thermal radiation degrades the resolution of infrared imaging, leading to low resolution outputs compared to visible light imagery. Consequently, in scenarios characterized by weak target features and complex backgrounds, existing methods still exhibit false alarms and missed detections. To address these challenges, we propose a novel Frequency-Gradient Collaborative Network with Channel Correction and Background Guidance (FGCBNet) for infrared small target detection. Specifically, we design an attention mechanism that integrates a multi-oriented gradient filtering module with a frequency domain learning module. The multi-oriented gradient filtering module leverages the inherent gradient characteristics of the target to suppress background noise in spatial domain, while the frequency domain learning module isolates high-frequency and low-frequency components to effectively separate target details from background interference. In addition, a self-correction channel attention module is developed to adaptively mitigate feature deviations across channels. Finally, a spatial-channel enhancement decoder progressively fuses deep and shallow features during up-sampling, employing background semantic guidance to further enhance target saliency. Experimental results demonstrated that the proposed method achieves superior performance on standardized public benchmarks.

Transformer-based methodologies in object detection have recently piqued considerable interest and have produced impressive results. DETR, an end-to-end object detection framework, ingeniously integrates the Transformer architecture, traditionally used in NLP, into computer vision for sequence-to-sequence prediction. Its enhanced variant, DINO, featuring improved denoising anchor boxes, has showcased remarkable performance on the COCO val2017 dataset. However, it often encounters challenges when applied to scenarios involving small object detection. Thus, we propose an innovative method for feature enhancement tailored to recursive prediction tasks, with a particular emphasis on augmenting small object detection performance. It primarily involves three enhancements: refining the backbone to favor feature maps that are more sensitive to small targets, incrementally augmenting the number of queries for small objects, and advancing the loss function for better performance. Specifically, The study incorporated the Switchable Atrous Convolution (SAC) mechanism, which features adaptable dilated convolutions, to increment the receptive field and thus elevate the innate feature extraction capabilities of the primary network concerning diminutive objects. Subsequently, a Recursive Small Object Prediction (RSP) module was designed to enhance the feature extraction of the prediction head for more precise network operations. Finally, the loss function was augmented with the Normalized Wasserstein Distance (NWD) metric, tailoring the loss function to suit small object detection better. The efficacy of the proposed model is empirically confirmed via testing on the VISDRONE2019 dataset. The comprehensive array of experiments indicates that our proposed model outperforms the extant DINO model in terms of average precision (AP) small object detection.

Both accuracy and efficiency are crucial for image captioning in real-world scenarios. Although Transformer-based models have gained significant improved captioning performance, their computational cost is very high. A feasible way to reduce the time complexity is to exit the prediction early in internal decoding layers without passing the entire model. However, it is not straightforward to devise early exiting into image captioning due to the following issues. On one hand, the representation in shallow layers lacks high-level semantic and sufficient cross-modal fusion information for accurate prediction. On the other hand, the exiting decisions made by internal classifiers are unreliable sometimes. To solve these issues, we propose DeeCap framework for efficient image captioning, which dynamically selects proper-sized decoding layers from a global perspective to exit early. The key to successful early exiting lies in the specially designed imitation learning mechanism, which predicts the deep layer activation with shallow layer features. By deliberately merging the imitation learning into the whole image captioning architecture, the imitated deep layer representation can mitigate the loss brought by the missing of actual deep layers when early exiting is undertaken, resulting in significant reduction in calculation cost with small sacrifice of accuracy. Experiments on the MS COCO and Flickr30k datasets demonstrate the DeeCap can achieve competitive performances with 4× speed-up. Code is available at: https://github.com/feizc/DeeCap.

With the development of remote sensing technology, remote sensing object detection has been widely applied in various fields, but it still faces some thorny challenges, such as the following: 1) the complexity of object scale changes in remote sensing images makes it difficult to improve the performance of small object detection and 2) remote sensing images have complex backgrounds and densely arranged small and weak objects, which pose a serious problem of feature interference. To alleviate these challenges, we propose an end-to-end adaptive feature separation network called AFSNet, which includes a scale-aware module (SAM) and a class-aware module (CAM). The SAM mainly enables feature maps of different resolutions to detect objects of different scales. Shallow feature maps mainly suppress the features of large objects they contain to focus on small object detection, while deep feature maps increase the detailed features of large objects they contain to focus on large object detection. The CAM is mainly used to distinguish the features in the feature map by category, separating the features of different categories into different channels, thus mitigating the problem of inter class feature interference, and blocking background interference. The effectiveness of this article has been proven on the NWPU VHR-10, IPIU-M, DIOR, and DOTA2.0 datasets. It can be widely applied in civilian, military, and other fields. Through experimental verification, our AFSNet achieved 97.70% mAP on the NWPU VHR-10 dataset, 78.9% mAP on the DIOR dataset, and 58.22% mAP on the DOTA2.0 dataset. Our code is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Xidian-AIGroup190726/AFSNet</uri>.

Pine wild disease poses a serious threat to the ecological environment of national forests. Combining the object detection algorithm with Unmanned Aerial Vehicles (UAV) to detect pine wild diseased trees (PWDT) is a significant step in preventing the spread of pine wild disease. To address the issue of shallow feature layers lacking the ability to fully extract features from small-sized diseased trees in existing detection algorithms, as well as the problem of a small number of small-sized diseased trees in a single image, a Shallow Pooled Weighted Feature Enhancement Network (SPW-FEN) based on Small Target Expansion (STE) has been proposed for detecting PWDT. First, a Pooled Weighted Channel Attention (PWCA) module is presented and introduced into the shallow feature layer with rich small target information to enhance the network’s expressive ability regarding the characteristics of two-layer shallow feature maps. Additionally, an STE data enhancement method is introduced for small-sized targets, which effectively increases the sample size of small-sized diseased trees in a single image. The experimental results on the PWDT dataset indicate that the proposed algorithm achieved an average precision and recall of 79.1% and 86.9%, respectively. This is 3.6 and 3.8 percentage points higher, respectively, than the recognition recall and average precision of the existing state-of-the-art method Faster-RCNN, and 6.4 and 5.5 percentage points higher than those of the newly proposed YOLOv6 method.