Gradient-Enhanced Feature Pyramid Network for Infrared Small Target Detection

Dim and small infrared target fast detection guided by visual saliency

Entropy-based window selection for detecting dim and small infrared targets

It is always a difficult challenge to detect small infrared targets under complex background and ensure high detection rate and low false alarm rate. In this paper, a detection method based on morphological Top-hat filtering and local energy difference is proposed. This method utilizes the advantage that morphological Top-hat filter can suppress the backgTround, and then segmented the image based on local energy difference to identify small targets. In this paper, the specific detection process of this method is given, and a number of small infrared target grayscale images under complex background are selected for the simulation experiment. The experimental results show that this method has a high detection rate and a low false alarm rate, and is of certain reference value for the detection of small infrared targets under complex background.

The infrared (IR) small target detection algorithm with a high detection rate, low false alarm rate, and high real-time performance has significant application value in the field of IR remote sensing. IR small targets in complex backgrounds have low contrast and low signal-to-noise ratio (SNR). Therefore, small target detection is more difficult. Traditional IR small target detection is generally implemented by local contrast methods (LCM), nonlocal autocorrelation methods (NAM), and adaptive segmentation. In this letter, a robust infrared small target detection network (RISTDnet) is proposed based on deep learning. In RISTDnet, a feature extraction framework combining handcrafted feature methods and convolutional neural networks is constructed, a mapping network between feature maps and the likelihood of small targets in the image is established, and a threshold is applied on the likelihood map to segment real targets. Experimental results show that the RISTDnet can detect small targets with different sizes and low SNRs in complex backgrounds and have better effectiveness and robustness against existing algorithms.

The infrared image in the sea background usually contains random noise, sea waves, fish scale light and other clutter. Sea waves and fish scale light interference are similar to small infrared targets, so it is difficult to eliminate such interference and detect small infrared targets quickly and accurately. In this paper, a small target detection method based on non-local similarity is proposed. The interference is eliminated by using the class feature similarity of interference and the overall feature uniqueness of the small target. Firstly, this paper performs filtering to remove random noise, and the suspected small targets can be extracted by executing Top Hat filtering. Secondly, a multi-feature vector is constructed for all suspected targets. Finally, this paper analyzes the characteristics of different interference and executes clustering step by step. The wave edge interference and fish scale light interference can be eliminated in turn. Feature threshold is set based on the interference that has been identified to obtain the true small infrared target according to the uniqueness of the characteristics of the true target. The experimental results show that compared with other small target detection algorithms, the proposed algorithm can detect small infrared targets in complex sea background effectively.

The detection of small infrared targets lacking texture and shape information in the presence of complex background clutter is a challenge that has attracted considerable research attention in recent years. Typical deep learning-based target detection methods are designed with deeper network structures, which may lose targets in the deeper layers and cannot directly be used for small infrared target detection. Therefore, we designed the attention fusion feature pyramid network (AFFPN) specifically for small infrared target detection. Specifically, it consists of feature extraction and feature fusion modules. In the feature extraction stage, the global contextual prior information of small targets is first considered in the deep layer of the network using the atrous spatial pyramid pooling module. Subsequently, the spatial location and semantic information features of small infrared targets in the shallow and deep layers are adaptively enhanced by the designed attention fusion module to improve the feature representation capability of the network for targets. Finally, high-performance detection is achieved through the multilayer feature fusion mechanism. Moreover, we performed a comprehensive ablation study to evaluate the effectiveness of each component. The results demonstrate that the proposed method performs better than state-of-the-art methods on a publicly available dataset. Furthermore, AFFPN was deployed on an NVIDIA Jetson AGX Xavier development board and achieved real-time target detection, further advancing practical research and applications in the field of unmanned aerial vehicle infrared search and tracking.

The detection of infrared small targets under low signal-to-clutter ratio (SCR) and complex background conditions has been a challenging and popular research topic. In this article, a spatial-temporal feature-based detection framework is proposed. First, several factors, such as the infrared target’s small sample, the sensitive size, and the usual sample selection strategy, that affect the detection of small targets are analyzed. In addition, the small intersection over union (IOU) strategy, which helps to solve the false convergence and sample misjudgment problem, is proposed. Second, aiming at the difficulties due to the target’s dim information and complex background, the interframe energy accumulation (IFEA) enhancement mechanism-based end-to-end spatial-temporal feature extraction and target detection framework is proposed. This framework helps to enhance the target’s energy, suppress the strong spatially nonstationary clutter, and detect dim small targets. Experimental results show that using the small IOU strategy and IFEA mechanism, the proposed multiple frame-based detection framework performs better than some popular deep learning (DL)-based detection algorithms.

Infrared (IR) small-target detection plays a vital role in various applications of both civil and military areas, such as marine rescue, forest fire prevention, and precise guidance. However, challenges stemming from the small size of IR targets and noise interference in complex backgrounds often limit the accuracy of IR small-target detection algorithms. Owing to the rapid development of deep learning, numerous convolutional neural networks (CNNs)-based methods have emerged in recent years, but they are inevitable to encounter the risk of target loss in deep layers due to the use of pooling layers. To address this problem, we present a wavelet transform-based augmented perception network, namely WTAPNet, in this article. First, an enhancement and enlarge (EE) module is designed to improve the network’s perceptual capability for IR small targets by magnifying image resolution and augmenting target features at the same time. Then, a discrete wavelet transform-based downsampling (DWTD) module and an inverse wavelet transform-based fusion (IWTF) module are proposed. These two modules collaboratively work to extract and fuse multiscale features, which simultaneously reduces information loss. Finally, a bottom-up path fusion strategy is exploited to highlight and preserve small-target features, which associate the lowest level with the highest level features and interconnect predictions from different hierarchical levels. Experimental results on the NUDT-SIRST dataset and the SIRST dataset demonstrate the superiority of our WTAPNet over other state-of-the-art IR small-target detection methods in terms of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F1$ </tex-math></inline-formula>-measure, recall, and other indicators. Our codes are publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/MinjieWan/WTAPNet</uri>.

Specific to the problem of infrared small-target detection in complex backgrounds, a multimechanism deep feature enhancement network model (MDFENet) was proposed. A lightweight multimechanism attention collaborative fusion module was proposed to efficiently fuse low-level features and high-level features to solve the problem that small infrared targets are easy to annihilate in the deep layer of the network. Based on the analysis of the background and target data, a normalized loss function was proposed, which integrates the segmentation threshold selection into the network and normalizes the probability of the network output to simulate a step function and reflect relative differences. Aiming at the sparseness of infrared target features, we used the subpixel convolution method to upsample the features to obtain high-resolution feature images while expanding the size of the feature map. We conducted detailed comparison and ablation experiments, comparing MDFENet with ALCNet, APGCNet, and other state-of-the-art networks to verify the effectiveness and efficiency of the network. Results show that the MDFENet algorithm achieves the optimal result in the balance of detection efficiency and lightweightedness on two datasets.

A multi-modal 3D object-detection method, based on data from cameras and LiDAR, has become a subject of research interest. PointPainting proposes a method for improving point-cloud-based 3D object detectors using semantic information from RGB images. However, this method still needs to improve on the following two complications: first, there are faulty parts in the image semantic segmentation results, leading to false detections. Second, the commonly used anchor assigner only considers the intersection over union (IoU) between the anchors and ground truth boxes, meaning that some anchors contain few target LiDAR points assigned as positive anchors. In this paper, three improvements are suggested to address these complications. Specifically, a novel weighting strategy is proposed for each anchor in the classification loss. This enables the detector to pay more attention to anchors containing inaccurate semantic information. Then, SegIoU, which incorporates semantic information, instead of IoU, is proposed for the anchor assignment. SegIoU measures the similarity of the semantic information between each anchor and ground truth box, avoiding the defective anchor assignments mentioned above. In addition, a dual-attention module is introduced to enhance the voxelized point cloud. The experiments demonstrate that the proposed modules obtained significant improvements in various methods, consisting of single-stage PointPillars, two-stage SECOND-IoU, anchor-base SECOND, and an anchor-free CenterPoint on the KITTI dataset.

Coarse-to-fine airway segmentation using multi information fusion network and CNN-based region growing

Among the most difficult difficulties in contemporary satellite image-processing subjects is cloud and cloud shade segmentation. Due to substantial background noise interference, existing cloud and cloud shadow segmentation techniques would result in false detection and missing detection. We propose a Location Pooling Multi-Scale Network (LPMSNet) in this study. The residual network is utilised as the backbone in this method to acquire semantic info on various levels. Simultaneously, the Location Attention Multi-Scale Aggregation Module (LAMA) is introduced to obtain the image’s multi-scale info. The Channel Spatial Attention Module (CSA) is introduced to boost the network’s focus on segmentation goals. Finally, in view of the problem that the edge details of cloud as well as cloud shade are easily lost, this work designs the Scale Fusion Restoration Module (SFR). SFR can perform picture upsampling as well as the acquisition of edge detail information from cloud as well as cloud shade. The mean intersection over union (MIoU) accuracy of this network reached 94.36% and 81.60% on the Cloud and Cloud Shadow Dataset and the five-category dataset L8SPARCS, respectively. On the two-category HRC-WHU Dataset, the accuracy of the network on the intersection over union (IoU) reached 90.51%. In addition, in the Cloud and Cloud Shadow Dataset, our network achieves 97.17%, 96.83%, and 97.00% in precision (P), recall (R), and F1 score (F1) in cloud segmentation tasks, respectively. In the cloud shadow segmentation task, precision (P), recall (R), and F1 score (F1) reached 95.70%, 96.38%, and 96.04%, respectively. Therefore, this method has a significant advantage over the current cloud and cloud shade segmentation methods.

This study highlights the importance of unmanned aerial vehicle (UAV) multispectral (MS) imagery for the accurate delineation and analysis of wetland ecosystems, which is crucial for their conservation and management. We present an enhanced semantic segmentation algorithm designed for UAV MS imagery, which incorporates thermal infrared (TIR) data to improve segmentation outcomes. Our approach, involving meticulous image preprocessing, customized network architecture, and iterative training procedures, aims to refine wetland boundary delineation. The algorithm demonstrates strong segmentation results, including a mean pixel accuracy (MPA) of 90.35% and a mean intersection over union (MIOU) of 73.87% across different classes, with a pixel accuracy (PA) of 95.42% and an intersection over union (IOU) of 90.46% for the wetland class. The integration of TIR data with MS imagery not only enriches the feature set for segmentation but also, to some extent, helps address data imbalance issues, contributing to a more refined ecological analysis. This approach, along with the development of a comprehensive dataset that reflects the diversity of wetland environments and advances the utility of remote sensing technologies in ecological monitoring. This research lays the groundwork for more detailed and informative UAV-based evaluations of wetland health and integrity.

Urban Informal Settlements (UIS) denote densely populated locales characterized by inadequate urban infrastructure standards, often exhibiting an amalgamation of rural and urban attributes, primarily situated within the confines of major cities or metropolitan regions. UIS mapping is a typical task that aims to identify pixels corresponding to urban informal settlements in remote sensing images. The extremely similar visual characteristics, sample uncertainty, and highly manual costs bring large-scale UIS mapping noteworthy challenges. In this paper, we propose a sample uncertainty-aware semi-supervised learning method guided by background-knowledge from crowdsourced geospatial data (namely SemiUIS) for UIS mapping and produces a 1-meter resolution UIS map in the Urban Agglomeration in the Middle Reaches of the Yangtze River, China (UAMRYR). Our proposed SemiUIS method integrates semisupervised learning with the guidance of background knowledge from crowdsourcing data to reduce the uncertainty of the generated pseudo-semantic segmentation samples to improve the quality of the constructed data set by jointly optimizing the background (Not UIS areas) and the foreground (UIS areas). In contrast, traditional semi-supervised learning methods only focus on explicitly optimizing the foreground (UIS areas) in the pseudo-sample generation process. Experiments were conducted in the 31 prefecture-level cities of UAMRYR. Visual interpretation of labeled remote sensing samples, street view images and two novel knowledge validation indicators, Global Knowledge Precision (GKP) and Local Knowledge Precision (LKP) are proposed to verify the UIS mapping results. The proposed method reached an overall accuracy (OA) of 90.58 %, mean intersection over union (mIoU) of 75.88 %, Accuracy (Acc) of 78.77 %, Intersection over Union (IoU) of 62.97 %, GKP of 90.33 %, and LKP of 85.01 %, respectively with only 36 (5%) training samples. This work will be available at https://github.com/RunyuFan/UisYangtze.

Small object detection in unmanned aerial vehicle (UAV) aerial imagery faces substantial challenges due to small target scales, complex backgrounds, noise interference, and so on. To enhance multi-scale feature representation and detection efficiency, this paper proposes MSEF-YOLO11s. Specifically, we first design a lightweight partial multi-scale (LPMS) module, which effectively aggregates cross-scale information and enhances multi-scale representations in the backbone for small objects. Secondly, to dynamically adjust feature weights and mitigate feature conflicts in the neck, we devise a multi-scale boundary-semantic alignment (MS-BSA) based on adaptive attention, which can further avoid computational redundancy for sufficient fusion. Finally, a lightweight shared detail detection head (LSDDH) replaces the decoupled head structure with shared convolutional layers, resolving the issue of parameter explosion associated with adding a dedicated small object detection head. Experimental results demonstrate the effectiveness of the proposed model. Specifically, compared to the baseline YOLO11s, MSEF-YOLO11s achieves an improvement of 6.6% in mAP50 on the VisDrone2019 test set, with only 4.4M increase in parameters. Furthermore, mAP50 on the TinyPerson test set increases from 22.8% to 28.1%, confirming the model’s strong generalization capability.