Prior Knowledge Enhanced Learning Approach for Infrared Small-Target Detection With Single-Point Supervision

Effective detection of small infrared (IR) targets buried in complex backgrounds and heavy noise plays an important role in IR search and track (IRST) systems. In this letter, an IR small target detection method called adaptive scale patch-based contrast measure (ASPCM) is proposed. Compared with existing detection methods based on the human visual system (HVS), our method can estimate the size of the potential target at each location. According to the estimated size, the local contrast between the target and the background is greatly enhanced, and the background clutter can be further suppressed. Experimental results on four real sequences of different complex background demonstrate that the proposed method can detect targets effectively and efficiently, even in the case of complex backgrounds and heavy noises.

Designing and learning a lightweight network for infrared small target detection via dilated pyramid and semantic distillation

Dim and small infrared target fast detection guided by visual saliency

ABSTRACTIn the field of infrared small target detection (ISTD), the ability to detect targets in dim environments is critical, as it improves the performance of target recognition in nighttime and harsh weather conditions. The blurry contour, small size and sparse distribution of infrared small targets increase the difficulty of identifying such targets in cluttered backgrounds. Existing methodologies fall short of satisfying the requisites for the detection and categorisation of infrared small targets. To address these challenges and to enhance the precision of small object detection and classification, this paper introduces an innovative approach called location refinement and adjacent feature fusion YOLO (LA‐YOLO), which enhances feature extraction by integrating a multi‐head self‐attention mechanism (MSA). We have improved the feature fusion method to merge adjacent features, to enhance information utilisation in the path aggregation network (PAN). Lastly, we introduce supervision on the target centre points in the detection network. Empirical results on publicly available datasets demonstrate that LA‐YOLO achieves an impressive average precision (AP) of 92.46% on IST‐A and a mean average precision (mAP) of 84.82% on FLIR. The results surpass those of contemporary state‐of‐the‐art detectors, striking a balance between precision and speed. LA‐YOLO emerges as a viable and efficacious solution for ISTD, making a substantial contribution to the progression of infrared imagery analysis. The code is available at https://github.com/liusjo/LA‐YOLO.

Infrared (IR) small-target detection plays a vital role in various applications of both civil and military areas, such as marine rescue, forest fire prevention, and precise guidance. However, challenges stemming from the small size of IR targets and noise interference in complex backgrounds often limit the accuracy of IR small-target detection algorithms. Owing to the rapid development of deep learning, numerous convolutional neural networks (CNNs)-based methods have emerged in recent years, but they are inevitable to encounter the risk of target loss in deep layers due to the use of pooling layers. To address this problem, we present a wavelet transform-based augmented perception network, namely WTAPNet, in this article. First, an enhancement and enlarge (EE) module is designed to improve the network’s perceptual capability for IR small targets by magnifying image resolution and augmenting target features at the same time. Then, a discrete wavelet transform-based downsampling (DWTD) module and an inverse wavelet transform-based fusion (IWTF) module are proposed. These two modules collaboratively work to extract and fuse multiscale features, which simultaneously reduces information loss. Finally, a bottom-up path fusion strategy is exploited to highlight and preserve small-target features, which associate the lowest level with the highest level features and interconnect predictions from different hierarchical levels. Experimental results on the NUDT-SIRST dataset and the SIRST dataset demonstrate the superiority of our WTAPNet over other state-of-the-art IR small-target detection methods in terms of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F1$ </tex-math></inline-formula>-measure, recall, and other indicators. Our codes are publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/MinjieWan/WTAPNet</uri>.

Specific to the problem of infrared small-target detection in complex backgrounds, a multimechanism deep feature enhancement network model (MDFENet) was proposed. A lightweight multimechanism attention collaborative fusion module was proposed to efficiently fuse low-level features and high-level features to solve the problem that small infrared targets are easy to annihilate in the deep layer of the network. Based on the analysis of the background and target data, a normalized loss function was proposed, which integrates the segmentation threshold selection into the network and normalizes the probability of the network output to simulate a step function and reflect relative differences. Aiming at the sparseness of infrared target features, we used the subpixel convolution method to upsample the features to obtain high-resolution feature images while expanding the size of the feature map. We conducted detailed comparison and ablation experiments, comparing MDFENet with ALCNet, APGCNet, and other state-of-the-art networks to verify the effectiveness and efficiency of the network. Results show that the MDFENet algorithm achieves the optimal result in the balance of detection efficiency and lightweightedness on two datasets.

Most current infrared small target detection methods attempt to fuse local and global information by using single-scale inputs and creating a multi-scale feature pyramid during network feeding forwards. Further to this, our research finds that using high-resolution inputs can improve recall, while low-resolution inputs improve precision. Nevertheless, solely focusing on global or local information can result in missing target and false alarm. To address these issues, we propose the BPR-Net to balance precision and recall via a novel multi-scale attention mechanism, which combines semantic and shallow features of multi-scale inputs. We first scale the input image into multiple images with varying resolutions and feed them into the network. In the encoder, Scale Fusion Module (SFM) fuses features from corresponding images of different resolutions. In the decoder, a Channel Fusion Module (CFM) fuses useful information from multiple channels. Furthermore, a Wavelet Transform cross-layer skip Layer (WTL) is employed to enhance the interaction between decoder layers for more effective multi-scale feature fusion. Experimental results demonstrate that our approach achieves a balance between recall and precision and yields state-of-the-art performance on challenging benchmarks including Sirst, MDvsFA, and SIATD. Notably, our approach achieves an F1 score of 0.9409 on the challenging benchmark SIATD, surpassing the state-of-the-art method by 16.7%.

With advancements in robust stereo matching and optical flow estimation networks, models pre-trained on synthetic data demonstrate strong robustness to unseen domains. However, their robustness can be seriously degraded when fine-tuning them in real-world scenarios. This paper investigates fine-tuning stereo matching and optical flow estimation networks without compromising their robustness to unseen domains. Specifically, we divide the pixels into consistent and inconsistent regions by comparing Ground Truth (GT) with Pseudo Label (PL) and demonstrate that the imbalance learning of consistent and inconsistent regions in GT causes robustness degradation. Based on our analysis, we propose the DKT framework, which utilizes PL to balance the learning of different regions in GT. The core idea is to utilize an exponential moving average (EMA) teacher to measure what the student network has learned and dynamically adjust the learning regions. We further propose the DKT++ framework, which improves target-domain performances and network robustness by applying slow-fast update teachers to generate more accurate PL, introducing the unlabeled data and synthetic data. We integrate our frameworks with state-of-the-art networks and evaluate their effectiveness on several real-world datasets. Extensive experiments show that our method effectively preserves the robustness of stereo matching and optical flow networks during fine-tuning.

The detection of small infrared targets lacking texture and shape information in the presence of complex background clutter is a challenge that has attracted considerable research attention in recent years. Typical deep learning-based target detection methods are designed with deeper network structures, which may lose targets in the deeper layers and cannot directly be used for small infrared target detection. Therefore, we designed the attention fusion feature pyramid network (AFFPN) specifically for small infrared target detection. Specifically, it consists of feature extraction and feature fusion modules. In the feature extraction stage, the global contextual prior information of small targets is first considered in the deep layer of the network using the atrous spatial pyramid pooling module. Subsequently, the spatial location and semantic information features of small infrared targets in the shallow and deep layers are adaptively enhanced by the designed attention fusion module to improve the feature representation capability of the network for targets. Finally, high-performance detection is achieved through the multilayer feature fusion mechanism. Moreover, we performed a comprehensive ablation study to evaluate the effectiveness of each component. The results demonstrate that the proposed method performs better than state-of-the-art methods on a publicly available dataset. Furthermore, AFFPN was deployed on an NVIDIA Jetson AGX Xavier development board and achieved real-time target detection, further advancing practical research and applications in the field of unmanned aerial vehicle infrared search and tracking.

The infrared image in the sea background usually contains random noise, sea waves, fish scale light and other clutter. Sea waves and fish scale light interference are similar to small infrared targets, so it is difficult to eliminate such interference and detect small infrared targets quickly and accurately. In this paper, a small target detection method based on non-local similarity is proposed. The interference is eliminated by using the class feature similarity of interference and the overall feature uniqueness of the small target. Firstly, this paper performs filtering to remove random noise, and the suspected small targets can be extracted by executing Top Hat filtering. Secondly, a multi-feature vector is constructed for all suspected targets. Finally, this paper analyzes the characteristics of different interference and executes clustering step by step. The wave edge interference and fish scale light interference can be eliminated in turn. Feature threshold is set based on the interference that has been identified to obtain the true small infrared target according to the uniqueness of the characteristics of the true target. The experimental results show that compared with other small target detection algorithms, the proposed algorithm can detect small infrared targets in complex sea background effectively.

Infrared small-target detection under heterogeneous background, as a challenging task, plays an important role in many applications. In practice, there are not only bright targets but also dim targets, e.g., rescue aircraft and vehicles in the forest fire scene. Considering that most existing infrared small-target detection methods are merely aimed at bright targets, a novel method using multiple morphological profiles (MMP) is proposed, which can detect various types of targets whose brightness varies greatly. In the designed morphological feature extraction, different attributes, i.e., area attribute and height attribute, are applied to extract spatial size and contrast information of small-target in the max-tree and min-tree, respectively. Furthermore, discontinuous pruning values are further utilized for different attributes, and a designed fusion strategy of different pruning values results in more robust detection performance. Experimental results validated on two synthetic data and six real data sets demonstrate that the proposed MMP can not only detect a variety of brightness of targets and different types of targets and kinds of spatial sizes of targets but also further improve the contrast between targets and background, and the background clutter is significantly suppressed.

Real-time small infrared (IR) target detection is critical to the performance of the situational awareness system in high-altitude aircraft. However, current IR target detection systems are generally hardware-unfriendly and have difficulty in achieving a robust performance in datasets with clouds occupying a large proportion of the image background. In this paper, we present new results by using an efficient method that extracts the candidate targets in the pre-processing stage and fuses the local scale, blob-based contrast map and gradient map in the detection stage. We also developed mid-wave infrared (MWIR) and long-wave infrared (LWIR) cameras for data collection experiments and algorithm evaluations. Experimental results using both publicly available datasets and image sequences acquired by our cameras clearly demonstrated that the proposed method achieves high detection accuracy with the mean AUC being at least 22.3% higher than comparable methods, and the computational cost beating the other methods by a large margin.

Small infrared target detection is a key and challenging issue in object detection and tracking systems. Existing algorithms can be mainly categorized into nonlocal-based or local-based methods. However, the detection performance degrades rapidly when facing highly heterogeneous backgrounds. This is mainly due to that they exploit only one kind of information (e.g., local or nonlocal) while sacrificing the other. Thus, an effective small target detection method is proposed to combine local and nonlocal priors. The former is obtained by a sliding dual window while the latter is realized by low-rank and sparse decomposition. Experimental results on three real datasets validate the effectiveness of the proposed framework, which is more stable and robust compared with several state-of-the-art methods, especially for the image scenes with heavy background clutters.

Small infrared target detection is one of the key techniques in infrared searching and tracking applications. A novel small target detection method based on the complex filter bank is proposed in this paper. Firstly, a training sample dataset is built and the complex filter bank is utilized to extract the feature representation of each pixel. Then, the support vector machine is used to detect the small target. Finally, a simple adaptive segmentation method is adopted to segment the result image. To evaluate the performance of the proposed method, background clutter suppression and small target detection probability experiments are conducted over real infrared small targets in two image sequences. The experiment results show that the proposed method can not only suppress background clutter effectively, but also detect targets accurately with low false alarm rate.

Multi-stage morphological operators for small infrared target detection