Exploiting Discriminating Features for Fine-Grained Ship Detection in Optical Remote Sensing Images

In recent years, many ship detection algorithms based on convolutional neural networks (CNNs) have been proposed to improve the performance of ship detection. However, with the increase in model complexity and size, it is challenging to deploy these models to resource-constrained edge platforms. In this letter, a low coupling algorithm that belongs to anchor-free methods is proposed for ship detection to reduce the model complexity and still obtain a competitive performance. The proposed low coupling network (LCNet) is easy to deploy and contributes to speeding up the inference and improving memory utilization. In addition, we propose a model compression process consisting of the quantization-aware training (QAT) method and a structural pruning method based on Taylor expansion, which can effectively reduce the model size according to hardware resource constraints. Comparative experimental results demonstrate that LCNet outperforms the state-of-the-art ship detection and natural object detection algorithms, with a 95.27% mAP and 88.91% F1 score on the HRSC2016 dataset. Our proposed model compression method also achieves a compression ratio of at least 80% with a negligible loss of performance.

Ship detection and classification is critical for national maritime security and national defense. Although some SAR (Synthetic Aperture Radar) image-based ship detection approaches have been proposed and used, they are not able to satisfy the requirement of real-world applications as the number of SAR sensors is limited, the resolution is low, and the revisit cycle is long. As massive optical remote sensing images of high resolution are available, ship detection and classification on theses images is becoming a promising technique, and has attracted great attention on applications including maritime security and traffic control. Some digital image processing methods have been proposed to detect ships in optical remote sensing images, but most of them face difficulty in terms of accuracy, performance and complexity. Recently, an autoencoder-based deep neural network with extreme learning machine was proposed, but it cannot meet the requirement of real-world applications as it only works with simple and small-scaled data sets. Therefore, in this paper, we propose a novel ship detection and classification approach which utilizes deep convolutional neural network (CNN) as the ship classifier. The performance of our proposed ship detection and classification approach was evaluated on a set of images downloaded from Google Earth at the resolution 0.5m. 99% detection accuracy and 95% classification accuracy were achieved. In model training, 75× speedup is achieved on 1 Nvidia Titanx GPU.

Ship detection in remote sensing plays an important role in civil and military fields. Owing to the complex background and uncertain direction, ship detection is full of challenge by using the commonly used object-detection methods. In this article, a new framework for detecting the arbitrary direction ships is proposed based on the improvement in the Faster region-based convolutional network (R-CNN), in which the shape of the bounding box is described by three sides, namely, vertical side, horizontal side, and short side, respectively. The inclination of the ship is obtained by calculating the arc-tangent value of the vertical side to the horizontal side. First, the better performing ResNet-101 is adopted to extract features over an entire image, which are shared by the region proposal network (RPN) and the head network. Then, the multidirection proposal regions that may contain ships are generated by the RPN. Next, the global and local features of the proposal regions are combined as the whole features of the regions by a multiregion feature-fusion (MFF) module, which can provide more detailed information of the regions. Finally, the head network uses the whole features of the proposal regions for bounding-box recognition through multitask learning, including classification, regression, and incline direction prediction (left or right). The proposed method is tested and compared with other state-of-the-art ship-detection methods on two open remote-sensing data sets and some large-scale and real images. The experimental results validate that the proposed approach has achieved better performance.

Ship detection in multispectral remote-sensing images is critical in marine surveillance applications. The previously proposed ship-detection methods for multispectral satellite imagery usually work well under ideal conditions. When meeting complex environments such as shadows, mists, or clouds, they fail to detect ships. To solve this problem, we propose a novel spectral-reflectance-based ship-detection method. Research has shown that different materials have unique reflectance curves in the same spectral wavelength range. Based on this observation, we present a new feature using the reflectance gradient across multispectral bands. Moreover, we propose a neural network called lightweight fusion networks (LFNet). This network combines the aforementioned reflectance and the color information of multispectral images to jointly verify the regions with ships. The method utilizes a coarse-to-fine detection framework because of the large-sense-sparse-targets situation in remote-sensing images. In the coarse stage, the proposed reflectance feature vector is used to input the classifier to rule out the regions without ships. In fine detection, the LFNet is used to verify true ships. Compared with some traditional methods that merely depend on appearance features in images, the proposed method takes advantage of employing the reflectance variance in objects between each band as additional information. Extensive experiments have been conducted on multispectral images from four satellites under different weather and environmental conditions to demonstrate the effectiveness and efficiency of the proposed method. The results show that our method can still achieve good performance even under harsh weather conditions.

Small ships in optical remote sensing images are hard to detect due to the lack of sufficient detail information. In this paper, we adopt the image super-resolution technology to solve this problem. Specifically, an effective super-resolution network is designed to generate clear super-resolution ship images from small blurry ones produced by the ship detector. Inspired by the idea of generative adversarial network (GAN), the super-resolution network is trained together with a discriminator network in an adversarial way, aiming at generating more realistic super-resolution images. Moreover, to eliminate false detections, the discriminator network is also used to distinguish ship and non-ship images via an additional classification branch. Experimental results demonstrate the effectiveness of the proposed method.

Ship detection in optical remote sensing images poses challenges such as complex scenes, large target scale differences, and imperfect fine-grained classification. Deep learning methods like YOLO have provided effective solutions. With emerging detection approaches like Transformers achieving new breakthroughs in accuracy, this study proposes an enhanced detection model by incorporating Transformer into YOLO network. The model replaces the Backbone of YOLOv5s with SwinTransformer-Tiny and refines the Loss function to Wise-IoU, tailored for remote sensing images and ship targets. Experimental verification on self-curated single-category dataset LEVIR-ship and multi-category dataset HRSC-4 shows that the comprehensive metric mAP50 values have increased from 93.8% and 96.2% to 95.2% and 96.5%, respectively. Both precision (P) and recall (R) values have significantly improved. There is also some enhancement in fine-grained classification metrics on the HRSC-4 dataset. The newly improved model demonstrates good performance.

High-resolution synthetic aperture radar (SAR) data have been widely used in marine environmental protection, marine environmental monitoring, and marine traffic management. Ship detection is one of the important parts of SAR data for marine applications. This letter focuses on the feature analysis of ships in high-resolution SAR images and proposes an improved optimizing algorithm for ship detection. A fast block detector is designed to extract sea clutter in a uniform local area, and then a constant false alarm rate detector is employed. Based on the kernel density estimation of ships, aspect ratio, and pixel points, ships are identified. TerraSAR-X and COSMO-SkyMed images are used to test our algorithm. The experimental results show that this algorithm can be implemented with time-saving, high-precision ship extraction, feature analysis, and detection.

The main aim of optical remote sensing image (ORSI) object detection is to ascertain the area as well as class of all targets in given images. Nowadays, object detection methods established from deep learning are progressively applied to the analysis of the information from ORSI. But because of the compound background as well as extensive object scale shown in ORSI, the difficulty of object detection has increased. There are still some challenges to be solved. First, when the two bounding boxes don't overlap, current indexes like intersection over union (IoU) cannot identify the distance between them. Secondly, the existing methods using convolutional neural network (CNN) could not take advantage of the characteristics of multi-level features so that these methods are not very good at recognizing small-sized objects. To deal with these above problems, this paper comes up with a ORSI object detection method established on multi-level feature fusion as well as improved boundary box regression compared to IoU. Firstly, a new measure called generalized intersection over union (GIoU) is practiced. That metric could measure the similarity betwixt two bounding boxes, regardless of whether they overlap or not. At the same time, we also directly use GIoU as loss. Lastly, a multi-level feature fusion structure is practiced and it is also integrated into the existing convolutional neural network. In this way, our method could take advantage of the multi-level features. Quantitative comparison of the addressed method with the baseline approach on large-scale dataset named Dior are conducted. The comparison with the most advanced method shows that our method has reached the most advanced performance.

Applications of remote sensing images in both defense and civilian sectors have spurred substantial research interest. In the field of remote sensing, object detection confronts challenges such as complex backgrounds, scale diversity, and the presence of dense small objects. To address these issues, we propose an improved deep learning-based model, the Global Multi-scale Fusion Self-calibration Network, which is expected to contribute to alleviating the challenges. It consists of three main components: the hierarchical feature aggregation backbone, which uses improved modules such as the receptive field context-aware feature extraction module, the global information acquisition module, and the simple parameter-free attention module to extract key features and minimize the background interference. To couple multi-scale features, we enhanced the fusing component and designed the multi-scale enhanced pyramid structure integrating the proposed new modules. During the detection phase, especially when focusing on small object detection, we designed a novel convolutional attention feature fusion head. This head is constructed to integrate local and global branches for feature extraction by leveraging channel shuffling and multi-head attention mechanisms for efficient and accurate detection. Experiments on the Detection in Optical Remote Sensing Images (DIOR), Northwestern Polytechnical University Very High-Resolution‐10 (NWPU VHR‐10), remote sensing object detection (RSOD), and DOTAv1.0 data sets show that our method achieves mAP50(mean average precision at 50% intersection over union) of 69.7%, 91.3%, 94.2%, and 70.0%, respectively, outperforming existing comparative methods. The proposed network is expected to provide new perspectives for remote sensing tasks and possible solutions for relevant applications in the image domain.

Salient object detection in optical remote sensing images (ORSI-SOD) is a task that detects the most prominent context in optical remote sensing images (ORSIs). ORSI-SOD is challenging due to the diverse sizes and shapes of the targets, their irregular distribution in the scene, and the occlusion caused by surrounding environments. Recently, many deep learning-based models have demonstrated promising performance in ORSI-SOD. However, there still remains considerable potential for addressing these challenges in ORSI-SOD. In this article, we propose a novel architecture, LSHNet. We propose a dual-branch architecture consisting of an edge encoder that leverages structure features using edges as the structure-prior and an image encoder that extracts context features from the image. We propose three modules. Image-structure fusion module (ISFM) integrates the two-stream features extracted from dual branch encoders through intrapatch and internal-patch attention mechanisms to utilize diverse receptive fields. Local-global feature fusion module (LGFM) transfers global features representing the target to local feature maps to discriminate the region of the targets from background clutters. The semantic cues updating module (SCUM) updates the representative features of the target from high-level to low-level. By integrating hierarchical information effectively, global features extracted from multilayers can be rectified. We experiment with the three main evaluated datasets in ORSI-SOD: ORSSD, EORSSD, and ORSI-4199. We demonstrate the promising results on the three datasets and analyze the effectiveness of the proposed modules in the ablation study.

针对光学遥感图像中微小目标空间分辨率低、有效特征不足等问题，在YOLOv5检测算法基础上，提出一种基于多尺度分层残差网络的光学遥感图像微小目标检测方法。设计了一种简单高效的多尺度分层残差特征提取模块，可在更细粒度水平上获得更丰富的感受野，强化神经网络的特征提取能力，进一步提升微小目标特征丰富度。在此基础上，进一步优化损失函数中的定位损失项，通过增加距离惩罚提升检测算法对微小目标的定位能力。在光学遥感微小目标检测数据集AI-TODv2和微小行人检测数据集TinyPerson上开展了系统对比实验，实验结果表明所提出算法相较于基准YOLOv5算法平均精度分别提升了5.5%和1.8%，有效提高了微小目标检测的召回率和准确率。

Aggregating transformers and CNNs for salient object detection in optical remote sensing images

Existing salient object detection methods for optical remote sensing images still face certain limitations due to complex background variations, significant scale discrepancies among targets, severe background interference, and diverse topological structures. On the one hand, the feature transmission process often neglects the constraints and complementary effects of high-level features on low-level features, leading to insufficient feature interaction and weakened model representation. On the other hand, decoder architectures generally rely on simple cascaded structures, which fail to adequately exploit and utilize contextual information. To address these challenges, this study proposes a Hierarchical Semantic Interaction Module to enhance salient object detection performance in optical remote sensing scenarios. The module introduces foreground content modeling and a hierarchical semantic interaction mechanism within a multi-scale feature space, reinforcing the synergy and complementarity among features at different levels. This effectively highlights multi-scale and multi-type salient regions in complex backgrounds. Extensive experiments on multiple optical remote sensing datasets demonstrate the effectiveness of the proposed method. Specifically, on the EORSSD dataset, our full model integrating both CA and PA modules improves the max F-measure from 0.8826 to 0.9100 (↑2.74%), increases maxE from 0.9603 to 0.9727 (↑1.24%), and enhances the S-measure from 0.9026 to 0.9295 (↑2.69%) compared with the baseline. These results clearly demonstrate the effectiveness of the proposed modules and verify the robustness and strong generalization capability of our method in complex remote sensing scenarios.

ABSTRACTThe amplitude of internal waves is an important parameter for the remote-sensing detection. However, there is no analytic method developed for the amplitude inversion of internal waves based on optical remote-sensing images. In this article, the deep-learning model is introduced to inverse the amplitude of internal waves based on a large number of optical remote-sensing images. The peak-to-peak distance and 15 types of texture characteristic parameters of images are computed, and the relationship between the amplitude of internal waves and the characteristic parameters of remote-sensing images are also investigated. In addition, based on the correlation difference between these parameters and the amplitude, three types of deep-learning model are established by selecting different parameters as input variables. Results show that the inverted amplitude of internal waves from our models shows good agreement with the in-situ data of internal waves, and the average errors between them are about 6.7%, 4.9%, and 3.6%, respectively, which indicates that the deep-learning model is effective for amplitude inversion of internal waves based on optical remote-sensing images.

Salient object detection (SOD) task for optical remote sensing images (RSIs) plays an important role in many remote sensing applications. Most of the existing methods train their networks depending on a large amount of pixel-wise datasets. However, such expensive and time-consuming training setting prevents the approaches becoming flexible and scalable solutions. To this end, we explore efficient SOD for optical RSIs based on easily accessible weak supervision source. In this work, we propose a novel end-to-end progressive attention-based feature recovery framework with scribble supervision. Specifically, to better locate challenging salient objects in optical RSIs, an object position module (OPM) is proposed to capture and enhance the long-range semantic dependence of objects’ position information, which depends on the complementary attention mechanism. And to restore the entire salient objects, a context refinement module (CRM) is proposed, which extract local contextual information for better propagating high-level semantics to low-level details. Moreover, to improve the adaptability of the network to the changing scenarios of optical RSIs, we propose a salient region correcting (SRC) mechanism to help the predicted salient regions rectify their saliency values by constraining the saliency relationship between predictions from different augmentation models. In addition, due to the lack of dataset for weakly supervised SOD for optical RSIs, we relabeled an existing large-scale optical RSIs dataset with scribbles, namely EORSSD-S. Experimental results on benchmark datasets demonstrate that the proposed method can outperform other weakly supervised SOD methods. And the proposed method even outperformed some fully supervised methods. https://github.com/melonless/PAFR.