Detection of Fragmented Rectangular Enclosures in Very High Resolution Remote Sensing Images

Content-based remote sensing (RS) image retrieval (CBRSIR) is a critical way to organize high-resolution RS (HRRS) images in the current big data era. The increasing volume of HRRS images from different satellites and sensors leads to more attention to the cross-source CSRSIR (CS-CBRSIR) problem. Due to the data drift, one crucial problem in CS-CBRSIR is the modality discrepancy. Most existing methods focus on finding a common feature space for various HRRS images to address this issue. In this space, their similarity relations can be measured directly to obtain the cross-source retrieval results straight. This way is feasible and reasonable, however, the specific information corresponding to HRRS images from different sources is always ignored, limiting retrieval performance. To overcome this limitation, we develop a new model for CS-CBRSIR in this paper named dual modality collaborative learning (DMCL). To fully explore the specific information from diverse HRRS images, DMCL first introduces ResNet50 as the feature extractor. Then, a common space mutual learning module is developed to map the specific features into a common space. Here, the modality discrepancy is reduced from the aspects of features and their distributions. Finally, to supplement the specific knowledge to the common features, we develop modality transformation and the dual-modality feature learning modules. Their function is to transmit the specific knowledge from different sources mutually and fuse the specific and common features adaptively. The comprehensive experiments are conducted on a public dataset. Compared with many existing methods, the behavior of our DMCL is stronger. These encouraging results for a public dataset indicate that the proposed DMCL is useful in CS-CBRSIR tasks.

Semantic segmentation is a fundamental task in remote sensing image understanding. Recently, Deep Convolutional Neural Networks (DCNNs) have considerably improved the performance of the semantic segmentation of natural scenes. However, it is still challenging for very high-resolution remote sensing images. Due to the large and complex scenes as well as the influence of illumination and imaging angle, it is particularly difficult for the existing methods to accurately obtain the category of pixels at object boundaries—the so-called boundary blur. We propose a framework called Boundary-Aware Semi-Supervised Semantic Segmentation Network (BAS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> Net), which obtains more accurate segmentation results without additional annotation workload, especially at the object boundaries. The Channel-weighted Multi-scale Feature (CMF) module balances semantic and spatial information, and the Boundary Attention Module (BAM) weights the features with rich semantic boundary information to alleviate the boundary blur. Additionally, to decrease the amount of difficult and tedious manual labeling of remote sensing images, a discriminator network infers pseudolabels from unlabeled images to assist semisupervised learning, and further improves the performance of the segmentation network. To validate the effectiveness of the proposed framework, extensive experiments have been performed on both the ISPRS Vaihingen dataset, and the novel remote sensing dataset AIR-SEG with more categories, and complex boundaries. The results demonstrate a significant improvement of accuracy especially on boundaries and for small objects.

The presence of shadows in high-resolution (HR) remote-sensing images reduces object detection accuracy. To address this problem, in this paper, we proposed a deep neural network algorithm for shadow detection by using the AISD and SSAD remote-sensing shadow image datasets. To improve the ability to extract spatial information from feature maps, we developed a cross-spatial attention module that focuses on semantic information in the horizontal and vertical directions at each position point on the remote-sensing image. This module overcomes the limitations of existing technologies in accurately judging small areas and suspected shadow areas and in missing or incorrectly detected shadow areas. In addition, to improve the ability to extract shadow features and the accuracy of shadow detection in remote-sensing images, we developed a channel attention module that assigns more attention to channels that conform to the shadow color characteristics. The network architecture comprises an encoder – decoder structure, with ResNeXt50 used as the backbone for the encoder and a multi resolution parallel fusion (MRPF) designed for the decoder; cross-spatial and channel attention were incorporated into the decoder unit. Experimental results demonstrated the superior performance of the proposed algorithm, with an F1 score of 92.6% for the shadow category on the test set, thus, outperforming other algorithms and making the proposed method an effective solution for shadow detection in HR remote-sensing images.

Building extraction from very high-resolution remote sensing images is a fundamental task and is widely used in applications such as change detection, disaster assessment, and real-time update of geographic information databases. However, due to the complexity of the geographical environment and the diversity of target features, accurate automatic building extraction remains very challenging. With the fast development of deep learning techniques, convolutional neural networks(CNN) have been widely used in remote sensing research and have achieved considerable results. But for large urban area-based building detection tasks, the CNN-based method usually gets into local optima and generates many false positive detections around building boundaries. To avoid the local optima and be aware of non-local information, this paper proposes a hybrid feature extraction model based on the combination of the CNN and Transformer to realize the automatic building detection from very high-resolution remote sensing images. Meanwhile, a multi-constraint weighting mechanism is proposed to enhance the ability of the model to recognize the regular geometric boundaries of buildings. Comprehensive experiments are conducted on the three different datasets. The proposed MC-TRANSU achieves the best F1-score, IoU, compared to the state-of-the-art methods such as SegNet, TransUnet, Swin-Unet, the detection accuracy improved around 5%. Quantitative and qualitative results verify the superiority and effectiveness of our model.

As the number of internet users increases so does the number of malicious attacks using malware. The detection of malicious code is becoming critical, and the existing approaches need to be improved. Here, we propose a feature fusion method to combine the features extracted from pre-trained AlexNet and Inception-v3 deep neural networks with features attained using segmentation-based fractal texture analysis (SFTA) of images representing the malware code. In this work, we use distinctive pre-trained models (AlexNet and Inception-V3) for feature extraction. The purpose of deep convolutional neural network (CNN) feature extraction from two models is to improve the malware classifier accuracy, because both models have characteristics and qualities to extract different features. This technique produces a fusion of features to build a multimodal representation of malicious code that can be used to classify the grayscale images, separating the malware into 25 malware classes. The features that are extracted from malware images are then classified using different variants of support vector machine (SVM), k-nearest neighbor (KNN), decision tree (DT), and other classifiers. To improve the classification results, we also adopted data augmentation based on affine image transforms. The presented method is evaluated on a Malimg malware image dataset, achieving an accuracy of 99.3%, which makes it the best among the competing approaches.

Oil is an important resource for the development of modern society. Accurate detection of oil wells is of great significance to the investigation of oil exploitation status and the formulation of an exploitation plan. However, detecting small objects in large-scale and high-resolution remote sensing images, such as oil wells, is a challenging task due to the problems of large number, limited pixels, and complex background. In order to overcome this problem, first, we create our own oil well dataset to conduct experiments given the lack of a public dataset. Second, we provide a comparative assessment of two state-of-the-art object detection algorithms, SSD and YOLO v4, for oil well detection in our image dataset. The results show that both of them have good performance, but YOLO v4 has better accuracy in oil well detection because of its better feature extraction capability for small objects. In view of the fact that small objects are currently difficult to be detected in large-scale and high-resolution remote sensing images, this article proposes an improved algorithm based on YOLO v4 with sliding slices and discarding edges. The algorithm effectively solves the problems of repeated detection and inaccurate positioning of oil well detection in large-scale and high-resolution remote sensing images, and the accuracy of detection result increases considerably. In summary, this study investigates an appropriate algorithm for oil well detection, improves the algorithm, and achieves an excellent effect on a large-scale and high-resolution satellite image. It provides a new idea for small objects detection in large-scale and high-resolution remote sensing images.

Hepatic cystic echinococcosis (HCE), a life-threatening liver disease, has 5 subtypes, i.e., single-cystic, polycystic, internal capsule collapse, solid mass, and calcified subtypes. And each subtype has different treatment methods. An accurate diagnosis is the prerequisite for effective HCE treatment. However, clinicians with less diagnostic experience often make misdiagnoses of HCE and confuse its 5 subtypes in clinical practice. Computer-aided diagnosis (CAD) techniques can help clinicians to improve their diagnostic performance. This paper aims to propose an efficient CAD system that automatically differentiates 5 subtypes of HCE from the ultrasound images. The proposed CAD system adopts the concept of deep transfer learning and uses a pre-trained convolutional neural network (CNN) named VGG19 to extract deep CNN features from the ultrasound images. The proven classifier models, k - nearest neighbor (KNN) and support vecter machine (SVM) models, are integrated to classify the extracted deep CNN features. 3 distinct experiments with the same deep CNN features but different classifier models (softmax, KNN, SVM) are performed. The experiments followed 10 runs of the five-fold cross-validation process on a total of 1820 ultrasound images and the results were compared using Wilcoxon signed-rank test. The overall classification accuracy from low to high was 90.46 ± 1.59% for KNN classifier, 90.92 ± 2.49% for transfer learned VGG19, and 92.01 ± 1.48% for SVM, indicating SVM classifiers with deep CNN features achieved the best performance (P < 0.05). Other performance measures used in the study are specificity, sensitivity, precision, F1-score, and area under the curve (AUC). In addition, the paper addresses a practical aspect by evaluating the system with smaller training data to demonstrate the capability of the proposed classification system. The observations of the study imply that transfer learning is a useful technique when the availability of medical images is limited. The proposed classification system by using deep CNN features and SVM classifier is potentially helpful for clinicians to improve their HCE diagnostic performance in clinical practice.

Based on analysis of roads features in high resolution remote sensing (RS) images, we improve the traditional algorithm for automatic edge detection and extraction and put forward a new methodology. Mathematical morphology reconstruction and boundary tracing algorithm are introduced to automatically detect and extract edges from high resolution RS images. Three factors, P factor, F factor and C factor, are defined to describe the shape characteristics of edges. K-mean cluster algorithm is then utilized to identify and extract road edges from the huge edge sets. Finally an example gives fine result of this methodology.

Over the last years, Convolutional Neural Networks (CNNs) have been widely used in remote sensing applications, such as marine surveillance, traffic management or road networks detection. However, since CNNs have extremely high computation, bandwith and memory requirements, the hardware implementation of a CNN on space-grade devices like FPGAs for the on-board processing of the acquired images has brought many challenges. In fact, FPGAs show great advantages in terms of reconfigurability and performance-power ratio, whichmakes them a suitable election to deploy CNNs, although the computational capabilities of this type of hardware devices available on-board is limited, so implementations have to be carefully planned. In this paper, the authors present their work towards the implementation of an efficient CNN onto a spacegrade FPGA in order to achieve the on-board processing of very-high resolution remotely sensed images as soon as the data are provided by the sensor. All this work has been conducted within the EU-funded VIDEO project. As it will be presented in this paper, the work includes the introduction of a methodology based on the project constraints, the evaluation of different state-of-the-art CNN architectures by means of a new efficiency measurement also proposed in this work, the introduction of a new efficient CNN architecture, and finally, its optimized hardware implementation by means of high-level synthesis tools.

In view of the characteristics of high score remote sensing data containing rich semantic information, combined with different processing requirements for high-score remote sensing images, this paper designs and implements a parallel classification and detection system based on high-score remote sensing image. With the development of remote sensing technology and artificial intelligence technology, it is of great significance to process high-score remote sensing image data based on deep learning convolution neural network. The system integrates sea land segmentation model and target detection model, which can realize different types of data processing requirements. It can realize the high-efficiency processing of large-scale images by means of parallel processing technology, and realize the detection effect of the specified targets by visual processing technology. It is applicable to the processing tasks of high-resolution remote sensing images of all types, and has a high level of intelligent and parallel processing.

As the rapid development of earth observation technology and deep learning, building extraction from remotely sensed imagery based on deep convolutional neural networks (DCNNs) has attracted wide attention in recent years. However, due to the heterogeneity of building shapes and sizes and the complexity of the surrounding objects, current building extraction methods still have challenges in boundary accuracy and complete building extraction. For these purposes, we proposed low-level feature enhancement and multi-scale attention pyramid aggregation network (LFEMAP-Net) that considers building boundary information and multi-scale feature expression to obtain higher accuracy building extraction. Firstly, low-level feature enhancement model is proposed based on prior edge information to enhance the representation of spatial details, effectively addressing issues related to information loss and fuzzy boundaries. Additionally, a multi-scale attention pyramid aggregation model is developed during the decoding stage to facilitate the fusion of features from different scales, thereby enhancing the extraction of building features. Experimental results on two publicly available datasets validate that LFEMAP-Net can overcome building extraction interruptions and boundary blur in complex scenes, and achieve boundary optimization and complete segmentation of buildings and achieve better performance than other advanced semantic segmentation models.

In recent years, there has been a significant increase in the production of high-resolution aerial and satellite imagery. Analyzing and extracting urban features from these images, especially building components, is a significant challenge in photogrammetry and remote sensing. Meanwhile, deep convolution neural networks have been used as a powerful model in the semantic segmentation of building features. However, due to the structure of these types of deep convolution networks, accurate retrieval of building boundary information during training will be difficult, and this will lead to ambiguous boundary areas. On the other hand, the use of distributed-based loss functions, such as the binary cross-entropy loss alone cannot improve the segmentation accuracy in the boundary areas of building features. Accordingly, in this paper, a derivative boundary loss function is introduced to optimize and enhance the extraction of buildings in border areas. To calculate the proposed boundary loss function, distance transform image (DTI), is obtained from ground-truth images and predicted images, that derived from the segmentation network. In this paper, a derivative method for calculating DTI is presented. Another advantage of the proposed loss function is its ability to be used in a wide range of deep convolution segmentation networks. The proposed method was investigated and tested using the ISPRS Potsdam and Vaihingen data sets. The results of this implementation show the superiority in the criteria of evaluation and improvement of buildings extraction if the proposed boundary loss function is used along with the distributed-based loss function.

In recent decades, high-resolution (HR) remote sensing images have shown considerable potential for providing detailed information for change detection. The traditional change detection methods based on HR remote sensing images mostly only detect a single land type or only the change range, and cannot simultaneously detect the change of all object types and pixel-level range changes in the area. To overcome this difficulty, we propose a new coarse-to-fine deep learning-based land-use change detection method. We independently created a new scene classification dataset called NS-55, and innovatively considered the adaptation relationship between the convolutional neural network (CNN) and the scene complexity by selecting the CNN that best fit the scene complexity. The CNN trained by NS-55 was used to detect the category of the scene, define the final category of the scene according to the majority voting method, and obtain the changed scene by comparison to obtain the so-called coarse change result. Then, we created a multi-scale threshold (MST) method, which is a new method for obtaining high-quality training samples. We used the high-quality samples selected by MST to train the deep belief network to obtain the pixel-level range change detection results. By mapping coarse scene changes to range changes, we could obtain fine multi-type land-use change detection results. Experiments were conducted on the Multi-temporal Scene Wuhan dataset and aerial images of a particular area of Dapeng New District, Shenzhen, where promising results were achieved by the proposed method. This demonstrates that the proposed method is practical, easy-to-implement, and the NS-55 dataset is physically justified. The proposed method has the potential to be applied in the large scale land use fine change detection problem and qualitative and quantitative research on land use/cover change based on HR remote sensing data.

We present a method to combine the Fisher vector representation and the Deep Convolutional Neural Network (DCNN) features to generate a rerpesentation, called the Fisher vector encoded DCNN (FV-DCNN) features, for unconstrained face verification. One of the key features of our method is that spatial and appearance information are simultaneously processed when learning the Gaussian mixture model to encode the DCNN features. Evaluations on two challenging verification datasets show that the proposed FV-DCNN method is able to capture the salient local features and also performs well when compared to many state-of-the-art face verification methods.

Road extraction from high-resolution remote sensing images (HRSIs) is one of the tasks in image analysis. Deep convolutional neural networks have become the primary method for road extraction due to their powerful feature representation capability. However, roads are often obscured by vegetation, buildings, and shadows in HRSIs, resulting in incomplete and discontinuous road extraction results. To address this issue, we propose a lightweight post-processing network called RIRNet in this study, which include an information inference module and a road direction inference task branch. The information inference module can infer spatial information relationships between different rows or columns of feature images from different directions, effectively inferring and repairing road fractures. The road direction inference task branch performs the road direction prediction task, which can constrain and promote the road extraction task, thereby indirectly enhancing the inference ability of the post-processing model and realizing the optimization of the initial road extraction results. Experimental results demonstrate that the RIRNet model can achieve an excellent post-processing effect, which is manifested in the effective repair of broken road segments, as well as the handling of errors such as omission, misclassification, and noise, proving the effectiveness and generalization of the model in post-processing optimization.