Multisource Information Fusion Network for Optical Remote Sensing Image Super-Resolution

Recently, the alternating direction method of multipliers (ADMM) has been used for distributed consensus optimization and is shown to converge faster than conventional approaches based on consensus subgradient. In this paper, we consider a convex optimization problem with a linearly coupled equality constraint and employ a dual consensus ADMM (DC-ADMM) method for solving the problem in a fully distributed fashion. In particular, by considering a non-ideal network where the agents can be ON and OFF randomly and the communications among agents can fail probabilistically, we propose a randomized DC-ADMM method that is robust against these non-ideal effects. Moreover, we show that the proposed randomized method is provably convergent to an optimal solution and has a worst-case O(1/k) convergence rate, where k is the iteration number. Simulation results are presented to examine the practical convergence behavior of the proposed method in the presence of randomly ON/OFF agents and non-ideal communication links.

In microgrids, the volatility and uncertainty of renewable energy make it challenging to achieve optimal performance using hierarchical control strategies with time-scale separation for economic dispatch and real-time control. To address this issue and enhance transient optimization, a distributed economic model predictive control (EMPC) strategy based on the alternating direction method of multipliers (ADMM) is proposed in this paper. To reduce computational complexity and communication overhead in distributed control, the ADMM method is employed to solve the distributed EMPC problem in parallel. Using Lyapunov-based analysis, the study establishes the system's asymptotic stability. Moreover, it demonstrates that the EMPC strategy achieves an average performance equivalent to-or even exceeding-that of the optimal steady-state. Simulations validate the proposed algorithm's effectiveness in economic dispatch and frequency restoration, while demonstrating its advantages in minimizing computational load and communication requirements.

The main aim of optical remote sensing image (ORSI) object detection is to ascertain the area as well as class of all targets in given images. Nowadays, object detection methods established from deep learning are progressively applied to the analysis of the information from ORSI. But because of the compound background as well as extensive object scale shown in ORSI, the difficulty of object detection has increased. There are still some challenges to be solved. First, when the two bounding boxes don't overlap, current indexes like intersection over union (IoU) cannot identify the distance between them. Secondly, the existing methods using convolutional neural network (CNN) could not take advantage of the characteristics of multi-level features so that these methods are not very good at recognizing small-sized objects. To deal with these above problems, this paper comes up with a ORSI object detection method established on multi-level feature fusion as well as improved boundary box regression compared to IoU. Firstly, a new measure called generalized intersection over union (GIoU) is practiced. That metric could measure the similarity betwixt two bounding boxes, regardless of whether they overlap or not. At the same time, we also directly use GIoU as loss. Lastly, a multi-level feature fusion structure is practiced and it is also integrated into the existing convolutional neural network. In this way, our method could take advantage of the multi-level features. Quantitative comparison of the addressed method with the baseline approach on large-scale dataset named Dior are conducted. The comparison with the most advanced method shows that our method has reached the most advanced performance.

Recently, a new two-step tensor-based distributed epileptic source localization method was proposed. Firstly, it performs Canonically Polyiadic (CP) decomposition of a Space-Time-Spike (STS) tensor capturing the occurrence of epileptic events in the ElectroEncephaloGraphic (EEG) data. This with the aim at estimating the spatial map of those distributed sources. Secondly, a source localization step is performed using the well-known Alternating Direction Method of Multipliers (ADMM) algorithm. This paper investigates to what extent this brain epileptic source imaging problem could be properly solved when a proximal gradient-based algorithm, the Proximal Alternating Linearized Minimization (PALM) technique, is used instead of the ADMM when the source localization step is considered. The convergence issue of the PALM scheme in this particular problem is discussed and a link with the one of the ADMM method is established. Furthermore, realistic simulations with epileptic EEG data show the efficiency of the new proposed PALM-based scheme.

Recently, the alternating direction method of multipliers (ADMM) has been used effectively to solve the multi-agent unconstrained optimization problems, where the objective function is the sum of privately known local objective functions of agents. In this paper, first, with the help of the edge-node incidence matrix, an unconstrained optimization problem is transformed into an equivalent optimization problem with only equality constraint and, thus, can be dealt with the ADMM conveniently. Second, a novel distributed inexact consensus ADMM is proposed to enable the agents to reach consensus on the optimal solution of the optimization problem. At the same time, the analysis of the linear convergence of the proposed algorithm is also provided under some mild conditions. Finally, some simulation results are presented to demonstrate the better effectiveness of the proposed algorithm than the standard consensus-based ADMM algorithm.

A joint compressed-sensing and super-resolution approach for very high-resolution diffusion imaging

There are many methods for dealing with spatial uncertainty. However, a separate discussion for attribute uncertainty of geographic information is still rare. With the rapid development of multiple sources of geographic data, the uncertainty of attributes and their fusion is important for accuracy assessment and decision making. Attribute uncertainty of geographic information is an important aspect of geographic information quality and will further affect geographic decision making. This paper has discussed the different types of attribute measurement and uncertainty of geo-data and information fusion schemes for multiple sources of geo-data. Two types of attributes, i.e. continuous variable attribute and categorical variable attribute, and their uncertainty, i.e. random and error, vagueness and ambiguity, are discussed. Five geographic information fusion schemes are given for multi-sources uncertainty attribute information, i.e. (a) univariate multi-sources data fusion, (b) multivariate continuous attributes fusion, (c) multivariate categorical attribute fusion, (d) multivariate normalized attribute fusion and (e) multivariate continuous-category attribute fusion. Methods for different fusion schemes are discussed, i.e. maximum likelihood estimation, multivariate regression, Artificial Neural Network, Dempster-Shafer evidence, multi-attribute decision making (MADA), and ordered weighted averaging (OWA). Finally an example of information fusion for precipitation estimation and flood susceptibility assessment are discussed.

Salient object detection in optical remote sensing images (ORSI-SOD) is a task that detects the most prominent context in optical remote sensing images (ORSIs). ORSI-SOD is challenging due to the diverse sizes and shapes of the targets, their irregular distribution in the scene, and the occlusion caused by surrounding environments. Recently, many deep learning-based models have demonstrated promising performance in ORSI-SOD. However, there still remains considerable potential for addressing these challenges in ORSI-SOD. In this article, we propose a novel architecture, LSHNet. We propose a dual-branch architecture consisting of an edge encoder that leverages structure features using edges as the structure-prior and an image encoder that extracts context features from the image. We propose three modules. Image-structure fusion module (ISFM) integrates the two-stream features extracted from dual branch encoders through intrapatch and internal-patch attention mechanisms to utilize diverse receptive fields. Local-global feature fusion module (LGFM) transfers global features representing the target to local feature maps to discriminate the region of the targets from background clutters. The semantic cues updating module (SCUM) updates the representative features of the target from high-level to low-level. By integrating hierarchical information effectively, global features extracted from multilayers can be rectified. We experiment with the three main evaluated datasets in ORSI-SOD: ORSSD, EORSSD, and ORSI-4199. We demonstrate the promising results on the three datasets and analyze the effectiveness of the proposed modules in the ablation study.

The recognition of sea ice is of great significance for reflecting climate change and ensuring the safety of ship navigation. Recently, many deep-learning-based methods have been proposed and applied to segment and recognize sea ice regions. However, there are huge differences in sea ice size and irregular edge profiles, which bring challenges to the existing sea ice recognition. In this article, a global-local Transformer network, called SeaIceNet, is proposed for sea ice recognition in optical remote sensing images. In SeaIceNet, a dual global-attention head (DGAH) is proposed to capture global information. On this basis, a global-local feature fusion (GLFF) mechanism is designed to fuse global structural correlation features and local spatial detail features. Furthermore, a detail-guided decoder is developed to retain more high-resolution detail information during feature reconstruction for improving the performance of sea ice recognition. Extensive experiments on several sea ice datasets demonstrated that the proposed SeaIceNet has better performance than the existing methods in multiple evaluation indicators. Moreover, it excels in addressing challenges associated with sea ice recognition in optical remote sensing images, including the difficulty in accurately identifying irregular frozen ponds in complex environments, the broken and unclear boundaries between sea and thin ice that hinder precise segmentation, and the loss of high-resolution spatial details during model learning that complicates refinement.

Convolutional neural network-based image super-resolution methods have achieved great success in recent years. However, the huge memory and computational costs make most of the existing methods difficult to be applied to resource-constrained scenarios such as edge devices. To tackle this problem, we propose a generic, lightweight and efficient feature fusion block to replace the commonly used 1*1 convolution. In addition, we propose the enhanced shallow residual blocks to improve the super-resolution performance. By combining these two novel blocks, we design an efficient similar feature fusion network for single image super-resolution, based on the observation that cross-layer features of the same channel usually have high similarities. The similar feature fusion block utilizes similarity as a guide for feature clustering, enabling efficient and high-performance cross-layer feature fusion. On the other hand, the enhanced shallow residual blocks are used as the base feature extraction model for the network to improve super-resolution performance in conjunction with the feature fusion module. In the enhanced shallow residual blocks, we combine convolution with identity connection to maintain the similarity of cross-layer features that are fed into the similar feature fusion block. The spatial attention mechanism is also introduced to reinforce the useful spatial features. Experimental results on the benchmark datasets show that the proposed method can achieve comparable results to state-of-the-art methods with a small number of parameters.

Small ships in optical remote sensing images are hard to detect due to the lack of sufficient detail information. In this paper, we adopt the image super-resolution technology to solve this problem. Specifically, an effective super-resolution network is designed to generate clear super-resolution ship images from small blurry ones produced by the ship detector. Inspired by the idea of generative adversarial network (GAN), the super-resolution network is trained together with a discriminator network in an adversarial way, aiming at generating more realistic super-resolution images. Moreover, to eliminate false detections, the discriminator network is also used to distinguish ship and non-ship images via an additional classification branch. Experimental results demonstrate the effectiveness of the proposed method.

Adversarially trained deep neural networks have significantly improved performance of single image super resolution, by hallucinating photorealistic local textures, thereby greatly reducing the perception difference between a real high resolution image and its super resolved (SR) counterpart. However, application to medical imaging requires preservation of diagnostically relevant features while refraining from introducing any diagnostically confusing artifacts. We propose using a deep convolutional super resolution network (SRNet) trained for (i) minimising reconstruction loss between the real and SR images, and (ii) maximally confusing learned relativistic visual Turing test (rVTT) networks to discriminate between (a) pair of real and SR images (T1) and (b) pair of patches in real and SR selected from region of interest (T2). The adversarial loss of T1 and T2 while backpropagated through SRNet helps it learn to reconstruct pathorealism in the regions of interest such as white blood cells (WBC) in peripheral blood smears or epithelial cells in histopathology of cancerous biopsy tissues, which are experimentally demonstrated here. Experiments performed for measuring signal distortion loss using peak signal to noise ratio (pSNR) and structural similarity (SSIM) with variation of SR scale factors, impact of rVTT adversarial losses, and impact on reporting using SR on a commercially available artificial intelligence (AI) digital pathology system substantiate our claims.

Deblurring, denoising and super-resolution (SR) are important image recovery tasks that are committed to improving image quality. Despite the rapid development of deep learning and vast studies on improving image quality have been proposed, the most existing recovery solutions simply deal with quality degradation caused by a single distortion factor, such as SR focusing on improving spatial resolution. Since very little work has been done to analyze the interaction and characteristics of the deblurring, denoising and SR mixing problems, this paper considers the multi-distortion image recovery problem from a holistic perspective and introduces an end-to-end multi-stream fusion network (MSFN) to restore a multi-distortion image (low-resolution image with noise and blur) into a clear high-resolution (HR) image. Firstly, MSFN adopts multiple reconstruction branches to extract deblurring, denoise and SR features with respect to different degradations. Then, MSFN gradually fuses these multi-stream recovery features in a determined order and obtains an enhanced restoration feature by using two fusion modules. In addition, MSFN uses fusion modules and residual attention modules to facilitate the fusion of different recovery features from the denoising branch and the deblurring branch for the trunk SR branch. Experiments on several benchmarks fully demonstrate the superiority of our MSFN in solving the multi-distortion image recovery problem.

With the rapid growth of Internet video image information, there is a large amount of redundancy in image data. Use less data stream information to transfer the image or the amount of information contained in the image. Its purpose is to reduce the redundancy of images, so as to store them at low bit rate and reduce the data storage space. In the general image compression method, the hybrid coding framework is adopted. Each algorithm adopts a fixed algorithm mode through a specific design algorithm, without global optimization. Image compression is mainly divided into prediction, transformation, quantization, digital entropy coding and other steps. At present, there are many researches on super-resolution network based on deep learning technology. The main function is to reconstruct high-resolution image replace image magnification low-resolution images such as linear interpolation, which has a great performance improvement image resolution, noise reduction, deblurring and so on, but there is no effective way to use super-resolution network applications to improve quality of compression reconstructed image quality. This paper involves a new method that using image super-resolution residual learning network to improve quality of compression image, our method, the reduced image is encoded into a content stream and a transmission corresponding parameter is encoded into a model stream. Firstly, the original image is scaled down 1/2 size of source image, then encode the small image into content stream with the existing codec. Secondly, the residual learning super-resolution (SR) network is used for image filtering to scale up reconstructed image with decode image resizing method and boost the quality of edge feature extraction of image. Our results show that there is significant performance improvement of h265 in low resolution reconstructed image (bits-per-pixel less than 0.1).

Abstract. As deep learning models advance, their use in Super Resolution (SR) tasks has become pivotal for enhancing remote sensing low-resolution (LR) satellite images. However, the decision making processes within these models remain opaque, especially in remote sensing applications where transparency is critical. This paper focuses on applying Explainable Artificial Intelligence (XAI) techniques, particularly Local Attribution Maps (LAMs), to analyze and interpret the internal behavior of both general purpose and remote sensing specific SR neural networks. General purpose models like Generative Adversarial Network for Super Resolution (SRGAN), Enhanced Deep Super-Resolution Network (EDSR), Efficient Super-Resolution Transformer (ESRT), and Hybrid-Attention Transformer (HAT), although highly effective in SR tasks, were originally designed for broader image enhancement challenges; in contrast, Hybrid-Scale Self-Similarity Exploitation Network (HSENet) and Multi-scale Enhanced Network (MEN) are tailored for the unique complexities of remote sensing, such as varied textures and intricate scene features. By leveraging LAMs, we highlight how different networks prioritize and process features such as edges, textures, and high frequency details to generate super resolved outputs. The comparative study between general purpose and remote sensing specific networks outlines each model’s strengths and weaknesses in managing the data present in remote sensing imagery. This approach addresses the previously unexplored application of LAMs in remote sensing for SR, contributing to the research in this field and providing deeper insights into the interpretability and transparency of widely used SR models. Furthermore, by drawing parallels between feature attribution in SR and classification tasks, we suggest new pathways for integrating semantic information to refine model transparency and performance in remote sensing applications.