An Enhanced Residual Feature Fusion Network Integrated with a Terrain Weight Module for Digital Elevation Model Super-Resolution

Recent research on single image super-resolution (SISR) using convolutional neural networks (CNNs) with the utilization of residual structures and attention mechanisms to utilize image features has demonstrated excellent performance. However, previous SISR techniques mainly integrated extracted image features within a deep or wide network architecture, ignoring the interaction between multiscale features and the diversity of features. At the same time, SISR is also a typical ill-posed problem in that it allows for several predictions for a given LR image. These problems limit the great learning ability of CNNs. To solve these problems, we propose a closed-loop residual attention network (CLRAN) to extract and interact with all the available diversity of features features efficiently and limit the space of possible function solutions. Specifically, we design an enhanced residual attention block (ERA) to extract features, and it dynamically assigns weight to the internal attention branches. The ERA combines multi-scale block (MSB) and enhanced attention mechanism (EAM) base on the residual module. The MSB adaptively detects multiscale image features of different scales by using different 3 × 3 convolution kernels. The EAM combines multi-spectral channel attention (MSCA) and spatial attention (SA). Therefore, the EAM extracts different frequency component information and spatial information to utilize the diversity features. Furthermore, we apply the progressive network architecture and learn an additional map for model monitoring, which forms a closed-loop with the mapping already learned by the LR to HR function. Extensive experiments demonstrate that our CLRAN outperforms the state-of-the-art SISR methods on public datasets for both ×4 and ×8, proving its accuracy and visual perception.

Recently, single image super-resolution (SISR) based on deep convolutional neural networks (CNNs) has been widely studied and achieved remarkable results. However, most of these methods mainly widen or deepen the network to get better results, ignoring the self-similarity in natural images. Some works have successfully leveraged this self-similarity by exploring non-local attention modules, but they only use non-local attention modules at one scale or two scales. And they need to calculate the self-similarity in each residual module, which makes the network more complex and takes a long time to train. We propose a multiple-scale self-similarity module (MSSS) that can be flexibly inserted into existing various super-resolution networks. MSSS uses convolution with different kernels and the trailing dilated convolution to obtain feature maps at different scales, and then calculates the correlation between multiple feature maps at different scales through two-head region-level recurrent criss-cross attention modules (TRRCCA), finally gets fusion output through channel attention. So MSSS can mine multiple-scale self-similarities. We only insert a MSSS module into the basic residual SISR network, and the result is significantly improved.

Higher performance is the eternal purpose for super-resolution (SR) methods to pursue. Since the deep convolution neural network is introduced into this issue successfully, many SR methods have achieved impressive results. To further improve the accuracy that current SR methods have achieved, we propose a high-accuracy deep convolution network (HDCN). In this article, deeper network structure is deployed for reconstructing images with a fixed upscaling factor and the magnification becomes alternative by cascading HDCN. \(L_2\) loss function is substituted by a more robust one for reducing the blurry prediction. In addition, gradual learning is adopted for accelerating the rate of convergence and compacting the training process. Extensive experiment results prove the effectiveness of these ingenious strategies and demonstrate the higher-accuracy of proposed model among state-of-the-art SR methods.

To solve the problem of missing data features using a deep convolutional neural network (DCNN), this paper proposes an improved gesture recognition method. The method first extracts the time-frequency spectrogram of surface electromyography (sEMG) using the continuous wavelet transform. Then, the Spatial Attention Module (SAM) is introduced to construct the DCNN-SAM model. The residual module is embedded to improve the feature representation of relevant regions, and reduces the problem of missing features. Finally, experiments with 10 different gestures are done for verification. The results validate that the recognition accuracy of the improved method is 96.1%. Compared with the DCNN, the accuracy is improved by about 6 percentage points.

In recent years, research on increasing the spatial resolution and enhancing the quality of satellite images using the deep learning-based super-resolution (SR) method has been actively conducted. In a remote sensing field, conventional SR methods required high-quality satellite images as the ground truth. However, in most cases, high-quality satellite images are difficult to acquire because many image distortions occur owing to various imaging conditions. To address this problem, we propose an adaptive image quality modification method to improve SR image quality for the KOrea Multi-Purpose Satellite-3 (KOMPSAT-3). The KOMPSAT-3 is a high performance optical satellite, which provides 0.7-m ground sampling distance (GSD) panchromatic and 2.8-m GSD multi-spectral images for various applications. We proposed an SR method with a scale factor of 2 for the panchromatic and pan-sharpened images of KOMPSAT-3. The proposed SR method presents a degradation model that generates a low-quality image for training, and a method for improving the quality of the raw satellite image. The proposed degradation model for low-resolution input image generation is based on Gaussian noise and blur kernel. In addition, top-hat and bottom-hat transformation is applied to the original satellite image to generate an enhanced satellite image with improved edge sharpness or image clarity. Using this enhanced satellite image as the ground truth, an SR network is then trained. The performance of the proposed method was evaluated by comparing it with other SR methods in multiple ways, such as edge extraction, visual inspection, qualitative analysis, and the performance of object detection. Experimental results show that the proposed SR method achieves improved reconstruction results and perceptual quality compared to conventional SR methods.

Synthetic aperture radar (SAR) image super-resolution (SR) has broad application prospects in military reconnaissance, marine monitoring, crop monitoring, and other fields. However, when the scale factor is large, current SAR image SR has limitations, such as insufficient feature extraction, difficulty of feature representation, and modeling difficulty of SR reconstruction. These problems have severely restricted the SR performance of SAR images. In addition, optical remote sensing images share higher resolution and richer high-frequency information than SAR remote sensing images. Therefore, we adopt high-resolution (HR) optical images as auxiliary images of SAR image SR to improve the performance of SAR image SR. Our research is conducted from two aspects: the extraction efficiency of HR optical image-guided information and the representation ability of low-resolution (LR) SAR image features. We propose an optical guidance residual networks (OGRNs) based on residual learning for SAR image SR. The OGRN uses optical images during network training, which enhances the performance of SAR image SR, even when optical images are missing during network testing. First, we extract the texture feature of SAR images through the designed residual learning module. Second, we extract the weight map of feature information for SAR image SR from HR optical images by the convolutional layer. Finally, the weight map is used to guide the weighted supervised SR reconstruction for SAR images. Extensive experimental results demonstrated that, under the guidance of optical images, OGRN achieves excellent performance in terms of both quantitative assessment metrics and visual quality.

Feedback attention network for cardiac magnetic resonance imaging super-resolution

As the acquisition of very high resolution (VHR) images becomes easier, the complex characteristics of VHR images pose new challenges to traditional machine learning semantic segmentation methods. As an excellent convolutional neural network (CNN) structure, U-Net does not require manual intervention, and its high-precision features are widely used in image interpretation. However, as an end-to-end fully convolutional network, U-Net has not explored enough information from the full scale, and there is still room for improvement. In this study, we constructed an effective network module: residual module under a multisensory field (RMMF) to extract multiscale features of target and an attention mechanism to optimize feature information. RMMF uses parallel convolutional layers to learn features of different scales in the network and adds shortcut connections between stacked layers to construct residual blocks, combining low-level detailed information with high-level semantic information. RMMF is universal and extensible. The convolutional layer in the U-Net network is replaced with RMMF to improve the network structure. Additionally, the multiscale convolutional network was tested using RMMF on the Gaofen-2 data set and Potsdam data sets. Experiments show that compared to other technologies, this method has better performance in airborne and spaceborne images.

In this paper, we propose a simple but powerful idea to improve super-resolution (SR) methods based on convolutional neural networks (CNNs). We consider a linear manifold, which is the set of all SR images whose downsampling results are the same as the input image, and apply the orthogonal projection onto this linear manifold in the output layers of the CNNs. The proposed method can guarantee the consistency between the SR image and the input image and reduce the mean squared error. The proposed method is especially effective for SR methods based on generative adversarial networks (GANs), composed of one generator and one discriminator, since the generator can learn high-frequency components while maintaining low-frequency ones. Experiments show the effectiveness of the proposed technique for a GAN-based SR method. Finally we introduce an idea of extension to noisy images.

Convolutional neural networks (CNNs) have recently made great progress in single image super-resolution (SISR) due to their powerful feature representation. However, most existing CNN-based SR methods mainly focus on designing deeper architectures to obtain superior representations, neglecting the global feature correlations inside networks, thus limiting the discriminative learning ability. In this paper, we propose a spatial graph convolutional network (SGCN) to amplify feature representation for high quality image rendering. Specifically, we present a residual feature refinement module (RFRM) in SGCN to encode correlations in both channel and spatial dimensions of features. A novel spatial graph attention (SGA) is deployed to model the correlations with awareness of global information for the feature enhancement. Further-more, we utilize the Gram matrix to infer the global relations in features, which adaptively measures the pixel-wise spatial attention with free parameters. Experimental results demonstrate the superiority of our SGCN over state-of-the-art SISR methods in terms of quantitative metrics and visual quality.

Over the past few years, deep learning has been introduced to tackle hyperspectral image (HSI) classification and demonstrated good performance. In particular, the convolutional neural network (CNN) based methods have progressed. However, due to the high dimensionality of HSI and equal treatment of all bands, the performances of CNN based methods are hampered. The labels of land-covers often differ between edge and the center pixels in pixel-centered spatial information. These edge pixels may weaken the discrimination of spatial features and reduce classification accuracy. Motivated by the attention mechanism of the human visual system, the spatial proximity feature selection with residual spatial–spectral attention network is proposed in this article. It contains a residual spatial attention module, a residual spectral attention module, and a spatial proximity feature selection module. The residual spatial attention module aims to select the crucial spatial information, which assigns weights to different features by measuring the similarity between the surrounding elements and their central ones. The residual spectral attention module is designed for spectral bands which are selected from raw input data by emphasizing the valuable bands and suppressing the valueless. According to the spatial distribution of features, the spatial proximity feature selection module is used to filter features effectively. Experiments on three public data sets demonstrate that the proposed network outperforms the state-of-the-art methods in comparison.

Recently, the deep convolutional neural network (CNN) has made remarkable progress in single image super resolution(SISR). However, blindly using the residual structure and dense structure to extract features from LR images, can cause the network to be bloated and difficult to train. To address these problems, we propose a simple and efficient distilling with residual network(DRN) for SISR. In detail, we propose residual distilling block(RDB) containing two branches, while one branch performs a residual operation and the other branch distills effective information. To further improve efficiency, we design residual distilling group(RDG) by stacking some RDBs and one long skip connection, which can effectively extract local features and fuse them with global features. These efficient features beneficially contribute to image reconstruction. Experiments on benchmark datasets demonstrate that our DRN is superior to the state-of-the-art methods, specifically has a better trade-off between performance and model size.

Deep-learning based approaches have been proven effective for Digital Elevation Model (DEM) super-resolution (SR) tasks. Previous networks typically treat DEM elevation values as single-channel image for input. However, DEM images alone cannot fully capture spatial and terrain features. Shaded relief images (SRIs), derived from DEMs, serve as crucial visual cues that intuitively convey terrain characteristics, addressing the limitations of DEM images and providing synergistic benefits for training DL models. The primary challenge in utilizing SRIs for guiding DEM SR lies in accurately selecting a consistent structure to extract and effectively integrate features from SRIs and DEMs. In this study, we propose an Attention-based Hierarchical Terrain Fusion (AHTF) framework for guided DEM SR. Specifically, an Attention-based Feature Fusion Module (AFFM) is designed to efficiently fuse relevant information from LR DEM and SRI, which includes a feature enhancement block to select valuable features and a feature recalibration block to fuse diverse terrain features. Additionally, we optimize the loss function from the perspectives of terrain analysis and visual effects. We validate AHTF on our newly constructed real-world Shade-DEM SR dataset and two open-source DEM SR datasets. Compared to the current state-of-the-art methods, our AHTF achieves the best results in terms of root mean square error (RMSE) for elevation, slope, and aspect. Furthermore, the extracted stream networks are closer to real-world conditions. This study offers new insights and methods for further research and application in the field of DEM super-resolution. Our dataset can be obtained at https://doi.org/10.6084/m9.figshare.25590945.

Forecasting stock prices has always been a complex and challenging task due to its high volatility, noise in data, and abrupt price fluctuations caused by different factors. Traditional approaches, such as quantitative, fundamental, and technical analysis, often struggle to accurately predict stock price movements due to their difficulty in processing non-linear data. The evolution of machine learning, particularly neural network models, opened new opportunities for improving prediction accuracy by handling non-linear and multidimensional data with high effectiveness. This study focuses on evaluating and comparing three neural network architectures, in which the last is a combination of the first two - Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), and a hybrid CNN-LSTM model, for forecasting stock prices. Using ten years of historical stock data, taken for Apple Inc. (AAPL) stocks as an example, ranging from 2015 to 2025, the models were trained and tested on key performance metrics such as Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), and R². Results show that while both CNN and LSTM models show good performance individually, the hybrid CNN-LSTM model can significantly outperform them, achieving the highest accuracy with the lowest error rates. By combining the strengths of CNN for feature extraction and LSTM for temporal sequence processing, the hybrid model effectively captures complex patterns in stock price movements. By iterating over the types of layers to use, we came to the conclusion, that by combining both models into one hybrid model, some layers, like one CNN layer, and one LSTM layer should be removed. This research demonstrates the potential of hybrid neural network models in financial time-series forecasting and comprehensively analyzes their advantages over non-hybrid models.

Reducing the impact of hazy images on subsequent visual information processing is a challenging problem. In this paper, combining with atmospheric scattering model, we propose an end-to-end multi-scale feature multiple parallel fusion network called MMP-Net for single image haze removal. The MMP-Net includes three components: multi-scale CNN module, residual learning module and deep parallel fusion module. 1) In multi-scale CNN module, a multi-scale convolutional neural network (CNNs) is adopted to extract different scales features from whole to local, and these features are fused multiple times in parallel. 2) In residual learning module, residual blocks are introduced to deeply learn detailed features, which can recover more image details. 3) In deep parallel fusion module, those features from residual learning module are deeply merged with the fused features from CNNs, and finally used to recover a clean haze-free image via the atmospheric scattering model. The experimental results show that on the average of three datasets (SOTS, HSTS, and D-Hazy), proposed MMP-Net improves PSNR from 20.91db to 22.21db and SSIM from 0.8720 to 0.9023 over the best state-of-the-art DehazeNet method. What’s more, MMP-Net gains the best subjective visual quality on real-world hazy images.