Lightweight Stereo Image Super-Resolution Using modified Parallax Attention

Efficient masked feature and group attention network for stereo image super-resolution

This paper reviews some of the methods of super resolution (SR) (with multiple images and single image as input) with a focus on super resolution for stereo images with their advantages and disadvantages. This paper has attempted to fill the void of non availability of the survey of SR techniques for stereo image.

This letter presents a novel approach for the Synthetic Aperture Radar (SAR) stereo imaging based on the Capon spectrum estimation technique. In order to deal with nonuniform sampling space and lead to super resolution in the elevation direction, Capon approach is used to focus the SAR data on a certain height. Results obtained on simulated data demonstrate the feasibility of the Capon based algorithm. Compared with the classical Fast Fourier Transform (FFT), the Capon based algorithm shows better resolution quality.

With asymmetric resolution stereo images as input, existing stereo matching algorithms significantly decline in prediction performance. To address this, we introduce SGANet (Super-resolution Guided Asymmetric Stereo Matching Network), a model that employs unsupervised training methods to overcome the difficulty of acquiring ground truth disparity. For the lower resolution side, this paper designs a stereo guided super-resolution module (SGSR), where the network generates a super-resolved image enriched with details guided by the higher resolution side. Additionally, for this module, we propose a feature consistency loss based on the image's feature space to measure the similarity between the real and super-resolved images. Experimental results on the autonomous driving dataset KITTI demonstrate the effectiveness of the SGSR module and the feature consistency loss in improving the disparity prediction performance of asymmetric resolution stereo images.

The spatial resolution and measurement accuracy of the digital image correlation (DIC) method are constrained by camera resolution. This limitation is primarily determined by hardware costs. However, in current stereo DIC measurements, only the gray level or its gradient from two images is used for integer-pixel matching and sub-pixel optimization. It implicitly treats the two images from different viewpoints as independent entities before correlating them. However, the inherent structural information has not been fully utilized. This previously overlooked structural information provides a novel approach to enhancing the accuracy of DIC by leveraging the inherent correlations between stereo image pairs. The realization of binocular super-resolution typically requires a relatively small parallax. Moreover, the DIC method can achieve image window pairing with small parallax through pre-matching. This implies that binocular super-resolution and Stereo-DIC can complement each other by sharing information. In this paper, the DIC method is employed for whole-pixel image matching, while the binocular super-resolution method, based on deep learning, is applied to process the matched image pairs. Building on previous experiments, extensive datasets containing diverse experimental scenes and various speckle patterns were compiled and utilized. Furthermore, the DIC method can establish training datasets with minimal parallax through integer-pixel matching, thereby achieving highly effective super-resolution results. Experimental results demonstrate that super-resolution images with a higher signal-to-noise ratio can be obtained. Additionally, it effectively provides more image details, which enhance the calculation accuracy and resolution of DIC.

In recent years, stereo image super-resolution based on convolutional neural network has been extensively researched and achieved impressive performance by introducing complementary information from another view. However, most existing methods still cannot fully capture both intra- and cross-view information due to the neglect of multi-scale information perception, multi-scale binocular alignment and the excitation of large scale to small scale in human vision system. And they generated blurry results due to the consideration of irrelevant information in search for cross-view information. To address these issues, we propose a multi-scale visual perception based progressive feature interaction network (MS-PFINet) for stereo image super-resolution. Specifically, to exploit comprehensive intra- and cross-view information for image reconstruction, we design a two-stream network with multi-branch structure to extract multi-scale features and progressively use cross-view interaction at larger scales to guide that at smaller scales. Moreover, to explore more proper and accurate cross-view information, we propose a feature transformer module (FTM) to search and transfer the most relevant features from another view by hard attention maps and soft attention maps, which are calculated by patch-wise similarity rather than pixel-wise. In addition, in order to encourage a more effective way to transfer texture features for the target view, we propose a perceptual texture matching loss to supervise the accuracy of feature transformer modules. Experimental results show that our proposed method is superior to the state-of-the-art methods in most cases.

Stereo image super-resolution (Stereo SR) has been a newly rising and challenging problem with the popular application of dual cameras, which can be used to promote the SR performance by adding auxiliary information from another viewpoint. Most of the existing excellent works have concentrated on leveraging the intrinsic feature correlation of two view images via exploring the non-local attention mechanism. However, they only perform interaction once for feature registration and fusion accompanied by the complex view transition constraint, which cannot fully take advantage of the information in the stereo image pairs. In this paper, we propose a joint feature aggregation network for Stereo SR to calibrate single-view features and integrate cross-view knowledge effectively. Specifically, we introduce a self-calibrated feature extractor to excavate multi-scale and multi-direction features within the single-view image. What's more, we design an adaptive fusion module with the cross-view attention mechanism, which is utilized to mine and fuse the long-range dependencies between the stereo image pairs so as to get rid of the inflexible cycle constraints. Extensive experimental results demonstrate that our proposal successfully achieves superior performance against the state-of-the-art methods on four datasets.

Inter-view dual-domain guided stable diffusion for real-world stereo image super-resolution

MSDBPN: multi-column smoothed dilated convolution based back projection network for stereo image super-resolution

Mars is accompanied by two small natural satellites. Deimos, the smaller of the two, is orbiting at a greater distance from Mars while Phobos revolves around the planet at a mean altitude of approximately 6,000km with respect to the Martian surface, deep in the gravity field of the planet. An in-depth study of the irregular shaped moon Phobos was commenced due to remaining questions on its orbital motion, shape, and physical parameters based on new high resolution images of the Super Resolution Channel (SRC) and the High Resolution Stereo Camera (HRSC) which are part of the European Mars Express Mission (MEX). Parameters of ephemerides models were questioned after the finding of large discrepancies of Phobos' position to astrometric observations. During the course of this study astrometric measurements were made in fly-by images of Phobos and through observations of Phobos' shadow on the Martian surface. While results of the latter analysis led to largely scattered differences compared to orbit prediction models with high uncertainties, the fly-by observations were in agreement with the ephemerides models within the error bands. Offsets were rather consistent and systematic indicating that Phobos is ahead of its predicted position by approximately 2 km. To be able to model the irregular shape of Phobos by means of different techniques - namely a digital terrain model and a model based on spherical harmonic functions - a new global control point network was established. The 665 object point coordinates with respect to the Phobos body-fixed coordinate frame were determined through observations in 53 SRC and 16 Viking Orbiter images. Coordinates of the object points were computed by means of a least-squares bundle block adjustment. The overall accuracy was estimated to be in the order of +/-17 m. The control network proved to be a valuable tool to observe the forced libration amplitude - a superimposed oscillation on the rotation of Phobos due to interactions between the irregularly shaped Phobos and the gravity field of Mars. A global digital terrain model (DTM), which is based on stereo images of the HRSC and Viking Orbiter images, was computed and was used to ortho-rectify SRC and Viking images. Based on the ortho-rectified images an accurate map of Phobos was prepared. Since the DTM representation of Phobos' shape is rather inefficient to obtain physical parameters, the coeffcients of the surface spherical harmonic function to degree and order 17 were determined through the control point coordinates. The analytical expression for Phobos was used to re-estimate the volume, bulk density of Phobos and the moment of inertia tensor. A volume of 5689km³ and a corresponding bulk density of 1.85 g/cm³ were determined. The results were in good agreement, but signiffcantly improved accuracy compared to previous estimates of these values. The moments of inertia were primarily computed assuming a homogeneous mass distribution within Phobos. The principle moments of inertia can be put in relation to the forced libration amplitude. The modeled forced libration amplitude agrees well with the observed value within its error bounds. Applying simple two layer mass distribution models, exact agreement between observation and model of the forced libration amplitude could only be achieved assuming extreme model parameters. This suggests a more complex mass distribution within Phobos.