A spectral-restored fusion method based on wavelet-packet transform

In this study, we used eight millet varieties and took visible-near-infrared hyperspectral images of 480 millet samples. Spectral and image characteristics, including texture and color features, of the millet samples were extracted from the hyperspectral images. Support vector machine (SVM) models for millet variety identification were established using the extracted spectral and image characteristics. An attention-convolutional recurrent neural network (attention- CRNN) model with attention mechanism was introduced for the identification of millet varieties, and the SVM and attention- CRNN models for millet variety identification were established using an image and spectral features fusion method. We found that the highest mathematical transformation method was the reciprocal logarithmic method. The identification accuracy of the SVM cultivar classification model with the reciprocal logarithmic spectral characteristics curve was 73.13%. The overall identification accuracy of the SVM model for the eight millet varieties using the image features was only 61.25%. The identification accuracy of the SVM model using the image and spectral information fusion method greatly improved the overall accuracy rate to 77.5%, and the minimum discrimination accuracy of the millet varieties increased from 50 to 65%. The overall identification accuracy of the attention-CRNN model was 87.50%, which is 10% higher than that of the SVM model, and the minimum discrimination accuracy of the millet varieties increased from 65 to 90%. The results show that the attention-CRNN model improved the overall identification accuracy of the eight millet varieties and greatly improved the minimum identification accuracy. The attention-CRNN model shows great potential for the nondestructive identification of millet and possibly other small grain varieties.

Sea ice as a disaster has recently attracted a great deal of attention in China. Its monitoring has become a routine task for the maritime sector. Remote sensing, which depends mainly on SAR and optical sensors, has become the primary means for sea-ice research. Optical images contain abundant sea-ice multi-spectral information, whereas SAR images contain rich sea-ice texture information. If the characteristic advantages of SAR and optical images could be combined for sea-ice study, the ability of sea-ice monitoring would be improved. In this study, in accordance with the characteristics of sea-ice SAR and optical images, the transformation and fusion methods for these images were chosen. Also, a fusion method of optical and SAR images was proposed in order to improve sea-ice identification. Texture information can play an important role in sea-ice classification. Haar wavelet transformation was found to be suitable for the sea-ice SAR images, and the texture information of the sea-ice SAR image from Advanced Synthetic Aperture Radar (ASAR) loaded on ENVISAT was documented. The results of our studies showed that, the optical images in the hue-intensity-saturation (HIS) space could reflect the spectral characteristics of the sea-ice types more efficiently than in the red-green-blue (RGB) space, and the optical image from the China-Brazil Earth Resources Satellite (CBERS-02B) was transferred from the RGB space to the HIS space. The principal component analysis (PCA) method could potentially contain the maximum information of the sea-ice images by fusing the HIS and texture images. The fusion image was obtained by a PCA method, which included the advantages of both the sea-ice SAR image and the optical image. To validate the fusion method, three methods were used to evaluate the fused image, i.e., objective, subjective, and comprehensive evaluations. It was concluded that the fusion method proposed could improve the ability of image interpretation and sea-ice identification.

There are many image fusion methods that can be used to produce high-resolution multispectral images from a high-resolution panchromatic image and low-resolution multispectral images. Starting from the physical principle of image formation, this paper presents a comprehensive framework, the general image fusion (GIF) method, which makes it possible to categorize, compare, and evaluate the existing image fusion methods. Using the GIF method, it is shown that the pixel values of the high-resolution multispectral images are determined by the corresponding pixel values of the low-resolution panchromatic image, the approximation of the high-resolution panchromatic image at the low-resolution level. Many of the existing image fusion methods, including, but not limited to, intensity-hue-saturation, Brovey transform, principal component analysis, high-pass filtering, high-pass modulation, the a/spl grave/ trous algorithm-based wavelet transform, and multiresolution analysis-based intensity modulation (MRAIM), are evaluated and found to be particular cases of the GIF method. The performance of each image fusion method is theoretically analyzed based on how the corresponding low-resolution panchromatic image is computed and how the modulation coefficients are set. An experiment based on IKONOS images shows that there is consistency between the theoretical analysis and the experimental results and that the MRAIM method synthesizes the images closest to those the corresponding multisensors would observe at the high-resolution level.

Synthetic aperture radar (SAR) is an important remote sensing sensor whose application is becoming more and more extensive. Compared with traditional optical sensors, it is not easy to be disturbed by the external environment and has a strong penetration. Limited by its working principles, SAR images are not easily interpreted, and fusing SAR images with optical multispectral images is a good solution to improve the interpretability of SAR images. This paper presents a novel image fusion method based on non-subsampled shearlet transform and activity measure to fuse SAR images with multispectral images, whose aim is to improve the interpretation ability of SAR images easily obtained at any time, rather than producing a fused image containing more information, which is the pursuit of previous fusion methods. Three different sensors, together with different working frequencies, polarization modes and spatial resolution SAR datasets, are used to evaluate the proposed method. Both visual evaluation and statistical analysis are performed, the results show that satisfactory fusion results are achieved through the proposed method and the interpretation ability of SAR images is effectively improved compared with the previous methods.

Multi-modality image fusion applied to improve image quality has drawn great attention from researchers in recent years. However, noise is actually generated in images captured by different types of imaging sensors, which can seriously affect the performance of multi-modality image fusion. As the fundamental method of noisy image fusion, source images are denoised first, and then the denoised images are fused. However, image denoising can decrease the sharpness of source images to affect the fusion performance. Additionally, denoising and fusion are processed in separate processing modes, which causes an increase in computation cost. To fuse noisy multi-modality image pairs accurately and efficiently, a multi-modality image simultaneous fusion and denoising method is proposed. In the proposed method, noisy source images are decomposed into cartoon and texture components. Cartoon-texture decomposition not only decomposes source images into detail and structure components for different image fusion schemes, but also isolates image noise from texture components. A Gaussian scale mixture (GSM) based sparse representation model is presented for the denoising and fusion of texture components. A spatial domain fusion rule is applied to cartoon components. The comparative experimental results confirm the proposed simultaneous image denoising and fusion method is superior to the state-of-the-art methods in terms of visual and quantitative evaluations.

In hyperspectral remote sensing, achieving high spatial resolution holds paramount importance for an array of applications, such as environmental monitoring, geographic mapping, and precision agriculture. Nevertheless, conventional hyperspectral images frequently grapple with the issue of restricted spatial resolution. We apply optimized inversion methods to hyperspectral image fusion and present an innovative approach for hyperspectral image fusion which combines the Hue–Intensity–Saturation (HIS) transform, the wavelet transform, and the Trust-Region Conjugate Gradient technique. This amalgamation not only refines spatial precision but also augments spectral faithfulness, which is a pivotal aspect for applications like precise object detection and classification. In the context of our investigation, we conducted a thorough validation of our proposed HIS, Wavelet, and Trust-Region Conjugate Gradient (TRCG-HW) method for image fusion using a comprehensive suite of evaluation metrics. These metrics encompassed the Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Correlation Coefficient (CC), Spectral Angle Mapper (SAM), and Error Relative Global Accuracy Score (ERGAS). The findings incontrovertibly establish TRCG-HW as the preeminent method among those considered. Our study effectively tackles the pressing predicament of low spatial resolution encountered in hyperspectral imaging. This innovative paradigm harbors the potential to revolutionize high-resolution hyperspectral data acquisition, propelling the field of hyperspectral remote sensing forward and efficiently catering to crucial application.

A new image fusion method based on hue-saturation-value (HSV) color space is proposed, the improved HSV fuses multi-spectral (MS) and panchromatic (PAN) images to improve spatial information and preserve spectral characteristics. The main advantage of the new fusion method is a simple and efficient way which can maximize the extraction of the spatial information and eliminate the disturbance of spectral information in PAN image. the difference of the low components information between PAN and the value component is filtered from the PAN image, and then value component are replaced by the new PAN image, the new HSV color space perform reverse transform to obtain a multispectral image with the high spatial resolution.SPOT-5 and QuickBird MS and PAN images were employed to execute the existing HSV and improved HSV fusion methods. Qualitative and quantitative analyses and classification accuracy assessment were conducted to evaluate the performance of the fusion methods. The results demonstrate that the improved HSV is better than traditional HSV methods. The new fusion method can achieve a wide range of balance between high spatial resolution retention and spectral characteristic preservation.

Deep learning methods for medical image fusion: A review

Image fusion at pixel level without precise registration always causes pseudo colors and other problem. Classification-based fusion scheme can effectively eliminate the false color at the edge of objective. However, the traditional per-pixel classification results in the well-known salt and pepper effect. The only way to smooth the image is to use filters, while impacted on the result of fusion. This paper proposes a method consist of a sequential application of segmentation, classification and fusion techniques. First, the image was multi-resolutely segmented into homogenous areas, and classified it by using the membership functions classifier and additional empirical rules. Subsequently, according to the restriction of the precise classification result, adjusting the multi-spectral image then achieved the fusion by using HSV color transformation. Finally, after compared the statistical properties of the fusion result by different methods, the proposed method showed satisfied result. I. INTRODUCTION With the appearance of plentiful kinds of remote sensing imagery, fusion of satellite images of different spatial resolutions play an important role. Via image fusion, we can obtain more information than can be derived from single kind of image. Pohl prompted that according to the stage which image fusion is performed, there are pixel, feature and decision level. Image fusion at pixel level always was effected by the accuracy of registration, the number of mixed pixel and other factors. The pixel in high-resolution satellite image is smaller sizes and combined with fewer spectral bands that cause greater spectral variation within a class and a greater degree of shadow. Hence, color break and pseudo colors often occur at the edge of object and shadow which fusing the high-resolution image only at pixel levels. During the high-resolution image fusion, for eliminating color break and pseudo colors, using its classification image as the prior knowledge and restricting the fusion area can take full advantage of spectral information from multi-spectral bands and texture information come from pan band to realize the image fusion. It's a method of image fusion that performing at pixel level and integrate with feature level fusion. In this way, phenomenon of pseudo colors can be avoided. In this paper, we propose an image fusion method based on object-oriented classification. Via image segmentation, feature extraction and image fusion based on object-oriented classification, the fusion image can be achieved. In this method, object-oriented classification more appropriate for high-resolution image to get accuracy classification result and overcoming the limit of per-pixel image analyses. Because much information is acquired in the relationship between adjacent pixels, including texture and shape information, which allows for identification of individual objects as oppose to single pixels

Improved Fusion Method for Remote Sensing Imagery Using NSCT and Extend IHS

A novel multi-modality image fusion method based on image decomposition and sparse representation

Hybrid pixel based method for multimodal image fusion based on Integration of Pulse Coupled Neural Network (PCNN) and Genetic Algorithm (GA) using Empirical Mode Decomposition (EMD)

Hyperspectral image fusion is a key technique of hyperspectral data processing. In recent years, many fusion methods have been proposed, but there is little work concerning evaluation of the performances of different image fusion methods. In this paper, a method called quantitative correlation analysis (QCA) is proposed, which provides a quantitative measure of the information transferred by an image fusion technique into the output image. Using the proposed method, the performances of different image fusion methods can be compared and analyzed directly based on the images of before and after performing the fusion. The correlation information entropy, based on the developed QCA, is also proposed and testified by numerical simulations. Typical hyperspectral data are applied to the proposed method. The results show that the method is effective, and its conclusions agree with the classification results in applications.

Image fusion method is an algorithm that synthesizes images from different sources into a new image. The advantage of image fusion technology is that different image information presented by multi-source sensors can complement and improve each other to extract the effective information in the source image. It can also obtain a more effective and richer image with description information. According to the technical methods of image fusion, an improved image fusion method for fast non-downsampled contourlet transform (NSCT) was selected for handheld. The source infrared image and source low-light image collected by the observer were image fusion. In this method, the scale filter and the directional filter were convolved respectively in the composition of image decomposition and reconstruction to reduce the number of iterations of image decomposition and reconstruction, improve the efficiency of algorithm operation, and ensure that the system worked in real time.

Image fusion is to get a fused image that contains all important information from source images of the same scene. Meanwhile, multi-scale transforms and sparse representation (SR) are the two most effective techniques for image fusion. However, the SR-based image fusion methods are time-consuming and do not take the structural information of the source images into consideration. In addition, different multi-scale transform-based methods have their inevitable defects waiting to be solved till now. Therefore, in this paper, a new image fusion method combining nonsubsampled contourlet transform (NSCT) with SR is proposed. A decision map for the low-frequency coefficients according to the high-frequency coefficients is made to overcome these problems. Furthermore, it can reduce the calculation cost of the fusion algorithm and retain the useful information of source images as far as possible. Comparing with conventional multi-scale transform based methods and sparse representation based methods with a fixed or learned dictionary, the proposed method has better fusion performance in the field of medical image fusion.