Infrared Image Super-Resolution by Using Sparse Dictionary and Nonsubsampled Contourlet Transform

This paper presents a novel infrared (IR) and visible images fusion methodology based on non-subsampled contourlet transform (NSCT). NSCT shows better performance compared with usual multi-scale decomposition for its multi-scale, shift invariance, multi-direction and efficient capture of geometric structures. The proposed fusion method uses NSCT for multiresolution decomposition of the source images. The low-pass NSCT adaptive fusion weights calculated from the IR source image’s pixel statistical characteristics. The high frequency directional coefficients with max absolute value are the coefficients of the fusion NSCT high frequency. Experimental results conforms that the proposed method have better performance compared with DWT, compressed sensing based on DWT (CS-DWT), NSCT, NSCT based on spatial frequency motivated pulse coupled neural networks (SF-PCNN-NSCT) from visual effects and a list of fusion quality evaluation metrics.

Recent studies have shown that Deep Learning (DL) algorithms can significantly improve Super Resolution (SR) performance. Single image SR is useful in producing High Resolution (HR) images from their Low Resolution (LR) counterparts. The motivation for SR is the potential to assist algorithms such as object detection, localization, and classification. Insufficient work has been conducted using Generative Adversarial Networks (GANs) for SR on infrared (IR) images despite its promising ability to increase object detection accuracy by extracting more precise features from a given image. This work adopts the idea of a relativistic GAN that utilizes Residual in Residual Dense blocks (RRDBs) for feature ex- traction, a novel residual image addition, and a Pixel Transposed Convolutional Layer (PixelTCL) for up-sampling. Recent work has validated the use of GANs for Visible Light (VL) images, making them a strong candidate. The inclusion of these components produce more realistic and natural features while also receiving superior metric values.

In this paper, an unsupervised change detection method for multitemporal remote sensing images is proposed. Firstly, the difference image is obtained from two multitemporal images acquired on the same geographical area but at different time instances. Then the difference image is decomposed by nonsubsampled contour let transform (NSCT). For each pixel in the difference image, a feature vector is extracted using the NSCT coefficients and the difference image itself which are in the same position. The final change map is achieved by clustering the feature vectors using kernel fuzzy c-means (KFCM) clustering algorithm into two classes: changed and unchanged. The change detection results are compared with those of several state-of-the-art methods. And the experimental results demonstrate that the proposed method yields superior performance.

The practice of integrating images from two or more sensors collected from the same area or object is known as image fusion. The goal is to extract more spatial and spectral information from the resulting fused image than from the component images. The images must be fused to improve the spatial and spectral quality of both panchromatic and multispectral images. This study provides a novel picture fusion technique that employs L0 smoothening Filter, Non-subsampled Contour let Transform (NSCT) and Sparse Representation (SR) followed by the Max absolute rule (MAR). The fusion approach is as follows: first, the multispectral and panchromatic images are divided into lower and higher frequency components using the L0 smoothing filter. Then comes the fusion process, which uses an approach that combines NSCT and SR to fuse low frequency components. Similarly, the Max-absolute fusion rule is used to merge high frequency components. Finally, the final image is obtained through the disintegration of fused low and high frequency data. In terms of correlation coefficient, Entropy, spatial frequency, and fusion mutual information, our method outperforms other methods in terms of image quality enhancement and visual evaluation.

For Infrared Image Super-Resolution (IRSR) technology, maintaining performance while reducing model complexity is critical for a wide range of applications. However, existing research on lightweight IRSR has been predominantly limited to modifying model architectures. This study proposes a new methodology that applies Knowledge Distillation, a representative model compression technique from supervised learning, to IRSR models. To this end, we extend the DCKD framework, previously used for RGB image super-resolution, to the IRSR domain and introduce new loss functions designed to maximize the preservation of key structural characteristics in infrared images, namely edge and spectral(Contourlet-domain) information. Through the proposed methodology, a lightweight student model trained with distilled knowledge from a high-performance, complex teacher model consistently achieved superior performance compared to the same architecture trained via standard supervised learning. This study demonstrates that Knowledge Distillation based methodology is effective for developing lightweight IRSR models and is expected to contribute to fields where high-efficiency IRSR is essential, such as real-time military surveillance, disaster response, and nocturnal reconnaissance.

Visible face recognition systems, being vulnerable to illumination, expression, and pose, can not achieve robust performance in unconstrained situations. Meanwhile, near infrared face images, being light- independent, can avoid or limit the drawbacks of face recognition in visible light, but its main challenges are low resolution and signal noise ratio (SNR). Therefore, near infrared and visible fusion face recognition has become an important direction in the field of unconstrained face recognition research. In this paper, a novel fusion algorithm in non-subsampled contourlet transform (NSCT) domain is proposed for Infrared and visible face fusion recognition. Firstly, NSCT is used respectively to process the infrared and visible face images, which exploits the image information at multiple scales, orientations, and frequency bands. Then, to exploit the effective discriminant feature and balance the power of high-low frequency band of NSCT coefficients, the local Gabor binary pattern (LGBP) and Local Binary Pattern (LBP) are applied respectively in different frequency parts to obtain the robust representation of infrared and visible face images. Finally, the score-level fusion is used to fuse the all the features for final classification. The visible and near infrared face recognition is tested on HITSZ Lab2 visible and near infrared face database. Experiments results show that the proposed method extracts the complementary features of near-infrared and visible-light images and improves the robustness of unconstrained face recognition.

Based on nonsubsampled contourlet transform (NSCT) and modified pulse coupled neural network (MPCNN), the paper proposes an effective method of image fusion. Firstly, the paper decomposes the source image into the low-frequency components and high-frequency components using NSCT, and then processes the low-frequency components by regional statistical fusion rules. For high-frequency components, the paper calculates the spatial frequency (SF), which is input into MPCNN model to get relevant coefficients according to the fire-mapping image of MPCNN. At last, the paper restructures the final image by inverse transformation of low-frequency and high-frequency components. Compared with the wavelet transformation (WT) and the traditional NSCT algorithm, experimental results indicate that the method proposed in this paper achieves an improvement both in human visual perception and objective evaluation. It indicates that the method is effective, practical and good performance.

Infrared and visible image fusion can obtain an integrated image containing obvious object information and high spatial resolution background information. Therefore, combining the characteristics of infrared and visible images to obtain the fused image has important research significance. In this paper, an effective fusion algorithm based on non-subsampled contourlet transform (NSCT) is proposed. The method is based on the application of a modulated pulse-coupled neural network fusion (PCNN) strategy and an energy attribute fusion strategy in the NSCT domain. First, NSCT is used to decompose the input original image into low frequency sub-images and high frequency sub-images. Then, the high frequency sub-images are fused via a multi-level morphological gradient (MLMG) domain PCNN and the low frequency sub-images are fused via the energy attribute fusion strategy. Finally, the fused sub-images are reconstructed by inverse NSCT. Experimental results demonstrate that the proposed algorithm has a better fusion performance in both subjective evaluation and objective evaluation.

A novel fusion algorithm for spatially registered infrared and visible images fusion based on the spiking cortical model (SCM) in nonsubsampled contourlet transform (NSCT) domain is proposed in this paper. NSCT can effectively surmount the defect of the absence of shift-invariance in contourlet transform to avoid pseudo-Gibbs phenomena. Source images are decomposed by NSCT to acquire the coefficients of lowpass subbands and highpass subbands. In the proposed fusion method, the energy of Laplacian that computed by the lowpass subband, which stands for the edge features of the lowpass subband, is employed and inputted to motivate SCM. For the highpass subbands, a modified spatial frequency that calculated through the highpass subband is introduced and applied as the gradient features of the highpass subbands to motivate SCM. Experiments demonstrate the proposed algorithm can make great progress in image fusion, and achieve better effect than conventional methods in both objective evaluation and visual appearance.

Infrared images hold significant application value in fields such as military reconnaissance, security surveillance, and medical diagnosis. However, issues like low resolution, high noise, and complex degradation characteristics severely hinder their practical application effectiveness. This paper introduces an infrared super-resolution reconstruction algorithm based on a random degradation model and Generative Adversarial Networks (GANs), addressing the diversity of infrared image degradation processes. The primary contribution lies in explicitly modeling key degradation parameters of infrared images (such as blur kernel and noise distribution) using a random degradation model, generating diverse low-resolution to high-resolution image pairs, and significantly enhancing the model’s generalization ability for complex degradations. Experiments on the Airo infrared dataset and a self-built infrared dataset demonstrate that when the resolution is increased by a factor of 2, 4, and 8, the 4× resolution reconstructed images exhibit notable advantages in terms of texture detail and noise suppression. Especially in 4× super-resolution reconstruction, compared to three typical deep learning algorithms, our algorithm improves the Peak Signal-to-Noise Ratio (PSNR) by 3.046 dB, 1.8489 dB, and 0.2108 dB, respectively, and the Structural Similarity Index (SSIM) by 0.0387 (4.76%), 0.0287 (3.48%), and 0.0131 (1.56%), respectively, with perceptual similarity decreasing by 0.2465, 0.13344, and 0.0514 (lower values indicate better perceptual quality). Subjective visual assessments further validate the algorithm’s significant advantages in noise reduction and weak texture restoration. This study proposes an infrared image super-resolution reconstruction method based on random degradation modeling, which holds significant theoretical and practical value in complex infrared degradation scenarios.

In this paper, a novel maximum a posteriori (MAP) super resolution (SR) algorithm is proposed. This algorithm estimates the registration parameters, the regularization parameter and the high resolution (HR) image simultaneously. The hyperparameters in the prior distributions of image and noise are regarded as random values, of which the ratio is the regularization parameter. By modeling the image, noise and hyperparameters correctly, the cost function is convex and each parameter has unique stable solution. In order to enhance the real time of SR algorithm, a fast block matching registration algorithm is proposed. The registration algorithm not only yields a dense motion field but also makes full use of the prior information in all low resolution (LR) images. Synthetic and real experimental results demonstrate the effectiveness of the algorithm as well as its superiority over conventional SR methods.

Multifocus image fusion using the nonsubsampled contourlet transform

Due to the obvious advantages, the technology of medical imaging has been widely utilized in the medical areas. Commonly, the medical images of different imaging mechanism are able to guide the radiologist for the precise diagnosis of disease and for more effective interventional treatment procedures. Consequently, the fusion of different medical images of the same scene is necessary. This paper proposes a novel fusion technique for medical images based on non-subsampled contourlet transform (NSCT). Due to the better competence of image information capturing, NSCT is utilized to conduct the multi-scale and multi-directional decompositions of source images. In addition, the models of both region average energy and region coefficient deviation are used for the fusion for low-frequency sub-images and high-frequency ones, respectively. In comparison with several current typical fusion techniques, the proposed one has remarked superiorities in terms of both subjective and objective evaluations.

Medical image fusion based on convolutional neural networks and non-subsampled contourlet transform

Infrared imaging has the advantage of all-weather working ability. Due to the limitation of the hardware and the high cost, the resolution of infrared image (IR) is very low. To improve the resolution of IR images, this paper exploits super-resolution (SR) method for IR images. A new SR framework by using random forests is proposed in this paper. Existing methods adopts single regression model for SR. However, which single regression model tends to overfit training data, and would lead to a poor performance. Furthermore, the existing methods are not suitable for real-time system due to the heavy time consuming. To resolve this problem, an ensemble regression model, i.e. random forests rather than single regression model is adopted in this paper. In addition, to achieve better results multi-regression models rather than a single regression model are trained on the clustered training data. Moreover, the features used in many SR methods cannot extract features on diagonal orientation. To resolve this problem, we adopt a second order derivative filter, which can extract features on diagonal orientation. The experimental results demonstrate the availability of the proposed method.