A Geometric Dictionary Learning Based Approach for Fluorescence Spectroscopy Image Fusion

Multi-focus image fusion is an effective way to overcome the shallow depth of field of ordinary cameras and to achieve all-in-focus imaging across the entire depth range. However, existing multi-focus image fusion methods are usually designed for single-view and static scenes, mainly because the current focal stack imaging technology for capturing multi-focus images can only capture one partially focused image at each moment. To simultaneously capture multi-focus images for all-in-focus imaging in dynamic scenes, we propose a computational imaging scheme for all-in-focus dynamic image fusion. Specifically, a multi-view multi-focus (MVMF) imaging technology that integrates the characteristics of both multi-view imaging and focal stack imaging is designed to simultaneously capture MVMF images. For the fusion of MVMF images, we design an MVMF image fusion method based on focus clarity evaluation. This method consists of three modules: image alignment, focus clarity evaluation, and image fusion, which can effectively fuse MVMF images into a high-quality all-in-focus image. In addition, to fill the gap of lacking dynamic multi-focus image datasets in the existing multi-focus image fusion field, we render an MVMF image dataset (i.e., DYN-MVMF) capturing dynamic scenes using the Blender software. On the DYN-MVMF dataset, we conduct extensive comparative experiments with state-of-the-art multi-focus image fusion methods. The experimental results validate the effectiveness of the proposed method in fusing MVMF images into a high-quality all-in-focus image. Moreover, we also conduct comparative experiments with the conventional focal stack imaging scheme on dynamic scene imaging and salient object detection tasks. The results demonstrate that our proposed computational imaging scheme can achieve all-in-focus imaging for each frame of dynamic scenes, thereby enabling more accurate salient object detection results. Therefore, the proposed scheme is expected to contribute to the development of high-performance imaging systems.

Multi-scale convolutional neural network for multi-focus image fusion

This paper proposes a novel multi-focus image capture and fusion system for macro photography. The system consists of three components. The first component is a novel multi-focus image capture device which can capture multiple macro images taken at different focus distances from a photographic subject, with high precision. The second component is a feature based method which can align multiple in-focus images automatically. The third component is a new multi-focus image fusion method which can combine multiple macro images to a fused image with a greater depth of field. The proposed image fusion method is based on Gaussian and Laplacian pyramids with a novel weight map selection strategy. Several data sets are captured and fused by the proposed system to verify the hardware and software design. Subjective and objective methods are also used to evaluate the proposed system. By analyzing the experimental results, it shows that this system is flexible and efficient, and the quality of the fused image is comparable to the results of other methods.

A novel dictionary learning approach for multi-modality medical image fusion

The multi-focus image fusion technique is to extract the focus regions from source images and compose them together to form a clear image in the full field of view. In order to further improve the accuracy of focus region detection and ensure its efficiency, a novel multi-focus image fusion method in spatial domain, based on guided filter and mixed focus measure, is proposed in this paper. Firstly, a guided filter is employed as an edge-preserving smoothing operator to process the source images, and the difference operator is used between the filtered images and the source images to extract salient feature. Subsequently, the salient feature maps are measured by the mixed focus measure, combining the sum of energy of edge (SEOE) and the sum of local variance (SLV), to detect the focus regions, and the initial decision map is obtained. For holes of different sizes in the initial decision map, the closing operation and the small area removal strategy are used to fill and connect the truncated regions, and then the opening operation and the guide filter are used to optimize the decision map boundary to obtain the final decision map. Finally, the multi-focus fusion image is obtained by the pixel-wise weighted-averaging rule according to the final decision map. Simulation results demonstrate that the method is superior to some existing fusion methods on both subjective visual perception and objective evaluation metrics.

Sparse representation (SR) has been widely used in many image processing applications including image fusion. As the contents vary significantly across different images, a highly redundant dictionary is always required in the sparse model, which reduces the algorithm stability and efficiency. This paper proposes a multi-focus image fusion method based on SR with adaptive sparse domain selection (SR-ASDS). Under SR-ASDS, numerous high-quality image patches are first classified into several categories according to their gradient information, and each category is applied into training a compact sub-dictionary. At the fusion process, a corresponding sub-dictionary is adaptively selected for a given pair of source image patches. Moreover, we present a general optimization framework for the merging rule design of the SR based image fusion. Numerous experiments on both clear images and the noisy ones demonstrate that the proposed method outperforms the fusion methods which use a single dictionary, in terms of several popular objective evaluation criteria.

Focus is limited and singular in many image capture devices. Therefore, different focused objects at different distances are obtained in a single image taken. Image fusion can be defined as the acquisition of multiple focused objects in a single image by combining important information from two or more images into a single image. In this paper, a new multi-focus image fusion method based on Bat Algorithm (BA) is presented in a Multi-Scale Transform (MST) to overcome limitations of standard MST Transform. Firstly, a specific MST (Laplacian Pyramid or Curvelet Transform) is performed on the two source images to obtain their low-pass and high-pass bands. Secondly, optimization algorithms were used to find out optimal weights for coefficients in low-pass bands to improve the accuracy of the fusion image and finally the fused multi-focus image is reconstructed by the inverse MST. The experimental results are compared with different methods using reference and non-reference evaluation metrics to evaluate the performance of image fusion methods.

Improved multi-focus image fusion using online convolutional sparse coding based on sample-dependent dictionary

Multi-modality image fusion applied to improve image quality has drawn great attention from researchers in recent years. However, noise is actually generated in images captured by different types of imaging sensors, which can seriously affect the performance of multi-modality image fusion. As the fundamental method of noisy image fusion, source images are denoised first, and then the denoised images are fused. However, image denoising can decrease the sharpness of source images to affect the fusion performance. Additionally, denoising and fusion are processed in separate processing modes, which causes an increase in computation cost. To fuse noisy multi-modality image pairs accurately and efficiently, a multi-modality image simultaneous fusion and denoising method is proposed. In the proposed method, noisy source images are decomposed into cartoon and texture components. Cartoon-texture decomposition not only decomposes source images into detail and structure components for different image fusion schemes, but also isolates image noise from texture components. A Gaussian scale mixture (GSM) based sparse representation model is presented for the denoising and fusion of texture components. A spatial domain fusion rule is applied to cartoon components. The comparative experimental results confirm the proposed simultaneous image denoising and fusion method is superior to the state-of-the-art methods in terms of visual and quantitative evaluations.

In this paper, we introduce a novel unsupervised deep learning (DL) method for multi-focus image fusion. Existing multi-focus image fusion (MFIF) methods based on DL treat MFIF as a classification problem with a massive amount of reference images to train networks. Instead, we proposed an end-to-end unsupervised DL model to fuse multi-focus color images without reference ground truth images. As compared to conventional CNN our proposed model only consists of convolutional layers to achieve a promising performance. In our proposed network, all layers in the feature extraction networks are connected to each other in a feed-forward way and aim to extract more useful common low-level features from multi-focus image pair. Instead of using conventional loss functions our model utilizes image structure similarity (SSIM) to calculate loss in the reconstruction process. Our proposed model can process variable size images during testing and validation. Experimental results on various test images validate that our proposed method achieves state-of-the-art performance in both subjective and objective evaluation metrics.

Sparse representation has been widely applied to multi-focus image fusion in recent years. As a key step, the construction of an informative dictionary directly decides the performance of sparsity-based image fusion. To obtain sufficient bases for dictionary learning, different geometric information of source images is extracted and analysed. The classified image bases are used to build corresponding subdictionaries by principle component analysis. All built subdictionaries are merged into one informative dictionary. Based on constructed dictionary, compressive sampling matched pursuit algorithm is used to extract corresponding sparse coefficients for the representation of source images. The obtained sparse coefficients are fused by Max-L1 fusion rule first, and then inverted to form the final fused image. Multiple comparative experiments demonstrate that the proposed method is competitive with other the state-of-the-art fusion methods.

Visual attention guided image fusion with sparse representation

Multi-focus image fusion is an important method used to combine the focused parts from source multi-focus images into a single full-focus image. Currently, to address the problem of multi-focus image fusion, the key is on how to accurately detect the focus regions, especially when the source images captured by cameras produce anisotropic blur and unregistration. This paper proposes a new multi-focus image fusion method based on the multi-scale decomposition of complementary information. Firstly, this method uses two groups of large-scale and small-scale decomposition schemes that are structurally complementary, to perform two-scale double-layer singular value decomposition of the image separately and obtain low-frequency and high-frequency components. Then, the low-frequency components are fused by a rule that integrates image local energy with edge energy. The high-frequency components are fused by the parameter-adaptive pulse-coupled neural network model (PA-PCNN), and according to the feature information contained in each decomposition layer of the high-frequency components, different detailed features are selected as the external stimulus input of the PA-PCNN. Finally, according to the two-scale decomposition of the source image that is structure complementary, and the fusion of high and low frequency components, two initial decision maps with complementary information are obtained. By refining the initial decision graph, the final fusion decision map is obtained to complete the image fusion. In addition, the proposed method is compared with 10 state-of-the-art approaches to verify its effectiveness. The experimental results show that the proposed method can more accurately distinguish the focused and non-focused areas in the case of image pre-registration and unregistration, and the subjective and objective evaluation indicators are slightly better than those of the existing methods.

Utilizing the contourlet’s advantages of multiscale, localization, directionality and anisotropy, a New method for multi-focus image fusion using wavelet based contourlet transform and region was proposed. In the contourlet transform, the Laplacian pyramid transform was replaced by the wavelet transform to obtain non-redundant wavelet-based contourlet transform. The source image is decomposed into low-frequency image and high-frequency images by wavelet based contourlet.. The fusion rule of Regional variance is adopted in low-frequency image, and the fusion rule of local energy are adopted in high-frequents subbands. Finally the fused image is obtained through inverse transform. Experimental results show that the proposed method has better subjective visual effect. Compared to traditional gradient pyramid algorithm and wavelet fusion algorithm, the proposed algorithm in paper has smallest mean square error.

Based on the PCNN model and contrast modulation method, a new multi-focus image fusion method is proposed in this paper. Send source images into the PCNN and compute the contrast. The characteristic of image region clustering enhances the veracity of contrast. Then using the normalization contrast modulation gets two fusion images. Finally, use local variance to get the new fusion image. The experiment indicates that the fusion image contains more information about the edge, texture and detail, and it has a better contrast. Compared with the common methods, the innovative method embodies better fusion performance in information, standard and average grads.