key: cord-0883407-guum9x7a
authors: Liu, Shuaiqi; Wang, Mingwang; Yin, Lu; Sun, Xiuming; Zhang, Yu-Dong; Zhao, Jie
title: Two-Scale Multimodal Medical Image Fusion Based on Structure Preservation
date: 2022-01-31
journal: Front Comput Neurosci
DOI: 10.3389/fncom.2021.803724
sha: dde2d0e291252ba0dc260b0defc996aeeb42a634
doc_id: 883407
cord_uid: guum9x7a

Medical image fusion has an indispensable value in the medical field. Taking advantage of structure-preserving filter and deep learning, a structure preservation-based two-scale multimodal medical image fusion algorithm is proposed. First, we used a two-scale decomposition method to decompose source images into base layer components and detail layer components. Second, we adopted a fusion method based on the iterative joint bilateral filter to fuse the base layer components. Third, a convolutional neural network and local similarity of images are used to fuse the components of the detail layer. At the last, the final fused result is got by using two-scale image reconstruction. The contrast experiments display that our algorithm has better fusion results than the state-of-the-art medical image fusion algorithms.

Medical imaging technology has developed rapidly, and medical images of various modalities have significant applications in clinical diagnosis and disease analysis (Wang S. et al., 2020; Li et al., 2021) . There are some common medical imaging technologies, such as CT, MRI, PET, and singlephoton emission computed tomography (SPECT). Due to different imaging technologies, various kinds of medical images contain complementary characteristics. For the same organ or the same tissue of the human body, medical images with different imaging technologies reflect various pathological characteristics. CT can well show dense structures, such as bones, while MRI has a higher spatial resolution for soft tissues. PET and SPECT are more conducive to display functional information of different tissues of the human body. However, limited by the equipment, the quality of medical images is often unsatisfying. Two or more medical images are merged into one image with comprehensive features by fusion methods for the sake of improving the accuracy of clinical diagnosis (Li et al., 2017; Amala Rani and Lalithakumari, 2019; Huang et al., 2020) . Medical image fusion aims at extracting more useful information of input images, improving the application of medical images, and helping doctors understand image contents better. Therefore, the study of medical image fusion is significant (Tirupal et al., 2020) .

The existing image fusion methods include pixel-level image fusion methods, feature-level image fusion methods, and decision-level image fusion methods (Dolly and Nisa, 2019; Zhao et al., 2021) . The pixel-level fusion methods fuse multiple registered images and preserve the maximum amount of information. Feature-level fusion algorithms extract different features from input images, such as textures and details, and then merge all the features into one image. Feature-level image fusion has lower requirements for image registration so it will lose part of source images' detailed information. Decision-level fusion is performed by the high-level abstracted information, so the detailed information will be lost severely (Kim et al., 2016; . In conclusion, pixel-level fusion algorithms are the simplest methods with the lowest computation cost. Therefore, we propose a pixel-level medical image fusion algorithm.

In general, pixel-level image fusion methods contain spatial domain methods and transform domain methods (Yu and Chen, 2020) . In spatial domain, the fusion operation is performed on pixels (Ashwanth and Swamy, 2020; Zhu et al., 2020) . Spatial domain fusion algorithms include principal component analysis (PCA)-based image fusion method (Benjamin and Jayasree, 2018) , guided filtering (GFF)-based image fusion method (Li et al., 2013) , improved sum-modified-Laplacian (ISML)-based image fusion algorithm (Liu et al., 2015a) , nuclear norm minimization (NNM)-based image fusion algorithm (Liu et al., 2015b) , convolutional neural network (CNN)-based image fusion algorithm (Liu et al., 2017b) , and so on. In transform domain image fusion algorithms, source images are first decomposed into low-and high-frequency components through multi-scale transformations. And then appropriate fusion rules are used to fuse different components. Finally, the final fused result can be obtained through inverse transformation. At present, scholars around the world have proposed many image fusion methods in the transform domain (Mao et al., 2020) , such as the waveletbased image fusion methods proposed in Sumir and Gahan (2018) , Panigrahy et al. (2020b) , and Rahmani et al. (2020) , the image fusion algorithms using non-subsampled contourlet transform (NSCT) proposed in Das and Kundu (2012) , Zhu et al. (2019) , , and Panigrahy et al. (2020a) , and the image fusion algorithms using non-subsampled shearlet transform (NSST) (Liu et al., 2017a; Ganasala and Prasad, 2020; Panigrahy et al., 2020c) . These algorithms have achieved good fusion effects. However, traditional transform domain image fusion algorithms need to perform frequency decomposition and synthesis during fusion, which often have problems, such as fused image distortion and structural information loss of the source images (Zeng et al., 2020) . However, in spatial domain, image fusion often needs to block the images during fusion, which often produces serious blocking effects. To maintain the structural and detailed information, Zhu et al. (2018) proposed an image fusion method based on image cartoon-texture decomposition and sparse representation. In recent years, many structurepreserving filters have appeared (Tomasi and Manduchi, 1998; Farbman et al., 2008; Xu et al., 2011; He et al., 2019) . These filters can well maintain the structural information in the images while smoothing textures, which is beneficial to restore the structures and neighborhood details of source images. Therefore, structure-preserving filters are increasingly used in image fusion. The proposed algorithm also applies a structure-preserving filter to overcome block effects and maintain structural information. Recently, deep learning-based fusion algorithms became a hotspot (Tomasi and Manduchi, 1998; Zhu et al., 2018) . A multifocus image fusion method by using CNN is proposed in Farbman et al. (2008) , which regards the generation process of the information feature focus map as a classification problem, and the fusion rule can be regarded as the classifiers used in general classification tasks. proposed a residual network-based multi-focus image fusion method. Subsequently, deep learning has also been applied in medical image fusion. Liu et al. (2015b) also proposed a CNN-based medical image fusion algorithm, which achieved satisfying fusion results. These algorithms can effectively integrate the design of fusion rules and the generation of a decision map, greatly simplifying the steps of image fusion. However, for the sake of obtaining better image fusion results, deep learning-based medical image fusion algorithms need to train a great number of samples. In the training of the network, the training data sets should have ground-true images, but it is not easy to obtain ground-true images of medical images.

Recently, deep learning-based fusion algorithms became a hotspot (Wang et al., 2021a,b) . A multi-focus image fusion method by using CNN is proposed in Liu et al. (2017c) , which regards the generation process of the information feature focus map as a classification problem, and the fusion rule can be regarded as the classifiers used in general classification tasks. proposed a residual network-based multifocus image fusion method. Subsequently, deep learning has also been applied in medical image fusion. Liu et al. (2017b) proposed a CNN-based medical image fusion algorithm, which achieved satisfying fusion results. Wang K. et al. (2020) proposed a CNN-based multi-modality medical image fusion algorithm. This algorithm uses the Siamese convolutional network to generate the weight map. The source images are decomposed by contrast pyramid and then fused based on the weight map. These algorithms can effectively integrate the design of fusion rules and the generation of a decision map, greatly simplifying the steps of image fusion. However, for the sake of obtaining better image fusion results, deep learning-based medical image fusion algorithms need to train a great number of samples. In the training of the network, the training data sets should have ground-true images, but it is not easy to obtain ground-true images of medical images.

For the sake of solving the above-mentioned problems of image fusion algorithms, we proposed a two-scale medical image fusion algorithm based on structure preservation. Aiming at different imaging modes of multimodal medical images, a twoscale decomposition operation is adopted to decompose source images into base layer components and detail layer components. In this algorithm, we used different strategies to fuse different components, making the fusion process more comfortable for the human visual system. In the fusion of the base layer components, an iterative joint bilateral filter is applied to improve the decision map, and the weighted sum strategy is used to fuse the base layer components in the spatial domain. In the proposed algorithm, the CNN is adopted to fuse components of the detail layer to produce an image fusion weight map. The components of the detail layer are then fused by employing a multi-scale method and a fusion strategy based on local similarity. In the proposed algorithm, we made full use of the structure-preserving property of the iterative bilateral filter. In addition, the application of CNN also improves the performance of medical image fusion. Compared with previous algorithms, the main contribution of our method consists of three points.

(1) We use a two-scale decomposition method to decompose source images into base layer components and detail layer components, which can make full use of scale information.

(2) For the sake of retaining more information, a fusion strategy based on an iterative joint bilateral filter is adopted to fuse base layer components. (3) A new fusion rule based on CNN and local similarity was applied to the detail layer to maintain more detailed information.

The rest of this article is as follows. Section "Related Works" introduces the related works. Section "The Proposed Algorithm" describes the proposed fusion method and its implementation steps. Section "Experiments and Discussion" shows the experimental results and analysis of this algorithm. Finally, in Section "Conclusion" we conclude our fusion method.

For a given source image I, it can be decomposed into the base layer component I b and the detailed layer component I d . I b can be obtained through the following optimization problem:

where g x = −1 1 and g y = −1 1 T are horizontal and vertical gradient operators, respectively. The regularization parameter η is set to 5. The optimization problem is a Tikhonov regularization problem, and it can be resolved efficiently by fast Fourier transform (FFT). By subtracting the base layer 

A bilateral filter is non-linear and it can achieve edge preservation during denoising (Tomasi and Manduchi, 1998) . In bilateral filtering, the filtered pixel value of a point depends Frontiers in Computational Neuroscience | www.frontiersin.org on the neighborhood pixels. Furthermore, the weights of the neighborhood pixels are determined by the distance and similarity of the two pixels. The bilateral filter can be represented as

where I is the input image. p and q are the coordinates of the pixels. I p and I q denote the pixel values of the corresponding positions. is a sliding window centered on p. J p is the output. σ s and σ r control the weight of spatial domain and range domain, respectively.

The weight of the bilateral filter is unstable in practical applications, so some artificial textures will appear near the edges. For the sake of improving the stability of weight, a joint bilateral filter (Zhan et al., 2019) is introduced. It can deal with the problem of the stability of weight by introducing a guide image as the basis for calculating the range weight.

In this article, the iterative joint bilateral filter is used to refine edges. In filtering operation, J 1 represents the initial input guide image and J t+1 denotes the output of the t-th iteration, that is

where k p is used to normalize,

where σ s is set to 8 and σ r is set to 0.2.

Convolutional neural network is widely used in behavior cognition, pose estimation, object recognition, neural network conversion, and natural language processing. It is one of the most commonly used models in deep learning. It is composed of input layer, hidden layer, and output layer. The hidden layer contains convolutional layer, pooling layer, and fully connected layer. There are three structural characteristics in the CNN model, which are local receptive field, weight sharing, and pooling. The local receptive field represents the connection between a single neuron and the neurons in the corresponding region of the previous layer. As the number of layers deepens, the corresponding receptive field of a single neuron in each layer increases in the original image, and the information expressed becomes more and more abstract. Weight sharing means that the weight of the convolution kernel of feature mapping is spatially invariant. The receptive field and weight sharing decrease the number of parameters for neural network training. Pooling is also called down-sampling, which can reduce data dimensions. The core idea of CNN is to combine these three structures to obtain the deep features of images. The calculation formula of the convolution kernel is

where M j represents the feature map from the input. k l ij represents the convolution kernel connected to the i-th feature map of the previous layer. * represents convolution operation. b l j represents the bias. f (·) represents the activation function.

In CNN-based image fusion algorithms, the generation of a decision map can usually be considered as a classification problem. Specifically, the decision map can be acquired through activity level measurement. Measuring the activity level of a source image is closely related to feature extraction. The higher the activity level means that the image block is clearer. Image fusion is similar to classification tasks. Therefore, the fusion rule in this process is equal to the classifiers in image classification. According to these ideas, a CNN-based multi-focus image fusion algorithm is proposed in Liu et al. (2017c) . The advantage of this algorithm is that it solves the difficulties of manually designing activity level measurement and the fusion rule. Therefore, the CNN-based image fusion algorithm solves the problem of fusion by designing the CNN model. By training a large number of image data to obtain the CNN model, it is more effective than manually designing the fusion rule. In Liu et al. (2017b) , combining with the CNN model, Liu et al. used a multi-scale method and a local similarity-based fusion rule to obtain a highquality fusion result. We applied this method to fuse detail components in our fusion algorithm.

Aiming at obtaining a satisfying medical image fusion result, we proposed a two-scale medical image fusion algorithm based on structure preservation. The framework of this fusion method is shown in Figure 1 . First, through the method in Section "Two-Scale Image Decomposition, " we decompose source images into For ease of description, we suppose that the two source images are I A and I B , and in this article, we have given the fusion steps of two images only. The process of multiple images fusion is similar to the fusion of two images. Traditional multi-scale decomposition methods need more than two scales to get a satisfactory fusion image. In the proposed algorithm, we decomposed the source images by using the two-scale decomposition method in Liu Y. et al. (2016) . Equations 1, 2 are employed to decompose source images I A and I B into base layer components I b A , I b B and detail layer components I d A , I d B . The following are descriptions of the fusion rules of base and detail layer components.

First, a local structure preservation filter is used to smooth the base layer components I b A and I b B . This filter is defined as where p represents the pixel position. µ k and σ 2 k are the mean and variance of the image pixel values in a sliding window k with radius r, respectively. The size of the window k is (2r + 1) × (2r + 1). σ 2 d is a constant. The mean value µ k represents the average smoothing intensity of the filter, and its variance σ 2 k reflects and measures the changes in local sharpness of the image. In Eq. 7, if σ 2 k σ 2 d , there is S p = I p for pixel p; if σ 2 k σ 2 d , then S p = µ k . Therefore, we set σ 2 d as the minimum variance of the source image to achieve smoothing of tiny pixels. In this study, after many experiments, we found that our algorithm works best when r is set to 3 and σ 2 d is set to 0.01. In this article, a two-stage gradient is adopted to describe the salient area of the image. The gradient image G p can be defined as

where ⊗ represents convolution. L is the gradient detection template, In this article, for getting more detailed information, we introduced a gradient detection template in Zhan et al. (2019) , that is

Since holes may be generated in homogeneous regions when performing gradient operation on the image, we adopted morphological closure operation to fill the image holes in this article, that isM

where E denotes the sliding square structure element with radius r . By using Eq. 10, we can obtain the structural gradientsM A p andM B p of each base layer component. Then, the weight map is constructed by using the strategy of taking the larger absolute value. For the image value at the pixel p at the same position, the obtained binary weight map B p can be calculated as follows.

B p is obtained by selecting the saliency structure between I b A and I b A during image fusion. For the sake of making the fused image have more spatial continuity, we can get the basic weight map W p by performing average filtering on B p with a window size of 5 × 5. To make the image look more natural, the iterative joint bilateral filter is employed to transfer the edge information in source images into the weight map. The final decision map W p is obtained through three iterations, that is

Finally, the weighted sum fusion method is used to acquire the fused base layer component.

Combined with CNN, the Laplacian pyramid and the fusion strategy based on local similarity are adopted for the fusion of detail layer components in the proposed algorithm. The CNN structure adopted is shown in Figure 2 .

It has two identical branches and each branch has three convolutional layers and a max-pooling layer. In each convolutional layer, an ReLu is added for non-linear mapping, which has fast convergence speed and a simple gradient operation. The size of the neuron's receptive field depends on the size of the convolution kernel. Choosing an appropriate convolution kernel size is significant for CNN. If the convolution kernel is too small, local features cannot be extracted effectively; and if the kernel size is too large, the information representation capability is reduced and the image cannot be represented well. Therefore, during net training, the kernel size and step of each convolutional layer are set to 3 × 3 and 1, respectively. And the scale factor and span of the max-pooling layer are set to 2 × 2 and 2, respectively. The 256 feature maps are connected to a 256-dimensional feature vector through a fully connected layer. The output is a fully concatenated two-dimensional vector of the 256-dimensional feature vector.

In this algorithm, the two detail layer components I d A and I d B are input into the two branches of the CNN to generate a fusion weight map W. Then, Laplacian pyramid decomposition is performed on each detail layer component, which is denoted by L {A} l and L {B} l , respectively ( l denotes the decomposition scale). The weight map W is decomposed by Gaussian pyramid and expressed by G Then, we calculate the local similarity measurement of L {A} l and L {B} l , that is

From Eq. 15, we can know that the range of M l (x, y) is [−1, 1]. The closer the value is to 1, the higher the local similarity of L {A} l and L {B} l is. The threshold t is set to fuse pixels with different similarities using different fusion rules. If M l (x, y) ≥ t, L {A} l and L {B} l are fused through the weighted average fusion method, that is

If M l (x, y) < t, the fusion will be carried out by maximizing the local energy, that is 

In the study, five groups of medical images are chosen for testing, to evaluate the performance of the proposed fusion algorithm. These test images are displayed in Figure 3 and are all from http://www.med.harvard.edu/aanlib/home.html. These test image sets contain five groups, among which groups ( (7) the rolling guidance filtering multimodal medical image fusion by combining CNN and nuclear norm minimization proposed in Liu S. et al. (2020) (abbreviated as RGF-CNM). To make it fair, when using the above-mentioned fusion algorithms for testing, the parameter settings of these algorithms have the same parameters in the articles published by the authors.

We also choose six indicators for objective evaluation of the proposed algorithm, namely, Mutual Information (MI) (Qu et al., 2002) , QAB/F metric (Xydeas and Petrovic, 2000) , Structural Similarity (SSIM) (Wang et al., 2004) , Visual Information Fidelity (VIFF) (Han et al., 2013) , Universal Image Quality Index (UIQI) (Wang and Bovik, 2002) , Piella index (Q, QW, QE) (Piella and Heijmans, 2003) , and R AB/F Q metric (Sengupta et al., 2020) . MI reflects the useful information that remained in a fused image. The QAB/F index is an objective measure based on edge information and is adopted to assess whether the fused image retains more edge information. SSIM represents the structural similarity between the fused image and the source images. VIFF measures the fidelity of the visual information of the fused image relative to the source images. UIQI measures image distortion through correlation loss, brightness distortion, and contrast distortion. The Piella index comprehensively reflects the similarity of intensity, contrast, and structure between the fused image and the source images. R AB/F Q metric reflects the edge and orientation strengths. For all of these indexes, the higher value means that the performance of the fusion method is better. Figure 4 shows the output images obtained after each step of our fusion method applied in Group (a). Figures 4A,B To verify the effectiveness of CNN for detail layer fusion, we conducted an ablation study on Group (a). Figure 5 shows the fusion results of the ablation study. Figures 5A,B show the source images. Figure 5C is the fusion result without CNN, that is, the detail layers are fused by the method which is the same as the method for fusing base layers. Figure 5D is the fusion image of the proposed algorithm. Table 1 shows the objective evaluation results of the ablation study. It can be seen from Table 1 that the fusion algorithm with CNN has better objective evaluation results. The ablation study proves the effectiveness of CNN for detail layer fusion.

We tested all the methods in the test image sets in Figure 3 and the fused images are displayed in Figures 6-10. Figure 6 shows the fusion results of Group (a). Figures 6A,B show the source images. And Figures 6C-J are the fusion results of GFF, CNN, CSMCA, NSCT-PCNN, NSCT-PCLLE, NSST-PAPCNN, RGF-CNM, and the proposed algorithm, respectively. Figure 6 shows that the fused image of our algorithm has a higher contrast, shows more structural information and textural information, and introduces less useless information among these algorithms. And the edges of Figure 6C are not clear enough, and GFF lost some structural information. Figure 6D retains the spatial information of source images, but some edges are lost. Figure 6E has low contrast, and the fusion effect is not ideal. Figure 6F shows that the CSMCA does not get a good fused image, and cannot be able to retain useful information effectively. Although Figure 6G retains more useful information of source images, but a small number of artifacts are produced. There are unwanted artifacts The bold values are the best metric values. The bold values are the best metric values. The bold values are the best metric values.

in Figure 6H . The contrast in Figure 6I is high, but the edges are blurred. Table 2 shows the objective evaluation results of Group (a) for all test algorithms. As seen in Table 2 , our fusion algorithm has the highest values on MI, QAB/F, SSIM, Piella, and UIQI. The maximum values of MI and QAB/F indicate that our algorithm retains more useful information and edges.

In Table 2 , the highest indicators of SSIM and Piella indicate the highest structural similarity between the fused image and the source images obtained by our algorithm. The maximum value of UIQI indicates that the correlation, brightness, and contrast of the fused image and the source images obtained by our algorithm are closest. Although the VIFF of our algorithm is not the highest, from the visual effects of Figure 6 , our fusion algorithm is the best. In all comparison algorithms, the VIFF value of CNN is the highest, which is better than our method. However, the other seven evaluation indexes are lower than our fusion algorithm, and the value of MI is far from ours, which indicates that CNN is not ideal in preserving the information of source images. Therefore, our algorithm is an effective medical image fusion algorithm. Figure 7F loses more useful information. In addition, Figures 7F,I produce different degrees of blocking effects and artifacts. Figures 7D,G ,H fuse the two source medical images well, but still lose a small part of detailed and textural information, and do not get the best visual effects. Compared to the contrasting algorithms, the fused image of our algorithm has the best visual effect. Table 3 shows the objective evaluation results of Group (b) for all test algorithms. Table 3 shows that the proposed algorithm gives the best objective evaluation results except VIFF and QW. It indicates that the proposed algorithm retains the most useful information and edges, the structural similarity of fused image and source images is the highest, and the correlation, brightness, and contrast are the closest. In other contrast algorithms, the values of SSIM, VIFF, and QW of NSST-PAPCNN fusion algorithm are the highest, but other evaluation indexes are lower than the proposed algorithm, especially MI and QAB/F. From the visual effects and objective evaluation, the proposed algorithm achieves a satisfying fusion effect. Figures 8-10 show the fusion results of groups (c), (d), and (e). Similar to Figures 6, 7 , the proposed method retains more structural and detailed information of the input medical images, has higher contrast and brightness, and introduces less useless information, such as blocking effects and artifacts from the fusion results of Figures 8C-J, 9C -J, 10C-J.

From the comprehensive view of the objective indicators in Tables 4-6, the proposed algorithm has higher objective evaluation indicators than other algorithms. In addition, Table 7 shows the running times of these algorithms. Compared to others, the running time of our algorithm is also relatively competitive. Therefore, the proposed algorithm is an effective and robust medical image fusion algorithm.

In addition, we applied the proposed algorithm to multifocus image fusion and infrared and visible image fusion. In Figures 11A1-F1 are the one source images of each group. Figures 11A2-F2 are the other one source images. Figures 11A-F are the fused images. As shown in Figure 11 , the proposed fusion algorithm is also effective in multi-focus image fusion and infrared and visible image fusion.

In this article, we proposed a two-scale multimodal medical image fusion algorithm based on structure preservation. The proposed algorithm decomposes the source images by two-scale decomposition, which fully uses the multi-scale information of the images. Our algorithm also adopts the structure preservation characteristic of the iterative joint bilateral filter and applies CNN in medical image fusion. From the visual effects and objective measures, the contrast experiments show that the proposed algorithm has better performance than the compared algorithms. However, the speed of the proposed algorithm is not ideal. In future, the computational speed of the proposed algorithm will be optimized for practical application in clinical practice.

Publicly available datasets were analyzed in this study. This data can be found here: http://www.med.harvard.edu/aanlib/home. html.

Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

LY performed the computer simulations. SL, XS, and JZ analyzed the data. SL wrote the original draft. XS and Y-DZ revised and edited the manuscript. MW polished the manuscript. All authors approved the submitted version. 

Recent medical image fusion techniques: a review

Medical image fusion using transform techniques

Improved medical image fusion based on cascaded PCA and shift invariant wavelet transforms

NSCT-based multimodal medical image fusion using pulse-coupled neural network and modified spatial frequency

A survey on different multimodal medical image fusion techniques and methods

Edge-preserving decompositions for multi-scale tone and detail manipulation

Functional and anatomical image fusion based on texture energy measures in NSST domain

A new image fusion performance metric based on visual information fidelity

Guided image filtering

A review of multimodal medical image fusion techniques

Joint patch clustering-based dictionary learning for multimodal image fusion

Pixel-level image fusion: a survey of the state of the art

Image fusion with guided filtering

Medical image fusion method by deep learning

Adaptive weighted image fusion algorithm based on NSCT multi-scale decomposition

Multi-focus image fusion based on residual network in non-subsampled shearlet domain

Multimodal medical image fusion using rolling guidance filter with CNN and nuclear norm minimization

Medical image fusion based on improved summodified-Laplacian

Medical image fusion based on nuclear norm minimization

A medical image fusion method based on convolutional neural networks

Structure tensor and nonsubsampled shearlet transform based algorithm for CT and MRI image fusion

Multi-focus image fusion with a deep convolutional neural network

Image fusion with convolutional sparse representation

Medical image fusion via convolutional sparsity based morphological component analysis

MRI and PET image fusion using the nonparametric density model and the theory of variableweight

Image fusion based on multiscale transform and sparse representation to enhance terahertz images

Multi-focus image fusion using fractal dimension

Fractal dimension based parameter adaptive dual channel PCNN for multi-focus image fusion

MRI and SPECT image fusion using a weighted parameter adaptive dual channel PCNN

A new quality metric for image fusion

Information measure for performance of image fusion

Image fusion of noisy images based on simultaneous empirical wavelet transform

Edge information based image fusion metrics using fractional order differentiation and sigmoidal functions

Image fusion using wavelet transform and GLCM based texture analysis for detection of brain tumor

Multimodal medical image fusion techniques -a review

Bilateral filtering for gray and color images

Multi-modality medical image fusion using convolutional neural network and contrast pyramid

Diagnosis of COVID-19 by wavelet Renyi entropy and three-segment biogeography-based optimization

Covid-19 classification by FGCNet with deep feature fusion from graph convolutional network and convolutional neural network

COVID-19 classification by CCSHNet with deep fusion using transfer learning and discriminant correlation analysis

A universal image quality index

Image quality assessment: From error visibility to structural similarity

Image smoothing via L0 gradient minimization

Objective image fusion performance measure

Medical image fusion with parameteradaptive pulse coupled-neural network in nonsubsampled shearlet transform domain

Infrared and visible image fusion based on a latent lowrank representation nested with multiscale geometric transform

A fast fusion method for visible and infrared images using fourier transform and difference minimization

Multimodal image seamless fusion

Region-and pixel-level multi-focus image fusion through convolutional neural networks

A novel fast single image dehazing algorithm based on artificial multiexposure image fusion

A novel multimodality image fusion method based on image decomposition and sparse representation

A phase congruency and local laplacian energy based multi-modality medical image fusion method in NSCT domain

We thank the High-Performance Computing Center of the Hebei University for their support.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.Publisher's Note: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.Copyright © 2022 Liu, Wang, Yin, Sun, Zhang and Zhao. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.