MEDICAL IMAGES FUSION ALGORITHM BASED ON PROBABILISTIC GAMMA-NORMAL MODEL WITH STRUCTURE-TRANSFERRING PROPERTIES

: In medical image processing, image fusion is the process of combining complementary information from different (multimodality) images to obtain a fused image, which plays a vital role in further analysis and treatment planning. The main idea of this paper is to improve the image content by fusing computer tomography (CT) and magnetic resonance (MR) images. We propose here the new algorithm based on the probabilistic gamma-normal model with structure-transferring properties. Firstly, we select the areas with the highest pixel intensity on original CT and MR images. In parallel with this, the structures of original images are distinguished using the probabilistic gamma-normal model. The weighted-fusion image can be obtained based on detected objects and structure. Finally, we smooth the weighted-fusion image using the structure-transferring ﬁlter and combine the smoothed image with the weighted-fusion image for obtaining the resulting image. The key point here is that we do not need to re-allocate the structure, which leads to the reduction of computation time. The proposed method gives the best result in terms of the spatial frequency metric and lower computation time than other image fusion methods.


INTRODUCTION
The fast development of computer technology promotes the progress of medical devices and consequently medical imaging. Different medical imaging methods provide different information about the same object. For example, computed tomography (CT) is best at picking up bone structures, so CT is used for identifying different bone injuries. On magnetic resonance (MR) imaging, soft tissues are clearly visible, such as muscles, blood vessels, cartilage, spinal cord and brain. On the other hand, CT scans have higher imaging resolution, and fewer motion artifacts due to their high imaging speed, than MR imaging. In medical image processing, image fusion is the process of combining complementary information from different (multimodality) images to obtain a fused image, which plays a vital role to improve in medical image analysis and treatment planning. The main idea of this paper is to improve the medical image content by fusing CT and MR images.
Researchers pay great attention to solving the problem of combining multimodal medical images. Recently, guided filters (He et al., 2013) have been actively used to solve image processing tasks, because they could extract a structure from some guided image and process another image, called an analyzed image, using extracted structure. Guided and analyzed images are images make on the same scene, but in different conditions. The guided filter-based fusion method (GFF) (Li et al., 2013) allows each source image to be decomposed into base and detail layers. First, contrast details are extracted to build weight maps, then the guided filter is used to refine weight maps of base and detail layers, and a resulting fusion image is generated by a weighted average method. In the paper (Yang et al., 2017), authors created a model for combining medical images based on a filter that controls a gradient region and a weight map (multiple gradient guided filtering for fusion, MGGFF), due to which it is possible to combine several visual characteristics, * Corresponding author including contrast measures, sharpness and structure severity. Medical image fusion can be also performed using multiscale transform (MST) techniques (Liu et al., 2015). Multiscale geometric analysis tools can be used for image fusion, including non-subsampled shearlet transform (NSST) (Liu et al., 2018). Besides, MST methods can extract information about the difference in resolution of original images and use it in combination with other image processing methods, such as, for example, a pulse coupled neural network (PCNN) (Yin et al., 2019). Medical image fusion can be also viewed as an optimization problem. Variational image fusion methods always contain two terms. The first term is the precision rate used to constrain grey approximation between the fusion image and original images, and the second term is the regularization constraint. In the paper (Ma et al., 2016), the authors proposed a method for combining images based on variation function, where the norm of total variation function (TV) was taken as regularization, and the quadratic norm was taken as the norm of accuracy. Total generalized variation (TGV) (Wang, Yang, 2018) can be used as a constraint on higher-order regularization for combining images since it can suppress a "stepwise" effect of processing at objects boundaries of the resulting image. Some of the abovementioned methods can add halos and blur around the edges of objects, and noise in the resulting image.
The paper (Wang, Yang, 2020) can be mentioned among the latest works, where features of intensity and geometric structure of images are extracted using the saliency detection method and the structure tensor, and then they are combined into a fusion image, which is processed by a filter based on the variational model. Such methods show high processing quality, but they have high computational complexity.
In this paper, we propose a new image fusion method based on a scheme from (Wang, Yang, 2020), but we use the probabilistic gamma-normal model with structure transfer properties (Gracheva et al., 2015, Gracheva, Kopylov, 2017 for ex-traction of features and structures, and obtaining the resulting image. This model shows good results in high dynamic range compression on medical images (Gracheva, Kopylov, 2019). The proposed method was compared with seven image fusion methods using three different metrics of quality estimation. The computational time of algorithms was also estimated. The proposed method gives the best result in the terms of the spatial frequency metric and lower computation time than other image fusion methods.

THE MEDICAL IMAGES FUSION METHOD
Bones and structures, which contain calcium, appear in white on CT images, soft tissues (such as the heart) appear in shades of grey, and tissues that are close in density to air (such as lungs, intestines) appear in black. The colour of objects on MR images is determined by the amount and density of fluid contained in it, respectively, abnormal objects in soft tissues are spots of varying intensity. Thus, combined CT and MRI medical images should include all key features of both types of images. The proposed method aims to preserve information about the intensity of main objects and highlight structures of soft tissues on images of computed tomography and magnetic resonance imaging.
The general scheme of the proposed approach is shown in  The first step of the proposed method is selecting areas with the highest pixel intensity on original CT and MR images. The probabilistic gamma-normal model is used here to smooth the original CT and MR images for noise and small details removing, after that smoothed images are binarized ( Figure 2). In parallel with this, the structure of objects on original images is also distinguished using the probabilistic gamma-normal model and normalized (Figure 3). Within the framework of the probabilistic gamma-normal model, two parameters are subject to evaluation -the field of values of the model accuracy (inverse variance), which represent the structure of objects in original image, and the hidden component of a two-component random field (Gracheva et al., 2015, Gracheva, Kopylov, 2017. Contrast features (areas) are used to detect object's intensity information, and structures are an effective tool to describe image geometry. The goal of the weighted fusion is to extract highintensity regions from input medical images, and preserve structure of other regions. We use results obtained at feature extraction stage (SCT , SMR) and structure extraction stage (STCT , STMR) for CT and MR images, respectively. So we calculate weight maps as follows (Wang, Yang, 2020) (see Figure 4) where SCT and SMR = CT and MR images after feature extraction stage STCT and STMR = CT and MR images after structure extraction stage The weighted-fusion image is obtained by combining original CT and MR images and selected contrast feature and objects structure (Wang, Yang, 2020) where wCT and wMR = weight maps are obtained by (1) X g CT and X g M R = original CT and MR images Y = the weighted-fusion image The process of obtaining the weighted-fusion image is shown in Figure 5.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIV-2/W1-2021 4th Int. Worksh. on "Photogrammetric & computer vision techniques for video surveillance, biometrics and biomedicine", 26-28 April 2021, Moscow, Russia The weighted-fusion result can be blurred and had some noise because of different grey matter and white matter distributions on original CT and MR images. We use the structuretransferring filter based on the probabilistic gamma-normal model to eliminate this effect and smoothed image acquisition.
The key point here is that we do not need to re-evaluate the structure of objects, which leads to the reduction of computation time. Combining the weighted-fusion image and smoothed image gives the resulting image.

THE PROBABILISTIC GAMMA-NORMAL MODEL
Within the framework of the Bayesian approach used here for solving image processing problems, the analyzed image (weighted-fusion image, which was obtained by (2)) and the processing result (smoothed image) are considered as the observable Y = (ys, s ∈ S), and hidden components X = (xs, s ∈ S) of a two-component random field (X, Y ) and are determined on a discrete set S = {s = (s1, s2) : s1 = 1, ..., N1, s2 = 1, ..., N2}. The estimation of the hidden field (X, Λ), where Λ = (λs, s ∈ S) is the random field of inverse variances, which represents objects structure, is reduced to solving the following optimization problem (Gracheva et al., 2015, Gracheva, Kopylov, 2017: (X,Λ|Y, δ, µ, η) = arg min where δs, s ∈ S = the precision of observations Λ = (λs, s ∈ S) = hidden field of irregularity factors, whose elements are inverse values of the precision of observations G = the variable adjacency graph having the form of a lattice for images µ and η = structural parameters which set the degree of image smoothing and selectivity, respectively Criterion (3) makes it possible to obtain estimatesΛ of Λ = (λs, s ∈ S) by the guided image, which, in turn, gives optimal estimatesX of the hidden field X = (xs, s ∈ S). Note that estimatesΛ are actually a combination of estimates, since two original medical images (CT and MR) act as the guided image.
The parametric procedure of dynamic programming was used to optimize criterion (3). The adjacency graph of image elements G is approximated by a sequence of horizontal and vertical trees. The proposed procedure has linear computational complexity relative to the original image size.
The developed structure-transferring filter has a property that it recovers small-scale details of the weighted-fusion image in the neighbourhood of large-scale structures of the input image. Based on the property of the structure-transferring filter, the fusion result is constructed by combining the smoothed image and source images.

EXPERIMENTAL RESULTS
CT and MR images from the database the whole brain atlas website (http:www.med.harvard.edu/aanlib/) were used for the experimental research. Pair CT and MR images are made for the same person and the same conditions. Some CT and MR image samples used in this experimental research are shown in Figure 6. have some blurred details in fused images of (Figures 7(g), 7(i), and 7(j)). The proposed method, FEVO and PAPCNN fusion methods preserve the lesion region and bone structures well without reducing the intensity contrast (Figures 7(c), 7(d), and 7(h)).
Since it is almost impossible to obtain medical images datasets with ground truth images, the quality of image fusion results based on the proposed method was assessed using popular image quality assessment metrics: spatial frequency (Li et al., 2001), mutual information (Hossny et al., 2008), gradient-based fusion metric (Xydeas, Petrovic, 2000).
The spatial frequency (SF) is to measure the overall grey difference in an image: where RF and CF = are the row and column frequency, respectively Mutual information (MI) measures the degree of dependency between two variables: , where H(A) = I(A, A) = entropy of images The gradient-based fusion metric Q AB/F measures the relative amount of edge information that is transferred from the original images X g CT and X g M R into the fused image Y : where Q X g CT Y and Q X g M R Y = are the edge preservation values at the pixel location (i, j) W X g CT Y and W X g M R Y = are the weights, which indicate the importance of Q X g CT Y and Q X g M R Y , respectively The SF, MI, Q AB/F metrics reflect different information including image spatial frequency, the correlation between the fused image and input images in grey level, and gradient level, as shown on diagrams in Figures 8-10 for the above pair of CT and MR images. Figure 8. Diagram of the quality assessment by SF metric Figure 9. Diagram of the quality assessment by MI metric Figure 10. Diagram of the quality assessment by Q AB/F metric Also, we compared image processing results with other modern image fusion methods by computation time (Table 1). Experiments are performed in MATLAB R2015a, Intel Core i7-5500U CPU 2.40 GHz.
The proposed method preserves rather well the characteristic features of the fusion images, such as high-intensity bone structures, features of tumors, and information about the texture of soft tissues. Comparing results of combined CT and MR images, presented in diagrams (Figures 8-10 based on the probabilistic gamma-normal model does not give a smoothing effect during fusion of original images. MI and Q AB/F metrics do not give the best results due to the poor quality of original images, which leads to distortion of objects boundaries in original images.

CONCLUSION
This paper presented a new method for combining medical CT and MR images based on the probabilistic gamma-normal model. The proposed method allows separately extracting features and structuring in original images, and then combining them to preserve the key details in the resulting image. The proposed method was compared with the other seven methods of fusion images, which show the superiority of the proposed method in terms of the SF metric. An important feature of the proposed method is low computation time in comparison with other methods. The proposed scheme can be used for developing fusion methods of other types of medical images, for example, MR-PET images.