A Multi-exposure Image Fusion Method with Detail Preservation

In view of the problems of uneven exposure in the image acquisition and the serious loss of details in the traditional multi-exposure image fusion algorithm, a method of image fusion with details preservation is proposed. A weighted approach to multi-exposure image fusion is used, taking into account the features such as local contrast, exposure brightness, and color information to better preserve detail. For the purpose of eliminating the noise and interference, using the recursive filter to filter. Compared with other algorithms, the proposed algorithm can retain the rich detail information to meet the quality requirements of spot welding image fusion and has certain application value.


Introduction
In recent years, image fusion has become a key research area in the field of information fusion.Image fusion includes two sources of images, one obtained by different sensors, and the other taken under the same exposure time of the same sensor.The image fusion system includes pixel layer fusion, feature layer fusion and decision layer fusion.In this paper, pixel-based multi-exposure image fusion algorithm can be used to fuse the most basic pixel units and extract the accurate details of the image.In addition, it is also a widely used method now.
Much research has been done at home and abroad on the multi-exposure image fusion.The multi-exposure image fusion algorithm based on the S-curve is proposed by Fuzheng Fang et al., But it has a great limitation and can only improve the fusion effect to some extent [1], Mertens et al proposed Laplacian-based exposure fusion, but this method could not take into account the global brightness and local details [2].Shen et al.Proposed a multi-exposure image fusion algorithm based on the improved pyramid, which can remain more details , But the algorithm is more difficult to calculate [3].Sujoy et al.Proposed multi-exposure and multi-focus image fusion algorithms [4].Vassilios et al.Proposed a new method that can be used to fuse two or more different exposure images in a high dynamic range scene [5].From the aspects of the results of the fusion image and the specific details of the information and so on, the above methods can not achieve the desired results.
For the purpose of preserving the details of the images better to show the richer image information, this paper proposes a multi-exposure image fusion method with detail preservation.Firstly, three characteristic indexes of the image are calculated: local contrast, exposure brightness and color information, the results of the characteristic index are used to calculate the weight map, and the weight map is corrected by recursive filtering.Finally, this article uses the method of weighted fusion to get the fusion image, which can remain abundant image detail information.

Image fusion algorithm
The proposed algorithm is suitable for static scene.The algorithm mainly involves three steps: calculating the image feature index, calculating the weight graph and weighted fusion.

Characteristic index calculation
According to the characteristics of static scene, selecting the three features of local contrast, exposure brightness and color information to calculate the exposure fusion weight.

The process of algorithm
The following is the specific calculation of this algorithm: First of all, the weight can be estimated by the local contrast, exposure brightness and color information, it's necessary to calculate the local contrast of each pixel firstly; according to the brightness of each pixel to determine the exposure of the pixel, and then the threshold processing step of the exposure brightness is also carried out.A larger weight is assigned to a pixel with a proper exposure, and a smaller weight is assigned to a pixel with an underexposed or overexposed area, the specific approach is: The original image is converted to a grayscale image and normalized, then processing the pixels and each pixel is assigned a corresponding weight using a Gaussian equation.In this paper, color saturation is used to measure the color information of the image as one of the parameters to estimate the weight.The three weights are integrated to obtain the initial weight map of image fusion.

Figure. 1. Algorithm flowchart
As the weight map will inevitably introduce noise and interference in the process, this paper uses recursive filter to filter, recursive filtering is a smoothing filter that preserves edge information in real time.It has the advantages of fast speed and perfect edge preservation.It has been widely used in image processing.The weight graph is processed by recursive filtering to get the exact smoothed weight graph, and the final fused image is obtained by weighted fusion using the obtained weight graph.The flowchart is the Figure 1 above.

Image fusion evaluation method
The quality evaluation of image fusion is very important for image fusion.Image fusion is a pre-processing part of a specific task.The evaluation of its fusion performance depends on whether it can improve the performance of subsequent tasks.For image observers, the meaning of the image mainly includes two aspects, one is the fidelity of the image and the other is the image comprehensibility.There are two main methods to evaluate the performance of image fusion algorithms: subjective evaluation and objective evaluation.

Colour illustrations
Structural Similarity (SSIM) is a typical evaluation method based on characteristic parameters [6], structural similarity, which measures the similarity of images from three aspects of brightness, contrast and structure respectively.It is a image quality evaluation index which is used widely.The brightness, contrast and structure using the following formulas:  and Y  represent the mean of X and Y respectively, X  and Y  represent the variances of X and Y respectively, XY  is the covariance of X and The average structural similarity is

Entropy
Entropy is an important index to measure the abundance of image information.If the entropy of the fused image is larger compared with the original image, it means the fused image has more information than before [7].Its formula is below: L represents the total gray level of the fused image, i P represents the ratio of the pixel number i N with the gray value of i to the total number of pixels of the image N namely: , which reflects the probability distribution of the pixel with the gray value of i in the image and can be regarded as the normalization of the image histogram.

Experimental results and analysis
In the experiment, we first use the standard images to analyze this article's lgorithm(Det), the Laplacian pyramid algorithm(Lap) [8,9], the contrast pyramid algorithm(Cos) [10],the weighted average method(Ave) [11] , the principal component analysis identification algorithm(Pca) [12],the improved wavelet transform method(Dwt) [13,14] have been tested to obtain a preliminary comparison.
The three images of standard image arch, color and puppet are respectively fused by the six algorithms of the algorithm, Laplacian pyramid, contrast pyramid, principal component analysis, improved wavelet transform and weighted average method.The results of the fused images are shown in Figure 2, 3 and 4. The results of image fusion are evaluated and the information entropy and average structural similarity of each fusion algorithm are calculated Looking at the datas in Table 1 and Table 2.     Analyzing the results of various algorithms of image arch firstly, the result of this article ' s algorithm is obvious at the top of the arch, and the shadows and wall textures are clearly expressed with more detail retained.The other five methods, some of which have shaded parts and some of the textures are fuzzy, information is incomplete .
From the results of the image color, we can see that the results obtained by this method and the improved wavelet transform have more edge information and can clearly observe the detail changes of the image.Based on the Laplacian pyramid and the contrast pyramid ,the contrast of the image results is low, and the results of both algorithms are slightly vague and the details are not enough.
From the fusion results of the image puppet , the method of this article's algorithm is better, other methods have more dark places, the contrast is relatively low, this article's algorithm retained more details.
As can be seen from the data in the table, the proposed algorithm has relatively good effect on information entropy and average structure contrast.

Conclusions
In this paper, it presents a multi-exposure image fusion algorithm that preserves more details.Then calculating the weight map using the selected features, the noise and interference are eliminated by using recursive filtering and the weights are estimated.At last, the images are fused.In the static scenario, making a comparation with the other five algorithms, according to the subjective and objective evaluation, especially through the two parameters of information entropy and structural similarity, can show that the proposed algorithm has some advantages and can obtain the high quality of the fused images , remained more details, made a good preparation for the follow-up work.

Table 1 .
Image entropy evaluation results

Table 2 .
Comparison of average structural similarity