Constant elasticity of substitution function based RANSAC for image stitching

. Feature matching is very important in image stitching. RANSAC algorithm is a representative algorithm for feature matching. However, RANSAC still has many shortcomings such as a large number of iterations, a large computational complexity and cannot completely eliminate mismatches. To address above problem, in this paper, we propose a novel method termed constant elasticity of substitution function based RANSAC (CES-RANSAC) for image stitching. Specifically, CES-RANSAC improves the RANSAC algorithm by constructing a utility function, optimizing the boundary of the utility function, calculating Cobb-Douglas coefficients. It also introduces Lindahl equilibrium to derive the return value t to help eliminate mismatches. Experiments show that compared with the traditional RANSAC algorithm, CES-RANSAC has improved matching accuracy and increased computational efficiency, which further improves the efficiency of the image matching algorithm.


introduction
Image stitching technology is mainly aimed at stitching two or more pictures with a certain overlapping area into a complete image [1][2]. Image stitching technology is widely used in fields such as medicine, industry, remote sensing images, and drone aerial photography. The most critical step in image stitching technology in feature matching. The speed and accuracy of feature matching directly determine the accuracy and speed of the stitching algorithm. Literature [3][4][5][6][7]introduces image matching algorithms that are currently widely used internationally, among which the typical algorithms for feature point-based matching are SIFT and SURF. Both of them generally use the RANSAC algorithm to eliminate mismatches to solve the transformation matrix, but the iteration time of the traditional RANSAC algorithm is significantly changed by the amount of original data, and the accuracy of the original data directly affects the accuracy of the solved transformation model. The current research on the RANSAC algorithm is roughly optimized based on the following two aspects [8][9][10]. On the one hand, when selecting a subset, a specific selection scheme or a constrained random selection can be used to replace the original completely random selection based on some known sample characteristics; on the other hand, when the model is calculated from the consensus set, all samples in the sample set whose errors with the model are less than a specific t can be added to the consensus set, and the model can be recalculated. From the above two perspectives, this paper proposes a preference-based utility function improved RANSAC algorithm, and applies it to image stitching. By purifying the original matching point to the data set, eliminating mismatch points, reducing the amount of original data of the RANSAC algorithm, improving the matching speed and the accuracy of the transformation matrix.

Constant Elasticity of Substitution function based RANSAC
The method in this paper is summarized as follows. First, for the feature points extracted by the SURF algorithm, the preferred utility function is introduced for grouping and matching, so as to obtain the matching point array of the reference image and image . Then use the substitution of the utility function to judge the data in the array, eliminating the data that does not meet the constraint conditions, and further increase the proportion of interior points. At the same time, for the return value that does not meet the threshold, Lindahl equalization is performed, and then the return value is added to the previously obtained consensus set to estimate the model, and the obtained model is used to obtain an accurate transformation matrix. The target image can be obtained by using the transformation matrix for registration and image fusion. The algorithm flow chart is shown in Fig. 1

Feature point matching and grouping
We first collect the feature points of the collected images through the SURF, and obtain the reference image matching point set and the corresponding image to be registered. Use the improved RANSAC algorithm to purify the collected feature point set. The specific method is to use the utility function of preference to achieve preliminary matching and grouping of feature points. Bring the sample points into the CES utility demand function, and obtain the reference image matching point set and the corresponding matching point set of the image to be stitched . Constant Elasticity of Substitution function is defined as

Alternative Flexible Judgment
For a set of matching points and , there are certain constraints, which need to meet the substitution elasticity of the utility function. We use this as a constraint condition to judge whether a set of matching points is a correct matching point. The constraint conditions must be satisfied as follows: This shows that the size of the substitution elasticity can determine the substitutability between the two images to be spliced, so that whether the new matching point meets the model can be determined according to the value of the substitution elasticity. Drawing lessons from the value in economics and many experimental tests, when δ is set to 0.34, the mismatch can be greatly reduced, and more correct matching points can be left for the RANSAC operation to obtain the conversion matrix H.

Lindahl equilibrium
In the introduction, we mentioned that the traditional RANSAC algorithm can also recalculate the model by adding all the samples in the sample set whose error with the model is less than a specific t into the consensus set. Because the previous step of operation may cause some sample points to be lost, resulting in a decrease in the splicing effect. We use Lindahl equilibrium to solve the specific error t that can be returned to further improve the splicing effect. Where t is defined as : In economics, it can be proved by the following theory that after the Lindahl equilibrium condition is satisfied, the sum of the marginal rate of substitution is 1. So here the specific error t, we take 0.66. We rejoin the specific error t<0.66 to the consensus set to solve the conversion matrix H.

Stitched image
Apply the transformation matrix obtained in the previous section to the image registration step. Through the transformation of the transformation matrix, the position coordinates of the matching point of the image are projected onto the new image after mapping. At the mapping coordinates of the matching point on the new image, the two images are connected. The left image of this point is completely image , and the right side is completely image . The stitching between the two images is realized.

Experimental results and analysis
In order to prove the effectiveness of the algorithm proposed in this paper, the algorithm simulation was carried out in the experimental environment of MATLAB2017b.The analysis of the processing results of the original RANSAC algorithm and the improved method of this article on several different scene pictures (891*1587). represents the matching accuracy of the original algorithm, represents the accuracy of the improved algorithm matching, is the average time consumed by the original algorithm, and is the average time consumed by the improved algorithm. and are defined as follows , , The above formula, represents the number of correct matching point pairs in the original algorithm, represents the total number of matching point pairs in the original algorithm; represents the number of correct matching point pairs in the improved algorithm, and represents the total number of matching point pairs in the improved algorithm. Table 1, represents the reduction of the logarithm of matching points between the improved algorithm and the original algorithm.
represents the increase in the matching accuracy of the improved algorithm compared with the original algorithm. ∆T means the average time consumed by the improved algorithm compared with the original algorithm. In this paper, different pictures are processed through experiments, and the processing result data of four groups of pictures (Figure 1, 2, 3, 4) are collected, as shown in Tables 1. The stitching effect is shown in the Fig. 2~3  Fig. 2. Two sets of images to be stitched. Through the data comparison in Table1 and the simulation effect of the improved algorithm, we believe that the preferred utility function optimizes the RANSAC algorithm to further improve the image stitching effect and shorten the algorithm calculation time.

Conclusion
In this paper, we propose a method for improving the utility function based on preference. The method improves the three existing problems of the RANSAC algorithm and proves it by simulation. Experiments show that this method can improve the accuracy of the transformation matrix calculated by the RANSAC algorithm, obtain a more ideal splicing effect, and at the same time increase the calculation speed to a certain extent. However, the determination of the threshold in this algorithm requires multiple experimental tests, and the threshold may also need to be adjusted appropriately according to the size of the image to be processed. At the same time, further optimization should be made in the stitching of rotated pictures, because the current images that need to be stitched may not always be able to achieve the same angle. For image stitching with an overlap area of less than 30%, consideration should be given.