A method of image processing with QR code ablated on rough and highly reflective metal surface by laser

For the purpose of solving the tough problem of the recognition of QR code which is marked on rough and highly reflective metal surface by laser, this work proposes a method of image processing based on multi-feature fusion. This method was requested to establish the integrated feature combined by color feature, texture feature and classify pixel points by means of k-means clustering, then optimize the image of QR code by morphology, Finally, this method was applied to the QR code Laser marking on the AL97 casting aluminum ingots to recognize, then compare with the accepted method OTSU algorithm, The experimental results show that the method is effective obviously. 1Instructions Laser ablation is an important method of Direct Part Marking(DPM) technology, which has the advantages of non-contact, high speed, low cost and high resolution[1]. It, direct ablation of QR code on material surface by laser, has become an inevitable trend for material identification, and has been widely used in metal material, glass, ceramics, plastics and other materials[2-5]. Metal is a common material in the manufacturing process, the surfaces of metal have the characteristics of texture, high reflection and uneven illumination, as a result, laser ablation of the QR code is very difficult to obtain high-quality images[6-7], thus affect the effects of subsequent QR code recognition. For this problem, Wang Juan proposed an improved DPM bar code localization algorithm based on the improved Smallest Univalue Segment Assimilating Nucleus(SUSAN) corner detection and the near-neighbor propagation clustering under the semi-supervisory mechanism for the different brightness and deformation of the QR code image of the PCB and metal parts[8].Liu Ning-zhong proposed an algorithm which uses inverse filtering to reverse-restore the bar code image as the core, it solves the problem that inaccurate detection location of bar code and the low recognition rate in the complex background[9]. Aiming at the problems of high brightness, blurring and distortion in DPM code of metal parts, Liu Zhi and Mei Hongfang proposed to combine image acquisition with homomorphic filtering and machine learning to enhance the fast positioning of bar code area[10], and Zheng H R et al proposed the method of homomorphic filtering combined with gradient projection to realize the rapid positioning of metal surface QR code[11]. considering the problems of QR code positioning in metal parts because of the complex background and uneven illumination, An accurate positioning method for 2d bar code of complex background metal parts based on edge and level set was proposed by Guo Gaifang et al[12],Wang Jia jing et al proposed a method to recovery QR code’s spilled information based on stochastic resonance algorithm for the problems of high light low contrast and partial exposure on metal surface[13]. In the background of the local high-light phenomenon of laser marking on the metal surface, the bar code information of the high light area are reconstructed by the bar code reconstruction method based on five-step reconstruction model[14]. The main starting points of the above research works are the optical field correction , bar code location and so on under the complex background, and these works have solved the difficulties in 2d bar code recognition of some metal surface successfully in certain specific occasions. However, in practical applications, there are a class of metals with rough and highly reflective metal surfaces, such as cast aluminum, cast zinc, cast steel, cast nickel, cast lead, which are difficult to apply the above algorithms, and their surfaces not only have the characteristics of common metal texture and high reflective, but also have the problem of image noise and uneven surface, as a result, these affect the recognition effect seriously. Considering the above situations, we select the aluminum ingot casting as the research object to propose QR code preprocessing method based on comprehensive feature clustering of color and texture in this paper. This method makes use of the combination of color feature and texture feature to compose comprehensive feature, and uses K-means clustering to © The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons Attribution License 4.0 (http://creativecommons.org/licenses/by/4.0/). MATEC Web of Conferences 232, 02024 (2018) https://doi.org/10.1051/matecconf/201823202024 EITCE 2018

Abstract. For the purpose of solving the tough problem of the recognition of QR code which is marked on rough and highly reflective metal surface by laser, this work proposes a method of image processing based on multi-feature fusion. This method was requested to establish the integrated feature combined by color feature, texture feature and classify pixel points by means of k-means clustering, then optimize the image of QR code by morphology, Finally, this method was applied to the QR code Laser marking on the AL97 casting aluminum ingots to recognize, then compare with the accepted method OTSU algorithm, The experimental results show that the method is effective obviously.

1Instructions
Laser ablation is an important method of Direct Part Marking(DPM) technology, which has the advantages of non-contact, high speed, low cost and high resolution [1]. It, direct ablation of QR code on material surface by laser, has become an inevitable trend for material identification, and has been widely used in metal material, glass, ceramics, plastics and other materials [2][3][4][5].
Metal is a common material in the manufacturing process, the surfaces of metal have the characteristics of texture, high reflection and uneven illumination, as a result, laser ablation of the QR code is very difficult to obtain high-quality images [6][7], thus affect the effects of subsequent QR code recognition. For this problem, Wang Juan proposed an improved DPM bar code localization algorithm based on the improved Smallest Univalue Segment Assimilating Nucleus(SUSAN) corner detection and the near-neighbor propagation clustering under the semi-supervisory mechanism for the different brightness and deformation of the QR code image of the PCB and metal parts [8].Liu Ning-zhong proposed an algorithm which uses inverse filtering to reverse-restore the bar code image as the core, it solves the problem that inaccurate detection location of bar code and the low recognition rate in the complex background [9]. Aiming at the problems of high brightness, blurring and distortion in DPM code of metal parts, Liu Zhi and Mei Hongfang proposed to combine image acquisition with homomorphic filtering and machine learning to enhance the fast positioning of bar code area [10], and Zheng H R et al proposed the method of homomorphic filtering combined with gradient projection to realize the rapid positioning of metal surface QR code [11]. considering the problems of QR code positioning in metal parts because of the complex background and uneven illumination, An accurate positioning method for 2d bar code of complex background metal parts based on edge and level set was proposed by Guo Gaifang et al [12],Wang Jia jing et al proposed a method to recovery QR code's spilled information based on stochastic resonance algorithm for the problems of high light low contrast and partial exposure on metal surface [13]. In the background of the local high-light phenomenon of laser marking on the metal surface, the bar code information of the high light area are reconstructed by the bar code reconstruction method based on five-step reconstruction model [14].
The main starting points of the above research works are the optical field correction , bar code location and so on under the complex background, and these works have solved the difficulties in 2d bar code recognition of some metal surface successfully in certain specific occasions. However, in practical applications, there are a class of metals with rough and highly reflective metal surfaces, such as cast aluminum, cast zinc, cast steel, cast nickel, cast lead, which are difficult to apply the above algorithms, and their surfaces not only have the characteristics of common metal texture and high reflective, but also have the problem of image noise and uneven surface, as a result, these affect the recognition effect seriously. Considering the above situations, we select the aluminum ingot casting as the research object to propose QR code preprocessing method based on comprehensive feature clustering of color and texture in this paper. This method makes use of the combination of color feature and texture feature to compose comprehensive feature, and uses K-means clustering to classify pixel, Finally, the QR code images with low contrast, uneven light intensity and image noise are optimized by morphological method. The experiments prove the effectiveness of the method.

Problem descriptions about Image of QR code on casting aluminum
Remelting casting aluminum ingots is the common metal raw material in industry, and QR code on its' surface is very bad in quality. As shown in Fig.1, there are not only texture and high-reflection in image of QR code on the surface of remelting casting aluminum ingots,but also noisy result from uneven surface. This phenomenon is common in casting metal material.
According to the test method of QR image in bar code standards, the quality of QR image can be described by contrast. Contrast formula is generally as follows: In the formula, C is the contrast of QR image, R G is the average of minimal 10% pixels gray scale, B G is the average of maximal 10% pixels gray scale. QR image contrast (C) is used as QR symbol contrast (SC), A graded standard of QR image quality is established according to SC as follows: We randomly selected one hundred laser marking QR images on Casting aluminum ingots to get the value of SC and analyse, statistics about SC illustrate that most of them quality grades is F for SC range from 0.02 to 0.15 and the QR images quality is too low to scan as Fig.2.

Analysis of QR image process method
QR code image processing could be considered as a segmentation between foreground and background. Supposing there is a QR code image with N pixels,after feature extraction and integration, we can get a feature vector with d dimensions for each pixel. Feature vectors of all pixels form feature vector set X,X can be expressed as: The image needs to be divided into two categories: the foreground and the background, so the number of divided classes is C=2. According to the overall probability density of the feature vector Xi of the Gauss mixed model, it can be expressed as:  When the feature distributions of all the categories meet the Gaussian distribution, Eq(2) can be written as : In the formula, (μ,Σj) are the Gauss distribution parameters of class j , representing the mean vector and the covariance matrix, respectively.
According to the Bayes formula, the j-class's posterior probability of the pixel corresponding to the feature vector Xi is calculated as: Combining Eq(4) and Eq (5), we can get as follow: On the basis of maximum likelihood function, the maximum expectation algorithm (EM) is used to get the ) | ( Then: Obviously, image segmentation can be achieved by approach of multi-feature integration from the above theoretical analysis.

Image Processing Algorithm Based on Feature Fusion
As the previous analysis, it can be feasible for QR code image recognition on rough and high brightness metal surface by the background segmentation method of multi -feature fusion. The QR code images have obvious differences in texture and color features. Therefore, the method of texture and color feature fusion is designed to divide the image. The specific steps are as follows: (a) Normalize the size of the original image.
(b) Convert color space and extract color features.
(c)Enhance texture by DoG and extract texture features.
(d) Fuse color features and texture features. (e) Use K-means algorithm for pixel clustering.
(f) Use morphological opening and closing method to optimize image and output binary images.

Extraction of color features
Analyzing Color feature of the QR code image on the surface of aluminum ingots and plotting RGB color space and Lab color space distribution, we can see scatter plot as  (Table 2). Obviously the distribution of Lab color space tends to be more clustered, and the level of interdependence of RGB components on each other is high, especially the brightness is determined by the three component (R G B) together and the similarities of different color space regions is different. So Lab color space is more suitable for color features. Finally,a three-dimensional vector C(L,a, b) can be defined as a description of the color features of pixel (x,y),that is

Extraction of texture features
The rough surface of aluminum ingot showed more high-frequency noise under natural lighting. But the laser ablation surfaces of aluminum ingot are flat and the reflectance was low, so they are mostly in the lowfrequency area. Otherwise, appropriate enhancement of image texture difference is conducive to get a good description of the texture features of QR code image on aluminum. The Difference of Gaussian(DoG) filter is similar to the Laplace of Gaussian(LoG) filter, and it is the optimal choice for detecting the intensity when ratio of the standard variance, σ 1 and σ 2, is 1.6: 1.0. Convolution results are expressed as follows:

The mathematical model of DoG filter is
For calculating faster and better, we attempted and found that the effects are pretty good for most images when 0 . 1 , 6 . The Co-Occurrence Matrix P is used to describe the QR code image texture. 14 kinds of texture feature can be extracted from P, but only three descriptors which are entropy, inertia and energy can be distinguished with eyes. The specific definition is as follows: entropy  

Unsupervised classification of comprehensive features
For the purpose of normalizing color feature vector and texture feature vector, all the pixel sub-features are extended to the to the 6-dimesion space, Considering that the contribution of color features and texture features to background segmentation is different, the method of constructing integrated Feature vector is as follow: ( y x T are color feature and texture feature after normalization, t w is texture feature weight. 100 representative QR code images on aluminum ingot and corresponding binary images processed manually were used to establish training atlas, and LMS (least squares) method was used for offline learning and training . It can get Clustering the Integrated features by K-means. The set Ω containing n samples can be expressed as follow: Define the center j C of the j-th class as follow: Where, j φ is the set of class j , and Use the Sum of the Squared Error (SSE) as objective function: The integrated features in Eq.17 are used as the features of pixels, the value of center points K in Kmeans is set to 2, and iterating clustering until SSE is stable. The pixels in first class are set to 0 and the pixels of second class are set to 1. The binary images ( Figure 5) are a clustering result. In order to solve the problem of image discontinuity result from wrong clustering to some pixels,we need to improve the binary images Fig.4b by morphological methods. Through many experiments we found that it is best to use square kernel with size of 15×15 to do morphological opening and closing operation for the image Fig.4b with size of 653×673. The final result after morphological operation is shown in Fig.6.

Experiment and statistics
The method proposed in the previous section is tested and verified. Pulsed fiber laser is used in experiment to direct ablation on the rough surface of aluminum ingots (AL99.70) to get 200 different QR code with size of 40mm*40mm, then acquire 400 QR Code images with size of 653X673 at different angles under natural light and flash light with OV9650 camera.
Then we use the reprocessing method proposed in this paper to process experimental samples, and select OTSU algorithm for comparison, table 3 shows the parts of experimental results, the recognition effect of the first part corresponds to OTSU recognition results, the second part corresponds to method proposed in this paper.× represents that it is unable to recognize,  represents that it is Recognizable.

 
According to the experimental statistics, the recognition rate after OTSU treatment is about 50%, and the recognition rate of this method is nearly 100% after processing. It is obvious that this method has an obvious advantage on the two-dimensional code image on dealing with rough and highly bright metal surface.

conclusion
Through theoretical analysis and experimental verification, we can draw the following conclusions: (a) Based on the theory of mixed Gauss model, the feasibility of Foreground Background Segmentation under multi-feature fusion method is verified.
(b) Through calculation and experiment, the recognition rate of QR code is nearly 100% when the center number k is 2, the texture feature weight t ω is 0.731, the size of the square core is 15×15, the actual size of the QR code is 40mm×40mm and the size of QR code image is 653×673.
(c) For the QR code recognition problems on rough and highly reflective metal surfaces, we propose an image processing algorithm based on multi-feature fusion and morphological optimization, and have a good recognition effect in the experiment, and it has a certain reference value to this class of metals whose surfaces are rough and highly reflective. Otherwise, this method has more advantages comparing with the OTSU algorithm