A Safety Evaluation-based Image Quality Method for Roads and Bridges

: A computer-aided automatic safety evaluation method is proposed based on quality evaluation on digital images of roads or bridges and other image information collected by highway monitoring devices. Images of qualified roads or bridges are selected to form a reference image database, and reference image sequence and evaluation image sequence are established separately. Then combined with the peak signal to noise ratio (PSNR) and the human visual characteristic information entropy, a safety evaluation function with dynamic weights is obtained. At last, the evaluating algorithm is used to compare similarities between evaluation images and reference images to judge the quality of roads or bridges and get a sequence of evaluation parameters sequence. If the value of the evaluation parameter is greater than the threshold, the road or bridge quality changes greatly, and therefore artificial inspection is required. The experimental results show that the evaluation is consistent with the subjective perception of human vision, and the method proposed in this paper has high degree of automation.


Introduction
As photography, remote transmission, and storage technologies develop, the digital image processing technology are combined more closely with road engineering and bridge monitoring [1], particularly in crack recognition and license plate recognition and positioning. Although there are numerous crack recognition algorithms, they are underperforming in continuity and integrity of crack images, and it is still a great challenge to identify complete fracture images rapidly and accurately. In addition, these algorithms are designed for 2D images and are easy to be affected by shadow, oil pollution and other factors [2,3].
In recent years, researchers tend to study the image quality evaluation based on human visual features [4][5][6], but the computation is complicated when being applied in evaluating images with complicated texture. In this regard, researchers put forward a variety of quality measurement methods. The MSSIM method [7] evaluates similarities between a distorted image and a reference image from luminance, contrast ratio, and structure. This method is characterized by simple calculation, but it is ineffective for blurred and seriously distorted images. IFC [8] and VIF [9] methods evaluate quality by calculating correlation of mutual information between distorted and reference images. Both methods have good performance, but wavelet decomposition brings complex calculation.
Subjective evaluation [10] is accurate and reasonable, but it is easily affected by the observer's competence, leading to instable results and difficulty to meet requirements of real-time processing [11]. The EBS algorithm [12] evaluates images' comprehensive quality using visual sensitivity characteristics of human eyes at the image edge and similarities between the image edge and background. The comprehensive quality of colored images is evaluated from the objective fidelity [13], visual distortion degree, and definition. The result shows that this method is as good as human eyes and is better than the traditional PSNR method. A comprehensive quality evaluation function for colored images is obtained through information entropy, average contrast, average grey degree, and key area standard deviation of human visual perception [14]. Evaluation parameters are clear, and the results are more in line with human subjective perception. In this paper, the image quality evaluation principle is used to conduct auxiliary safety assessment for roads or bridges and form a time quality evaluation sequence, providing data for effective maintenance management and performance prediction.

Time-series Reference Image Library
Images of a road or bridge that has been identified the best in quality are selected as reference images R. Pavement images are collected at a specified time interval. Assume that the image at the i th ( 1 i > ) moment is the image E i to be evaluated. By comparing it with a reference image i R , a quality evaluation parameter D i is obtained. The image taken at moment i is then used as the reference image N i at the (i+1) th moment.
There are three sets of time series in Figure. 1 Figure.3 Bridge monitoring images A sequence of road or bridge monitoring images is obtained using the method in Figure.1. As shown in Figure.2 and Figure.3, R represents reference images; E represents evaluation images, and the sampling interval is one month.

Correlation Evaluation Function
Multiple channels, luminance nonlinearity, visual attention mechanism, and masking effect are underlying psychological characteristics of the human visual system that have been found, and are regulated by the nervous system. In this paper, PSNR is combined with the information entropy, a human visual parameter, to serve as a measure index, and a correlation comparison method is put forward based on human visual characteristics. The correlation of E i and R i is compared objectively by using this method to obtain an evaluation parameter, which is used to illustrate the quality and safety of road or bridge pavement.
The information entropy can be calculated using formula (1). The higher the gray or chroma level is, the better the amount of information is, and vice versa.
In this formula, ( ) P i is the distribution probability of pixels on the i th gray level. If ( ) 0 The information entropy is the biggest when the image distribution is uniform, as shown in formula (2). When the gray or chroma is only one, it is defined as 0.
If lαβ color space is used as an example, the information entropy can be calculated using the following formula: InEn represents the information entropy of the whole image whole image. l InEn , InEn α , and InEn β are red, green, and blue information entropies, respectively.
The gray difference can be calculated via PSNR to evaluate the correlation between E i and R i , and the calculation formula of PSNR is shown in formula (4). E i and R i are decomposed in lαβ color space, and the quality measurements are obtained on three color components. This approach is characterized by simple calculation, but fails to take the structural relationship between pixels into account.
MSE is the mean square error between E i and R i . M N × represents image pixels. PSNR measures image quality by comparing gray difference between corresponding pixels of the image. In combination with InEn , the information entropy of correlation between Ei and R i is shown in formula (5).
Where, corr , en and ' en are correlation, evaluation image information entropy, and reference image information entropy. Information entropy is used to ensure the consistency of objective evaluation results with the subjective feeling. The correlation evaluation function based on visual characteristics is shown in formula (6). Where, cef is the quality evaluation function, D represents the evaluation parameter, p stands for the subvectors of the color space, which are l , α and β , respectively. a and b are weight coefficients, which can be adjusted dynamically according to the gray value or structural relationship characteristics of the neighboring image. Specific calculations are shown in formula (7)

Correlation evaluation process
The human visual characteristic correlation method based on the information entropy is used to evaluate the image quality. The smaller the evaluation parameter value D is, the higher the correlation, and the safer the road or bridge is. Figure. 4 illustrates the specific calculation flowchart.

Simulation Results
To Figure.2 and Figure.3, we apply PSNR, the method in literature [10], and the method in this paper for safety evaluation respectively. According to Figure.4, quality evaluation parameters are calculated using formula (6) and the results are shown in Table 1 and Table 3. Corresponding weights are shown in Table 2 and Table 4.

Experimental Analysis
In Table 1 and Table 3, the data on the diagonal are evaluation parameter sequences [D1, D2, D3] and [D1, D2, D3, D4] of the method in this paper. From the perspective of subjective judgment, the quality order of evaluation images in Figure.2 is E1, E2, and E3, and the sequence of evaluation parameters is [159.48, 162.41, 162.74]. The quality order of evaluation images in Figure.3 is E4, E5, E6, and E7, and the sequence of evaluation parameters is [71.64, 75.85, 76.44, 78.43]. According to the principle that the higher the value is, the higher the correlation is, and the better the quality becomes, the objective quality evaluation result is consistent with the subjective judgment, and the experiment result shows that the method is effective. E3 in Figure.2 and E5 in Figure.3 are points of manual inspection, and values of evaluation parameters R1 and R4 are the largest. For detailed data, see Table 1 and Table 3.
The PSNR method has no obvious difference in the result, so it is invalid. The evaluation results in literature [10] are consistent with those in this paper, but the computational cost of the former is more than four times of the latter. The distribution of dynamic weights a and b is related to the gray value and structural characteristics of the image. If the impact of the gray value is large, value a is larger. If the impact of structural characteristics is large, value b is larger.

Conclusions
To evaluate the safety for road or bridge by using images and with the help of computers, an auxiliary safety evaluation method based on reference images is proposed in this paper. An objective evaluation function and the calculation flowchart of visual information entropy are also obtained in combination with PSNR. The method is robust and highly automated, and complies with subjective feelings of human eyes.