Detection for Deformed and Sheltered Circular Traffic Signs

In this paper, a simple and efficient algorithm for detecting deformed and sheltered circular traffic signs under the complex natural scenes is proposed. First, the image is segmented by adaptive hybrid gradation method. Then the circular traffic sign will be detected by an enhanced random eclipse fitting method. The experimental results show the detection rate reaches 96.37%, and the proposed algorithm can succeed to detect most of circular traffic signs in complex conditions and handle the adverse influence of traffic signs’ deformation occlusion and the background interference whose colors are as same as traffic signs’ well in particular.


INTRODUCTION
With the popularity of cars in modern society, security issues of road vehicles have become increasingly prominent.Driver assistance systems as a part of in-telligent transportation systems have become a hot topic of national research institutions in order to solve the traffic congestion and other traffic safety issues.
Most traffic signs' colors are red, yellow and blue, and most traffic signs' shapes are circular, triangular and rectangular.Because traffic signs have these dis-tinctive features, they can be detected by some color and shape segmentation algorithms.
Different methods about color and shape segmenta-tion have been proposed.On the color segmentation: Asakura [1] proposed simple vector filter (SVF) which can extract specified color pixels with high speed.Huang Zhiyong and Sun Guangmin [2] used fixed thresholds of RGB substruction to extract red and blue pixels.Kehtarnavaz [3] transferred RGB model to HIS model to detect prohibition signs.On the shape seg-mentation: Hough transform [4] is widely used for circular road signs detection as a classical method, but it has poor quality of real-time performance.Hough gradient method [5] improves the real-time performance, but the algorithm needs many thresholds which should be manually set.In recent years, there are also algorithms for circular signs detection based on freeman chain [6] and eclipse fitting method [7].These detection methods have a good real-time performance, but the results of detection for deformed or sheltered signs are still unsatisfactory.In this paper, a new color segmentation algorithm called adaptive hybrid gradation method which can adapt to more different lighting conditions is proposed and a new shape segmentation algorithms called enhanced random eclipse fitting method is also proposed to strengthen the detection for sheltered and deformed circular traffic signs.

Chromatic aberration of three color-components [8] (CATCC)
As traffic signs have special colors, most unrelated pixels in the image can be removed by certain algo-rithms.Hu Mudan proposed an algorithm called chromatic aberration of three color-components.Steps of the algorithm are as follows: First, transfer a color image to a gray-scaled image of chromatic aberration.The conversion relationship is as formula (1) and (2), in these formulas, CA_red and CA_blue represent the pixel value of red and blue gray-scaled image of chromatic aberration.Choose O >1 to highlight red, yellow and blue components.The value of O is set to 1.6.
After that, segmentation thresholds TH1 and TH2 of red and blue gray-scaled images of chromatic aberration are calculated by Otsu algorithm [9].Then the red and blue gray-scaled image of chromatic aberration is segmented by threshold TH1 and TH2.The segmentation relationship is as formula (3) and (4).ABSTRACT: In this paper, a simple and efficient algorithm for detecting deformed and sheltered circular traffic signs under the complex natural scenes is proposed.First, the image is segmented by adaptive hybrid gradation method.Then the circular traffic sign will be detected by an enhanced random eclipse fitting method.The experimental results show the detection rate reaches 96.37%, and the proposed algorithm can succeed to detect most of circular traffic signs in complex conditions and handle the adverse influence of traffic signs' deformation occlusion and the background interference whose colors are as same as traffic signs' well in particular.Finally, the color image is segmented according to formula (5).CA_color represents the pixel value in the color image.

Keywords
The experimental results show that this algorithm has a good effect on image segmentation and also has a good real-time performance.However, after a series of experiments, it is found that this algorithm is unsatisfactory when the weather is a fog day or the traffic signs have partly fade.Also, if the background's colors are similar to the traffic signs' colors, this algorithm cannot extract the traffic signs from the image.

Adaptive hybrid gradation method
In order to make this algorithm adaptive to more complex illumination conditions.Adaptive hybrid gradation method based on the chromatic aberration of three color-components is proposed.
First, the value of O should be dynamically changed according to illumination conditions.Define the parameter K0.K0 represents the ratio of background pixels' number and the total number of the red and blue gray-scaled image of chromatic aberration.Simultaneously, in order to remove background interference of the large area in the color image after segmentation (marked as img_dst)), The red and blue gray-scaled image after Otsu threshold segmentation ( marked as img_red and img_blue) should be detected for connected regions.Assume traffic signs may existin these areas of connectivity.Specific steps of adaptive hybrid gradation method are as follows: (1) Set O is 1.6, and transfer the origin color im- age to the red and blue gray-scaled images of chromatic aberration by formula (1) and ( 2).
(2) Calculate K0 by formula (6). ( recalculate the red and blue gray-scaled image of chromatic aberration, and go to step (2), or go to step (4).(4) Calculate the segmentation thresholds TH1 and TH2 of the red and blue gray-scaled image of chromatic aberration by Otsu algorithm.
(5) Segment the red and blue gray-scaled image by formula (3), (4).Then Segment the origin color image by formula (5).( 6) Detect all the connected [10] regions of img_red and img_blue to find out those regions whose area is larger than 7% of img_dst.Mark these regions as big_area(if the area of these regions is all less than 7% of img_dst, the algorithm ends ).Convert the format of pixels in the big_area from RGB to HSV.Calculate average saturation of each big_area (marked as S_ave)).Set the value of a pixel in big_area to zero if the pixel's saturation is less than 1.2* S_ave, or keep the pixel's value.
Most background pixels are removed from the color image by adaptive hybrid gradation method.Next, extract all the contours from the color image after segmentation.

ENHANCED RANDOM ECLIPSE FITTING METHOD
In this paper, enhanced random eclipse fitting method which is based on random ellipse fitting method [11] is proposed to extract the contours which are circular.

The expression of the general equation of an ellipse
In the two-dimensional coordinate plane, ellipse equations can be expressed by formula (7), ( 8) and (9).Inequality (8) ensures the curve is elliptic, but not hyperbolic or parabolic.Formula ( 9) can make the solution of the equations be nonzero and ensure that any integer multiple of a solution is the same solution.
First, put n points on a contours into formula (11), calculate the value of A, B, C, D, E, F which can make the value of ET smallest.According to Extreme Value Theorem, there must exist a relationship equation as a formula (12) in order to make ET smallest.

Web of Conferences MATEC
Next solve linear equations ( 12) and ( 9) to get the value of A, B, C, D, E, F. Put the value of A, B, C into inequality (8) to find out whether it is satisfied.If the inequality is satisfied, the contour will be considered as an ellipse.

Random ellipse fitting method based on the least square method( REFLS)
The difference between ellipse fitting method and random ellipse fitting method is that ellipse fitting method extracts all the points on a contour to fit an ellipse and random ellipse fitting method extracts only several points randomly( the number of extracted points should be greater than 6) .The steps of random ellipse fitting method are as follows: (1)Select six points on a contour randomly, and then put them into linear equations ( 12) and ( 9).
(2)Calculate the value of A, B, C, D, E, F to get a fitting eclipse.
(3)Put each point on the contour into formula (10) to calculate the value of eti.If eti > TH3*Seq_length( TH3 is a threshold and is generally set between 0.05 and 0.1, Seq_length represents the length of a contour) , this point will be considered as a matching point on the fitting ellipse.Count the number of matching points (marked as numcomp) and the total number of the contour (marked as numtotal), and then calculate the ratio of matching points accounting for all points on the contour (marked as ratio).ra-tio=numcomp / numtotal.
(5)If ratio> 0.3 and inequality (8) is satisfied, the contour is considered as a real ellipse, or it is not an ellipse.

Enhanced eclipse fitting method (EEFLS)
It is found that the performance of random ellipse fitting method is better than ellipse fitting method, but it still cannot detect all the elliptical contours because some contours are seriously incomplete.As is shown in Figure 1( a) , ( b) , ( c) , a circular traffic sign is sheltered by a tree, so the area of the traffic sign after segmentation is incomplete and the contour of this traffic sign is no more an ellipse but an irregular contour which can be hardly detected.Figure 1 (d) shows the result after image thinning.It can be seen there exists an elliptical contour.
This paper proposes enhanced random eclipse fitting method to extract the contours of all the circular traffic signs as far as possible.The steps of enhanced random eclipse fitting method are as follows: (1)Segment a color image by adaptive hybrid gradation method.Convert the color image after segmentation to a binarized image.The value of all non-zero pixels will be set to 255.Then do an opening operation [12] on the binary image.After that, extract all the contours from the binary image.
(2)Check each extracted contour to determine if it can be fit to an ellipse by random ellipse fitting method.If so, set the value of pixels on the contour and inside the contour to zero in the binary image.
(3)Use a new improved fast parallel thinning algorithm [13] to process the binary image.There will be only single-pixel-width skeletons left.Remove the glitch of the skeletons.
(4)Extract all the skeletons in the binary image.Check every extracted skeleton if it can be fit to an ellipse by random ellipse fitting method (here, a skeleton can be regarded as a contour) again.
(5)If a contour or a skeleton is successfully fit to an ellipse in step ( 2) and ( 4), calculate the ratio of the long axis and short axis of the ellipse.If the ratio is less than 3, the corresponding contour or skeleton will be considered as a contour of circular traffic sign.

EXPERIMENTAL RESULTS AND ANALYSIS
The experimental images of this paper come from real natural road scenes with a variety of illumination, background interference, sign's deformation and occlusion by manual shooting.

Experiment one: the comparison of 4 kinds of segmentation algorithms
The first experiment makes comparisons between the adaptive hybrid gradation method, chromatic aberration of three color-components, HSV and SVF.From the results of the first and second group of experiments, it can be found the traffic sign can only be fully segmented by adaptive hybrid gradation method when there exists background interference of large size.
From the results of the third and fourth group of experiments, it can be found adaptive hybrid gradation method can remove more background pixels than SVF and keep the value of pixels of traffic signs in better condition than HSV and chromatic aberration of three color-components simultaneously when the traffic sign fades or the lighting condition is not good.So it is clear that this improved segmentation method in this paper can better deal with the problem of background interference and is adaptive to poor lighting conditions.

Experiment two: the comparison of four kinds of contour segmentation algorithms
In the second experiment, the proposed contour segmentation algorithm of enhanced eclipse fitting method (EEFLS) will be compared with Hough gradient method (HG), ellipse fitting algorithm based on the least square method (EFLS) and random ellipse fitting algorithm based on the least square method (REFLS) on detection results and real-time  From the first and fourth groups of experiments, it can be seen that Hough gradient method, ellipse fitting algorithm and random ellipse fitting algorithm cannot detect the traffic signs when they are sheltered.From the second and third groups of experiments, it can been seen that Hough gradient method is likely to cause missing detection( the second group) and error detection( the third group), ellipse fitting algorithm is likely to cause missing detection( the third group) and error detection( the second group) and random ellipse fitting algorithm is like to cause missing detection( the second and third groups) .
Table 1. is a statistical experimental results of 4 algorithms further.It can been seen from the experimental data in Table 1 that the comprehensive detection rate of enhanced eclipse fitting method is highest, and its error detection rate is also lowest, especially for the deformed and sheltered signs.However, the real-time performance of enhanced eclipse fitting method needs to be improved.This algorithm cannot detect traffic signs for some special complex scenes including: (1) dense fog weather, (2) seriously faded traffic signs, and (3) the overlap of several traffic signs.The failed detection for scene (1) and scene ( 2) is as traffic signs almost lost its distinctive characteristics of red, blue or yellow, any algorithm of color segmentation is unable to deal with such kind of cases and cannot segment these traffic out.The failed detection for scene (3) is as enhanced eclipse fitting algorithm only judge if a contour is a circular traffic sign but cannot determine how many circular traffic signs the contour contains.

CONCLUSION
This paper makes two contributions for traffic detection on the basis of other people's previous work: (1) Propose adaptive hybrid gradation method based on chromatic aberration of three color-components, change the fix weight which is used to enhance color segmentation to an mutative weight which changes with illumination changes.The improved algorithm can better segment traffic signs under poor lighting conditions than other color segmentation algorithms and can deal with the background interference well by using 3 mixing grayscales including red grayscale, blue grayscale and saturation grayscale.(2) Propose enhanced eclipse fitting algorithm to strengthen the detection for deformed and sheltered traffic signs and increase the detection rate for circular traffic signs.
As the environment of traffic signs is a natural out-door scene, the effect of detection would be influenced by different illumination.So the main direction of algorithm improvement should be focused on how to make the algorithm adapt to these various illumination scenes.In addition, the real-time performance of the algorithm proposed in this paper needs to be improved.
CA_red0 and CA_blue0 respectively represent the number of background pixels of the red and blue gray-scaled image of chromatic aberration.M represents the width of the image and N represents the height of the image.The circumstance that K0 is too large reflects the value of most pixels' red, blue component of the origin image is too low.The value of O should be increased to highlight the red and blue component.

2
Ellipse fitting method based on the least square method (EFLS)Define the variables eti and ET.eti represents fitting error of each point on a contour and ET represents the fitting error of all the points on a contour.The expression of eti and ET are respectively shown as formula(10) and(11).

Figure 4 .
Figure 4. Comparison of four kinds of segmentation algorithms under fog and haze day.