Abnormal Appearance Detection of Substation Based on Image Comparison

Based on image comparison, a novel algorithm for abnormal appearance detection of substation is proposed. Previous spatial states of an object are compared to its current representation in a digital image. Firstly, saliency maps are acquired using a fast implementation method of salient region detection. Based on saliency maps, image registration was completed by ORB (Oriented Fast and Rotated Brief). Then, sliding widow algorithm is applied to transform the whole image comparison problem into sub-image comparison problem. Textural feature and shape feature vectors (TSFVs) representing contents of images are generated by feature level fusion. Finally, decisions are automatically made as to whether or not change at the outline has occurred by the Euclidean distance of TEFVs. Experimental results show that the proposed method has good performance in abnormal appearance detection


Introduction
Inspection of electric power substation equipment plays an important role in daily maintenance of the electric power system.In recent years, robotics applications have attracted an increasing amount of attention in many research areas, especially in the construction, inspection, and maintenance of infrastructures and equipments in the power industry.Zhou et al. described the development and application of intelligent substation inspection robots, and pointed out that instrument recognition is important function of the substation inspection robots [1].Currently, many works have been down in abnormal temperature detection based on infrared cameras, and less work for abnormal appearance detection using visible-light images.Most of equipment failure is mainly carried out by workers through observing pictures collected by cameras.Compared to other industries, research on image analysis and fault recognition of electrical equipments within substation was undeveloped.Li et al. proposed an image processing approach for abnormal appearance of electric power equipment using Mean Shift segment method [2].A template matching algorithm based on the shape characteristics of image analysis is proposed to apply in the image recognition of power transformer [3].But these methods obtain high precision only in the case that image matching achieves accuracy up to pixels.Meanwhile, different templates are needed to make artificially for different electrical equipments.To overcome above shortcomings of these algorithms, a new algorithm is proposed to detect different types of equipment failure according to the routinely inspection way of robots in the substation.
By automatic control and remote control means, substation equipment inspection robots developed by our laboratory, shown in Fig. 1, can take infrared thermal imager and visible CCD to inspect the outdoor highvoltage equipments instead of human.While the intelligent robot is automatically moving inside the substation, it stops at the predefined stop point and turns the pan-tilt to acquire images at a fixed angle.Aiming at current image taken by camera, the other picture which has the same content as current one can be found in the database of last inspection.If image change is detected in multiple images of the same scene taken at different times, there is strong possibility that instruments abnormality occurred in this region.

Figure 1. Inspection robot developed by Electric Power
Robotics laboratory of SGCC.Hence, a new method suitable to detect abnormal appearance of substation is proposed in this paper based on images comparison.Fig. 2 shows a flow chart of our approach to implement abnormal appearance detection of electrical equipment within substation.First, saliency maps of two images with the same scene taken at different times T 1 and T 2 were obtained by local salient region detection algorithm.Then, image registration of salient maps was implemented based on ORB features.Sliding window algorithm divides the whole image into many blocks.Textual features and shape features are extracted respectively and fused at feature level to represent the content of blocks.Finally, we detect abnormal appearance through calculating Euclidean distance of fusion features of images participated in comparing.

Image Registrations
Scale invariant feature transform (SIFT) algorithm is widely used in image registration because of its invariance to image scale and rotation [4].However, this method still needs to be improved, because threshold values are excess and uncertain, and computation is complex due to the high dimension of descriptor.A new descriptor named ORB with the same matching performance as SIFT and lower computation cost was proposed by Rublee et al. [5].Here, image alignment is implemented based on ORB features extracted from saliency maps of two comparing images instead of SIFT.

Salient Region Detection
Salient region detection is useful in applications such as image segmentation [6], object detection and recognition [7], and region-based retrieval [8].Salient regions are usually identified as those regions of an image that are visually more conspicuous by virtue of their contrast with respect to surrounding regions.Computationally detecting such salient image regions remains a significant goal, as it allows preferential allocation of computational resources in image analysis.In order to reduce time consuming and remove the uninformative regions of inspection images, salient region detection algorithm is applied here to separate foreground before image registration.
Achanta et al. proposed a method for finding salient regions using low-level features of luminance and colour [9].Saliency is determined as the local contrast of an image region with respect to its neighborhood at various scales.The method is fast, easy to implement and generates high quality saliency maps of the same size and resolution as the input image.Hence, this algorithm is used here to get saliency maps of images shot by inspection robots.The specific step of salient region detection is detailed in paper [9].Some results are shown in Fig. 3.

Region of Interest
ORB is proposed as a very fast binary invariant descriptor based on BRIEF, it greatly speeds up processing procedure for feature extraction and matching compared to traditional key point descriptor, such as SIFT and speeded up robust features (SURF) [10].Ref. [5] pointed out ORB is an order of magnitude faster than SURF, and over two orders fast than SIFT.Here, ORB algorithm is used to detect feature points and build descriptors of two comparison saliency maps.Then, RANSAC (random sample consensus) processing via transformation parameters estimation is employed to remove possible wrong matching points [11].Transform matrix R is finally calculated by matching key points.
After getting transform matrix R, two corresponding images are finally converted into the same coordinate system.Four vertexes coordinate of current image mapped to the coordinate system of previous image.Four vertexes coordinate after space transformation can be expressed as top-left (x 1 , y 1 ), top-right (x 2 , y 2 ), bottomleft (x 3 , y 3 ), bottom-right (x 4 , y 4 ).According to the four vertexes coordinate, boundary coordinate of the regions of interest (ROI) are defined as:

Image Comparison
Detecting regions of change in images of the same scene taken at different times is of widespread interest due to a large number of applications in diverse disciplines, especially remote sensing methodologies [12] [13].Here, abnormal appearance detection of equipment within substation, including appearance or disappearance of objects, or shape changes of objects, is implemented through change detection of two images taken at different inspection times.This is an image comparison problem, where an object is compared to itself, to identify whether or not it has changed.
For images comparison, the decision as to whether or not a change has occurred corresponds to feature difference.Aiming at images with rich contents, single feature has limitation in representing image content.In order to overcome the shortcoming of the low efficiency of previous single feature representing, a feature level fusion approach is proposed to implement a fusion strategy of textural features and shape features for change detection.

Salient Region Detection
The sub-image covered by the sliding window is divided into different overlapping blocks with the same size.Each block has 100 100 pixels.Step size of sliding window is 50 pixels so that adjacent blocks have 50% overlaps.The region with abnormal appearance may be divided into different blocks by sliding widow method with no overlap.If this is the case, change in image would be shared by different blocks.From the segmentation results in Fig. 4, we can see that overlapped block can avoid the abnormal appearance region divided into different blocks.There are at least one block including most of the area of abnormal appearance.

Moment Invariants Features
Moment has been widely used in pattern recognition to describe the textural characteristics of different objects, and its calculation is related to the area, center, and breadth axel of the image or region.Invariant moment containing the global information of images reduce the sensitivity to the quality of pattern extraction.Hu proposed 2D moment invariants theory [14], and deduced seven moment invariants, which are of scale invariance, translation invariance and rotation invariance in continuous condition.In our paper, an improved computation technique [15] is adopted to make moment feature suitable to describe the texture feature of image.Then, we can obtain a 1×35 feature vector to represent the textual information of a sub-image.

Histogram of Gradient
Histogram of gradient (HOG) has been widely used for analyzing shape information, and has been demonstrated that it was powerful in object recognition and image classification [16].In practice, each sub-image is divided to 25×25 pixel cells.For pixels falling into the same cell, the 9 bin histograms are summed up.Then each block contains a 36 dimension histogram vector of concatenated 9 orientation bins from 2×2 cells.In order to reduce time consuming and information redundant, blocks in HOG extracting process are taken with no overlap.Final a 1×144 dimensions shape feature descriptor is obtained.

Feature Level Fusion
Canonical correlation analysis (CCA) as a statistical tool has been proposed for identifying linear relationships between two sets of variables [17].Given a pair of meannormalized feature sets U= [u 1 , ,u n ], V=[v 1 , ,v n ], u i R p , v i R q , i=1, ,n.Here U, V represents textural and shape feature codes respectively.In general, CCA can be seen as the problem of finding basis vectors w u and w v for two sets of variables U and V such that the correlations of projections of the variables onto corresponding basis vectors are maximized, while the projected variates within each data set are uncorrelated.More formally, CCA can be described as , ( , ) arg max Solving ( 2) in an optimization manner, the vector pairs (w ui ,w vi ), i=1, ,d (d min(p,q)), can be obtained.More details about derivation and solution of CCA can be found in [18].Let W u = [w u1 , ,w ud ], W v =[w v1 , ,w vd ], the feature fusion strategy is defined as 0 0 While Z is called a textual-shape feature vector (TSFV).In general, shape features achieve higher efficiency comparing to moment features in change detection.To improve the discriminability of the fused feature vectors, feature fusion strategy here are modified as where , are the weights assigned to the two different features.Certainly, the values of parameters and ( < ) can be determined by defining some learning strategies if the prior information about features is unknown.Convincingly, Z w (called WTSFV) is better than Z in representing the properties of image.Final features Z 1 and Z 2 of two comparing sub-images are obtained.Finally, change detection is completed using Euclidean distance of Z 1 and Z 2 compared to a prior threshold value .Regions detected with change are labelled with colour boxes ultimately.

Experiments
We collected our own database to evaluate the performance of the proposed algorithm.The collected database contains two sections: locale images of substation photographed by inspection robots (Data-I) and man-made images with foreign bodies photographed in the laboratory environment (Data-II).Data-I has 320 pairs of images taken by different inspecting times and Data-II has 4 pairs of images taken by digital camera in different experimental environments.Every image in Data-II is resized as the same size 704×576 as images in Data-I.
From the results of image registration as shown in Fig. 5, we can see obviously fewer matching points detected in saliency maps.173 pairs of matching points were detected based on original images and the whole image registration processing is completed in 2.37 seconds.However, 57 pairs of matching points were detected based on saliency maps and the whole image registration processing is completed in 1.75 seconds including the time consuming of salient regions detection.The time cost of points matching and RANSAC processing based on saliency maps is fewer than that of original images.The same results were presented in other images registration.In order to evaluate the performance of abnormal appearance detection, a group of man-made abnormal images were acquired by camera in different scenes.The experimental results shown in Fig. 6 show that every abnormal region was detected successfully.Hence, using the proposed algorithm to implement images comparison has a high capability in abnormal appearance detection of substation.Besides, the proposed approach combined with autonomous robotic system improves the efficiency and reliability of substation inspection.

Conclusions
A new abnormal appearance detection of substation based on image comparison has been proposed in this paper.In order to reduce the influence of background and time consumption, image registration is implemented based on ORB features extracted from saliency maps.Then, sub-images were segmented from the whole image using sliding window algorithm.Moment invariants feature and HOG is extracted to describe the textural and shape information of sub-image respectively.Finally, Change detection based on fused features is implemented to detect abnormal appearance of substation.Experimental results have shown that the proposed method performed well in abnormal appearance detection of robots inspection in substation.

Figure 2 .
Figure 2. Flow chart of the proposed method.

Figure 5 .( 5 )
Figure 5. Feature points matching: (a) original images; (b)saliency maps Data-I contains only one image with abnormal appearance.This abnormal region with foreign body was detected by our method as shown in the last column of Fig.6.Except this, the other fault regions detected by image comparison in Data-I are false alarm.This database is exploited to evaluate false positive rate (FPR) of our algorithm.The FPR is defined as,

Figure 6 .
Figure 6.Given input images taken at time T1 (top); Given input images taken at time T2 (middle); Results of abnormal appearance detection by our algorithm (bottom).
where w and h respectively represents width and height of previous image.Then ROIs of two images with the same size are extracted according to top-left corner ROI left .x,ROI up .y)and bottom-right corner coordinates (ROI right .x,ROI bottom .y) of the rectangle.