Deep Learning for Intelligent Substation Device Infrared Fault Image Analysis

. As an important kind of data for device status evaluation, the increasing infrared image data in electrical system puts forward a new challenge to traditional manually processing mode. To overcome this problem, this paper proposes a feasible way to automatically process massive infrared fault images. We take advantage of the imaging characteristics of infrared fault images and detect fault regions together with its belonging device part by our proposed algorithm, which first segment images into superpixels, and then adopt the state-of-the-art convolutional and recursive neural network for intelligent object recognition. In the experiment, we compare several unsupervised pre-training methods considering the importance of a pre-train procedure, and discuss the proper parameters for the proposed network. The experimental results show the good performance of our algorithm, and its efficiency for infrared analysis.


Introduction
Infrared fault image is an important way to evaluate the situation of substation device.According to previous study, some fault of substation device can be diagnosed by analysing its infrared image.Currently, infrared fault image analysis is mainly based on a manual way, which has high requirements for the professional quality of an operator.To improve the efficiency, studies turn to automatic extraction and analysis of the infrared images by artificial intelligent ways.
Currently, main manufacturers of thermal imaging cameras such as FLIR Systems Inc. provide some closed tools to show their specific data.These tools help the operators to get internal information of the thermal image.It seems that analyse the thermal information in signal ways is possible.Unfortunately, without the specific port and for other manufacturers, the tools cannot be used.
Most studies adopted basic imaging processing algorithms and did not make further analysis.Considering the difficulty to get internal data and drawback of previous woks, we turn to automatically detect the fault region and recognize its belonging device part in a novel image processing way.The former is an image segmentation process which takes advantage of superpixel segmentation method in this paper.Different from other segmentation methods aiming at edges or textures, a superpixel one emphasizes the homogeneous property of small superpixel regions.The latter is a typical pattern recognition problem which is applied by a machine learning method through exacted features.Different from typical feature description methods for visual images include SIFT [1], SURF [2], HOG [3] et al, we prefer to the deep learning method [4].The popular deep learning methods include deep belief networks (DBN) [5], convolutional neural network (CNN) [6], deconvolutional network (DNN) [7]and other algorithms [8][9].With the concept local receptive field, CNN method is more suitable for image classification.The performance of CNN for image recognition has already be proved by several practical applications [10][11][12].
Hence, the first contribution of our work is taking advantage of the imaging characteristics for infrared image segmentation.The use of superpixel segmentation makes the method suitable for various shapes of fault region.Our second contribution is incorporating the deep learning algorithms into unsupervised device part recognition.Not considering the selection of feature description, this way makes the classification result more reliable and robust.

Fault Region Detection
An infrared fault image generated from a thermal imaging camera is created from the temperature matrix and the given palette.The image can be divided into three parts: a) the background region; b) the device region; c) the fault region.Generally, the fault region is inside the device region and of great saliency in contrast to its neighbors.Different from the traditional visible light image, structured features such as textures, labels are lost in a regular infrared thermal image.Besides, pixels inside each kind of region are homogeneous with similar properties.Therefore, we adopt the concept of superpixel segmentation.The original infrared image is firstly divided into several homogeneous regions.The insight property of the fault region is further used to detect the demanded region.

Fault Region Segmentation
Superpixel algorithms group pixels into perceptually meaningful atomic regions, which can be used to replace the rigid structure of the pixel grid.We here choose three popular superpixel algorithms for compare: Turbopixel [13], SLIC [14] and Entropy rate superpixel (ERS) [15].The segmentation results of a sample image are shown in Figure 1.We can see that SLIC and ERS algorithms are not suitable for our demands with their irregular boundaries.The regular lattice generated by Turbopixel maybe more suitable.The segmented regions are labeled as Ri; i = 1; 2; … n, n is the predefined segmentation number.There are several color palettes in a thermal imaging camera for the operator to select, such as Ironbow, BlueRed, Grayscale, Amber.Among these palettes, the Ironbow palette is proved to be suitable for the infrared fault analysis procedure.Based on this commonly used palette, we further choose the HSV color space for fault region detection.The focused fault region is more often displayed with nearly yellow and white color, which is: Here, , , r g b are three channels in the RGB color space.To highlight these regions, we modify the standard hue value(H) equation as shown in (2).With (2), each R i can be represented by its regional H value where taking the mean value [ , , ] r g b of the region's color as the regional color value as shown in Figure 3

Fault and Device Region Validation
As mentioned above, the device region is the main object in the image.To diagnose the fault, we should know the location where the fault occurs and its belonging device part.We here choose three properties to find the most likely fault region: Average brightness i R I : the average gray value of R i ;Solidity i R S : the ratio between the region's area and area of its minimum convex hull; Center distance i R d : the distance from the image center to the region.
By maximizing the objective function (3), the fault region can be extracted as shown in Figure 3(c), and labelled as R s .max / The binarized images are shown in Figure 3 (d).As the fault region is inside the device region, the approximate device fault part can be greedily searched by taking the fault region as the searching center as shown in Figure 3 (e).

Deep Learning for Device Part Recognition
After the fault and device region are extracted, we should further recognize the type of the belonging device part.The type of the device part could show the most possible causes of heating and other properties.Here, we turn to choose the state-of-the-art deep learning algorithms for device parts recognition.
As shown in Figure 2, we change the network in [16] called a convolutional-recursive network (CRNN).The network is combined by two stages.The first stage is a CNN-layer with convolution and subsampling, while the second stage is a multi-layer RNN(recursive neural network) [9].The features generated by the last two RNN layers are fully-connected to a softmax classifier.

Pre-train
Before trained by a CRNN network, the filters for convolution should better first pre-trained by unsupervised learning algorithms as discussed in [17].As mentioned in [18] and [19], they found out that K-means algorithm attains best performance with only one parameter to tune.On the other hand, [20] proposed a new feature learning algorithm called sparse filtering.In this paper, we choose Kmeans, Sparse Filtering and RICA [21]

Training
The main idea of CNNs is to convolve filters over the input image in order to extract features.First of all, we resize all the input samples to size p p u .Then convolve each sample with 1 K filters of size r r u .When the convolutional stride is s, the output feature size is K p p u u .In the RNN stage, all features in one layer are connected with features in the next layer.Each layer shares the same weights, and the feature size is gradually reduced until the final response size of each feature map is1 1 u .Assuming the size of local receptive field is l l u , the number of RNNs in each layer is 2 K .The size of final feature of each sample when combing the last two layers' output is

Experiments
The performance of the analysis result mainly depends on the fault region detection, device part recognition with known device type.We have collected a large amount of infrared samples containing various device parts for evaluation.

Image Segmentation
Three images in which faults occur in different part to display the segmentation process are shown in

Parameters Comparison
In this section, we take the analysis of a current transformer for example.According to controllable appearance and fault, the current transformer is divided into three parts: joint, inner join and bushing.Each class has 3000 samples for train, and the total number is 9000.1500 images are chosen for test.with enough samples of different part appearance, the classification algorithm could get comparable performance for other kind of devices such as voltage transformer, arrester and so on.According to Sect.3, we here define the sample size as 67 67 u , the filter size as 8 8 u with stride 1.After convolution, the output feature map size is 60 60 u .With pooling stride 2, the feature map size after the 1st stage is 27 27 u .We take the receptive field in RNN stage as 3 3 u , after 3 layers the size becomes 1 1 u .With the defined multi-layer feature output, the final feature dimension generated by each sample is 2 1 10 K K u u .
When taking 1 K as 64, the optimized filters of the three pre-train algorithms can be shown in Figure 4.In this process, when taking advantage of sparse filtering, we find the objective function is hard to converge.The filters generated by sparse filtering are a little chaotic.Therefore, our pre-train method and parameters decision is focus on RICA and K-means. 1 K is changed from 16 to 128 with step 4. The classification precision is shown in Fig. 5. Then fix 1 K as 64, change 2 K from 16 to 128 with step 8, the result is shown in Fig. 5. Similarly, we also find the performance of K-means is much better than that of RICA.When 1 K is larger than 64 and 2 K larger than 64, the number of filters and RNN does not influence much.Therefore, we take the parameters with 1 2 64, 64 K K .

Discussion and Future Work
Our processing object is a purely infrared image generated by a thermal imaging camera without knowing its original temperature matrix.In this case, after region detection and recognition, we could know the area of fault region, its belonging device part, its location in the device and the temperature rise and difference in pseudo-color distance.The recognition of the temperature label is a typical digit recognition problem and already has mature solutions.With the recognized temperature range, we could get the precise temperature distribution by the look-up table of the selected Ironbow palette.This demand requires further study and is not the main purpose in this paper which would be realized in our future work.Meanwhile, further and deeply analysis would be practical based on the results of this paper.In this paper, a novel method for intelligent infrared fault image analysis of substation device is presented.We first use a Turbopixel segmentation method to approximately segment an infrared image into multiple homogeneous regions.The regions are further checked by their hue property based on the most commonly used palette.To deeply analyze the fault, its belonging device part is classified by the CRNN network, which is the key point of the automatic analysis.From the experimental results, the algorithm of this paper is of high accuracy, and can be adaptable to infrared analysis of different substation devices.Due to the lake of temperature range, we do not make further statistics in this paper.And this is our aim of further study to get an entire automatic study of infrared fault image analysis for substation device.

Figure 1 .
Figure 1.Results of Different Superpixel Segmentation Algorithms.(a)(d) are the results of Turbopixel, (b)(e) are the results of SLIC and (c)(f) are the results of ERS.
(b).With threshold H > 0.65 the binary image can be further obtained.

Figure 2 .
Figure 2. Convolutional-Recursive Network algorithms for comparison.The comparison results are shown in the experiments part.

Figure 3 .Figure 3 .
Here, (a) is the input infrared images, (b) is the H value gray image, (c) is the validated fault region, (d) is the binarized image for device region, and (e) is the detected fault region and its located device part.The solid box represents the bounding box of fault region, and the dashed box stands for the device part needed to be classified in the deep learning step.The superpixel segmentation ensures pixels inside homogenous region to be smooth, contrast between fault region and the background region are strong enough in the H value gray image.Therefore, the fault region and the device region are easy to be extracted.From the results, we can see that the segmentation method is applicable for fault with different locations, shapes and sizes.Image segmentation Procedure.(a) is the input infrared image, (b) is the H value gray image, (c) is the validated fault region, (d) is the binarized image for device region, and (e) is the detected fault region and its located device part.

Figure 4 .Figure 5 .
Figure 4. Pre-train Results.(a) is the filter from sparse filtering, (b) is the filter generated by RICA, (c) is generated by K-means.