Anomaly Detection of Electro-data Based on Deep Convolutional Neural Network

: As diversity of electro-data anomaly, the methods based on artificial feature are becoming more difficult to detect anomalies among a great deal of electro-data. Hence, this paper proposes a novel method which is based on deep convolutional neural network (DCNN) to detect anomaly electro-data. This method models the sample data with time information and electrical parameters, and labels them as normal or abnormal automatically. Further, the paper improves the designing DCNN to extract precise features from large scale of electro-data to get high accuracy. The results of the case analysis show that our method can detect anomaly electro-data more exact and stable than the traditional methods. The abnormal precision rate and abnormal recall rate of our approach reach 92.7% and 91.3% respectively.


INTRODUCTION
With the development of smart grid, State Grid Corporation of China has deployed about 500 million smart meters.Electricity data collection system could collecte a great deal of electro-data from smart meters.Among such mass electro-data, it's hard to discover anomaly caused by device failure or power stealing through artificial way.Hence, a lot of mechine learning based methods about anomaly detection of electro-data had been proposed.Generally, those approaches could be classified into supervised and unsupervised learning: 1) Unsupervised learning based methods: they utilize unlabeled data to extract the internal features and discover anomoly, such as Cheng(2015), Wang(2016) and Ramos(2016).The effect of these methods depends on preconfigured parameter, and they have high time complexity and less scalablity.2) Supervised learning based methods: they attempt to find the inherent correlation between the feature and the label to train a classifer as detector, such as Fan(2013), Jindal(2016) and Yang(2016).As the models of this category are pretrained, the time complexity is lower when check anomaly data.However, the ensemble performance is strongly dependend on artificial feature model.Hence, this paper attempts to extract more precise and scalable features through deep learning to improve the arruracy of anomaly detection.Deep learning has self-learning ability to extract advanced features from raw data, and classifies data accurately.So far, many well-known deep neural networks such as GoogleNet (Szegedy, 2015), VGGNet (Simonyan, 2014), ResNet (He, 2016) have obtained better results through deepening neual network.These deep convolutional neural networks(DCNN) are expert in the areas of computer vision, speech recognition, however, detecting anomaly electro-data is much different from it.Hence, there are still many challenges on applying DCNN for detecting anomaly electro-data.Firstly，electro-data is different from image or audio and it is consist oftime-series data, such as voltage and current.Secondary, the image and audio processed by DCNN have quite high dimension, but the dimension of electro-data is significantly smaller.Furthermore, a well trained DCNN needs a great deal of labled data, however, the labeled anomaly electro-data are limited.the input data should contain a label attribution, but only extremely scarce data is labeled, so the raw electro-data must be effectively processed before training a neural network.Hence, we propose a method to resolve it and the main contribution of this paper includes: 1) we define a spatiotemporal model about electro-data to enhance variety of sample; 2)we provide an effective method to label electro-data to support training DCNN; 3)we improve designing of DCNN on topology and optomize training of DCNN for anomaly detection.
The rest of this paper is organized as follows.In section 2, we discuss the method of data labelling and modelling.Section 3 introduces an improved DCNN model, and it's training and optimizing.In section 4, we present experiments and results.Conclusion is presented in section 5.

ANOMALY ELECTRO-DATA
All electro-data are collected by electricity data collection system which equipmented with plenty of function in complex structure.Such system contains mass smart meters and various terminals whose malfunction may result in data anomaly.In conclusion, the main causes and phenomena of data anomaly are shown in Table 1.

DATA LABELLING
It's need a large number of well labeled sample data to train a supervised DCNN to detect anomaly electro-data.Due to there are large amount of electro-data in the electricity data collection system, it is impossible to label them by artificial method even partial of them.Therefore, we retrieve 4 months data that two months before and after the artificial inspection which guided by management rule of state grid.According to result of artificial inspection, we classify all data into two categories: NORMAL and ABNORMAL.The detailed policy of labeling is shown in Table 2. To validate the effectiveness of this label method, we randomly select 50 cases for each categories and check by human.The result shows the accuracy of labeling could reach 95%.

DATA MODELLING
This paper follows the concept of computer graphics to model data, first in the temporal dimension different types of electrical data are aggregated at a rate of 24 hours per day, then in the spatial dimension electrical data collected from different natural days are integrated, lastly the electrical data are in the form of 24x24 matrix similar to the images.The specific structure is shown in Figure 1.We further equipment a sliding window to select continuous 4 days data as a sample data and 3 days data overlap between two adjacent samples according to time span, obtaining more training and testing data sample.

DESIGNING OF DCNN
In order to imporve the accuracy of anomaly detection of electro-data, this paper improve the designing of DCNN based on the data smaple modelling mentioned above as follows: 1) The convolution layers have the same number of filters with size of 3*3 for the same output feature size; 2) Doubel the number of filters while downsampleing convolution features.
The deep neural network architecture is pedicted in Figure 2. The number of filters in four convolutional layers are 64, 64, 128 and 128, respectively.In this work, the maximun pooling layer with step size of 2 is taken for down-sample, then the feature size is reduced by half.Softmax classifier is used as the output layer of DCNN, which includes a fully connected layer and Softmax function layer to achieve Softmax regression for multiclass classification problems.

OPTIMIZATION AND TRAINING
In this paper, the loss function of the model is multiclassification cross entropy, the formula is as follows: where yi is the true label of data, i y ˆis the prediction label.Moreover, this paper designs a learning rate schedual algorithm, as follows： where lr i is the learning rate of the i th iteration, lr min is the lowest learning rate, and iters is the total number of iterations.

DATASET
To validate our approach, 500 10kv users who belong to Shandong Electric Power Company are selected randomly, and their electro-data during January 2014 to December 2015 are retrieved.For detail, each user offers 4 months data that two months before and after the artificial inspection.Finally, there are 55000 effective data samples with the ratio of normal to abnormal is 9:1 after a series of data processing, then the normal samples were divided into training and testing set with the ratio of 8:2, so as the abnormal.

BASELINE
We select the traditional machine learning models including SVM classifier, Decision Tree, Random Forest, Adaboost and Multi-Layer Perceptron (MLP) for comparing.The core algorithms used in these traditional models are all implemented by using scikit-learn package, and the feature construction of these models is mainly refering to Fan(2013), Wang(2016), Chen(2014) and Zhang(2017).

METRICS
This paper adopts the following evaluation metrics： where ARR is abnormal recall rate, and APR is abnormal precise rate.TP is the number of items correctly labelled as belonging to the positive class, FP is the number of items incorrectly labelled as belonging to the negative class, and FN is the number of items incorrectly labelled as belonging to the positive class.Figure 4 shows the convergence process of our DCNN and MLP in ARR and APR for comparing robustness.From Figure 4, our DCNN has better convergence effect and is more stable in APR during the whole training process.More precisely, our DCNN has no over-fitting and has stable convergence value with 92.7%, indicating that the network structure proposed in this paper is better.In addition, from the convergence process of APR, the optimization strategies used in this paper avoid over-fitting as well.Thus, our DCNN has good generalization performance overall, and can be applied in the future.

CONCLUSIONS
With the increasing of amount of electro-data, it's hard to detect anomaly among them by artificial way accurately.Due to this problem, this paper presents a deep learning based.Firstly, we provide a spatio-temporal based model about electro-data that allows it cound be adapted to DCNN.Secondary, this paper proposes a method of smaple labeling to support training of DCNN.Moreover, we provide a designing rule about DCNN and a optimization strategy to train DCNN.At last, through case study, our approach obtains more stable APR and ARR.The contribution provides an optional way to make maintenance of electricity data collection system to be more intelligent and effective.

Figure 3
Figure3shows the comparison of our DCNN and other traditional machine learning models in ARR and APR about anomaly detection.we can see that our DCNN is o b v i o u s l y s u p e r i o r t o o t h e r t r a d i t i o n a l models in evaluation metrics, and has better results for 2% lead in APR than MLP indicating that DCNN has better f e a t u r e l e a r n i n g a b i l i t y t h a n M L P .

Figure 4 :
Figure 4: The convergence process of deep neural network.

Table 1 :
Causes and phenomena of anomaly data.