Research on GRU-based credibility prediction of Web services

The service system is based on the SOA architecture, and its component services are usually deployed by third-party service providers in an open network environment. This openness also brings confusion to service system while extending functions. Unavailability of a single service may result in the unavailability of the entire service system. This paper uses Web service credibility as a standard to measure whether Web service is available. Web service credibility is calculated by 12 factors that affect quality of Web service. According to time series of Web service credibility in the past, credibility at next time period can be predicted. This paper proposes a Gated Recurrent Unit (GRU) algorithm which uses grid search algorithm and adaptive moment estimation (Adam) to solve above problem. In this algorithm, grid search algorithm is used to get the best hyper-parameters of network and Adam is used to correct the gradient in the gradient descent. Finally, based on a large number of real Web services, the GRU prediction algorithm is verified by experiments. Experimental results show that the GRU algorithm has higher prediction accuracy than other methods in Web service credibility prediction.


Introduction
The main advantage of service system is openness. This openness makes it easy for service system to accommodate a new service to extend its functionality. However the openness also brings confusion to service system [1]. The openness may lead to unforeseen emergencies in service system [2]. So it is necessary to predict running state of component service in the future. In summary, predicting whether Web services are available has become a research hotspot.
Based on above problems, this paper considers whether Web service is available from three aspects of user, service and environment. The paper uses Web service credibility as a standard to measure the availability of Web service. Web service credibility is calculated by 12 factors such as suitability, accuracy, cost et al. Based on previous credibility time series, this paper conducts credibility prediction in future.

Related works
Some scholars have predicted reliability and credibility of Web service. Lin et al. [3] used fully-connected network to train Web services However, fully-connected is less effective in predicting time series than Recurrent Neural Network (RNN). Ding et al. [4][5] used Autoregressive Integrated Moving Average Model (ARIMA) to predict reliability of service system, which resulted in accumulation of prediction errors. Silic et al. [6] clustered parameters and proposed a reliability model of atomic Web services. Its purpose was to carry out fault-tolerant recovery of service system. Wang [7] proposed a dynamic Bayesian network model based on motifs. But Wang [7] did not verify application effect of the model. This paper uses 12 factors summarized in paper [8] to calculate credibility.
In recent years, artificial neural network has developed rapidly [9]. In fully-connected network, nodes in each layer are disconnected. This shortcoming leads to fullyconnected network being poor in processing time series data. In order to make up for the shortcomings, RNN appeared [10]. Long Short-Term Memory (LSTM) compensates for the vanishing gradient problem in traditional RNN [11]. GRU was proposed by Cho in 2014, it has similar or even better performance than LSTM on many tasks [12].
In this paper, improved GRU is used to predict future credibility. Traditional GRU neural network is improved from aspects of hyper-parameter optimization and network training. Finally, GRU is compared with fully-connected network proposed in paper [3] and ARIMA model proposed in paper [4][5].

Web service credibility
Paper [8] defines Web service credibility using 12 factors. 12 factors are listed in Table 1.

Factor
Definition of the factor Suitability Whether the functionality provided by Web services meets the needs of users.

Accuracy
The accuracy of Web service execution results.

Interoperability
The ability of Web services to call each other.

Security
The ability of Web service to provide authentication and encryption.

Reliability
The ability of a Web service to operate without failure under specified time and conditions.

Availability
Whether a Web service can be called and returned with results, regardless of whether the result meets the user's expectations.

Response time
The total time from a user makes a request to the user gets results.

Cost
The cost paid by service user to service provider.

Accessibility
Whether a web service can be requested.

Maintainability
The ability of adapting to Web services, it is provided by service provider.

Satisfaction
The satisfaction of users to Web service.

Reputation
Reputation of Web service provider.
Calculation of each factor has been explained in detail in paper [8], and will not be described here. Web service credibility is calculated using formula (1).
(1) T(s) represents credibility of Web service s , n represents number of factors, w i represents the weight of i-th factor, v i represents the value of i-th factor. This paper uses aliquot method to obtain factor weights, as shown in formula (2).

GRU network
GRU is a variant of LSTM, presented by Cho Kyunghyun in 2014 [12]. LSTM was proposed by Schmidhuber in 1997 [13]. LSTM can overcome problem of insufficient long-term memory of RNN [14]. However, LSTM is more complicated, and its training time is longer. GRU has improved above problems in LSTM. Fig. 1 shows that forward transfer process of GRU network at time t [15]. It simplifies forgetting gate, input gate, and output gate in LSTM into a reset gate and an update gate [16]. Reset gate is r t in Fig. 1, r t will determine importance of output h t−1 of hidden layer at previous moment to new memory result ht. If h t−1 is not correlated with new memory, reset gate can completely eliminate past hidden layer information.
Update gate is z t in Fig. 1, z t decides how much to pass ht to next state. If z t ≈ 0 then h t−1 completely passes h t , if z t ≈ 1 then new memory h t completely passes to next hidden layer [17].

Improved grid search algorithm
This paper uses grid search algorithm to optimize hyperparameters. Grid search refers to gridding variable area, traversing all grid points, solving objective function value that satisfies constraint function, and finally selecting the best point [18]. The number of hidden layers is n ∈ [1,12], its initial step size is step n = 6 , learning rate is η ∈ [0.005, 0.3], its initial step size is step η = 0.15.

Adam algorithm
In 2014, Kingma [19] proposed Adam algorithm, which combined advantages of both AdaGrad and RMSProp optimization algorithms. Adam algorithm steps are shown in Algorithm 2 [19].

Detailed steps of the prediction algorithm
The GRU algorithm in this paper can be divided into following steps. ①Formula (3) is normalization formula, where x ′ is normalized value, x is original value, maxA is maximum value of factor A, minA is minimum value of factor A.
②Calculating Web service credibility and getting time series of credibility.
③Optimal number of hidden layers n and learning rate η are obtained by improved grid search algorithm.
④After getting input of the algorithm, using formula (4) to formula (7) to get output of hidden layer. x t represents input value at time t, h t represents output value of hidden layer at time t, W xr , W hr , W xz , W hz , W hh , W hx respectively represent weight of input layer and reset gate, hidden layer and reset gate, input layer and update gate, hidden layer and update layer, hidden layer and hidden layer, hidden layer and input layer, b r , b z , b h respectively represent bias terms of reset gate, update gate, hidden layer, h t represents new memory. Activation functions f and g are shown in formula (8) and (9).
⑤After obtaining output value of hidden layer, output value of output layer is obtained by using formula (10).
⑥Assuming that error function at time t is E t which shown in formula (11), error function of sample is shown in formula (12), y t ′ represents actual value at time t.
If error is not within error range, error back propagation is performed using Back Propagation Trough Time (BPTT) algorithm. Error term δ of each neuron is inversely calculated.

= ℎ (13)
⑦After error item is obtained in step ⑥, gradient of each weight is calculated according to corresponding error term. Calculation formula is shown in (14).
In traditional GRU, Stochastic Gradient Descent (SGD) algorithm is used to find appropriate weight to achieve the goal. In this paper, Adam is used instead of SGD algorithm.
Weight is updated according to Adam, and step ④⑤⑥⑦ are repeated until error satisfies condition.

Experiment
This paper uses data provided in paper [8], which includes 2507 Web services from different fields. This paper selects continuous data of 1000 Web services for a week, a total of 168000, forms Web service credibility time series. Split window is 5 hours, so there is a total of 33600 time series, of which 30000 are used as training data and 3000 are used as testing data. Based on the time series, the detailed steps in 3.5 are used to analyse the application process of the algorithm. According to detailed steps in 3.5, the length of split window used in the experiment is 5 hours.
This paper uses Mean Absolute Error (MAE) as evaluation indicators of prediction. Formula is shown in (15), where y n ′ and y n respectively represent actual output value and predicted value in predicted sequence, N is total number of predictions in effective prediction sequence.
Processor used in this experiment is Intel(R) Core(TM) i5-2520M CPU@2.50GHz, memory is 6. 00 GB, operating system is Windows 7(64 bit), language is Python 3.6, development environment is PyCharm 2018.1.4. The GRU prediction algorithm proposed in this paper is compared with fully-connected network in paper [3] and ARIMA model in paper [4][5].
In order to prove applicability of the proposed algorithm, this paper compares traditional GRU, the GRU using grid search, the GRU using grid search and Adam algorithm. Fig. 2 shows the 50-hour prediction results of the three algorithms.   2 shows that GRU has a strong predictive power in terms of time series. However, the GRU algorithm using grid search has higher prediction accuracy, because grid search can perform efficient parameter tuning, and it can make the hyper-parameters of GRU more reasonable and effective. And the GRU algorithm using grid search and Adam has higher prediction accuracy than the GRU algorithm using grid search. SGD tends to fall into local optima, Adam has improved this shortcoming.
In addition, the algorithm of this paper is compared with the fully-connected network in paper [3] and the ARIMA algorithm in paper [4][5]. In order to increase credibility of prediction results, this paper carries out 20 experiments for each algorithm. Results are shown in Fig.  3. It can be seen from Fig. 3 that in terms of MAE, maximum and minimum of GRU using grid search and Adam are 0.32 and 0.5, maximum and minimum of fullyconnected network using grid and Adam are 0.41 and 0.63, maximum and minimum of ARIMA are 0.52 and 0.78, regardless of how number of experiments changes, error of the GRU algorithm is always lower than other two methods, accuracy of the GRU algorithm is higher. Since ARIMA algorithm cause accumulation of errors, its prediction accuracy is lower.

Conclusion
This paper proposes a Web service credibility prediction algorithm based on GRU. The proposed algorithm uses grid search and Adam to improve prediction effect. Finally, based on a large number of real Web service credibility, the paper conducts experiments. Results of the experiments show that: (1) GRU has a good overall effect on time series prediction.
(2) Because GRU is sensitive to hyper-parameters such as learning rate during training, too low or too high learning rate will affect prediction results. In order to improve prediction effect, grid search algorithm or ant colony algorithm can be used to select hyper-parameters.
(3) Compared with other methods of prediction, GRU has higher prediction accuracy.
In summary, this paper verifies applicability of GRU network in Web service credibility prediction. Based on current work, following work can be carried out : other parameter optimization methods can be used to obtain hyper-parameters; expanding number of hidden layers to verify training effect of multiple hidden layer GRU network.