A Quantitative Evaluation Method of Satellite Network Control Protocol based on Multi-attribute Utility Theory

Satellite network control protocol plays an important part in satellite communication system. Different design schemes of protocol have very different protocol attribute values. It is difficult to select a proper protocol solution from many solution alternatives with quite different protocol attributes, especially some mutual conflicted protocol attributes. Thereupon, this paper provides a multi-attribute utility theory based quantitative method to evaluate these protocol design schemes, in order to make a wise decision on how to select protocol solution. This method introduces several critical protocol attributes by utilizing a Petri net model, and aggregates them into a decision making matrix. On the basis of multi-attribute utility theory, quantitative evaluation values are achieved with the support of user attribute preference weight vectors. In the end, the paper presents experiments and discussions.


Introduction
Satellite Network Control Protocol (SNCP) is one of the most important parts in satellite communication system.It allows users to achieve high reliability and high performance communications.But to design and develop a satisfying SNCP is not an easily solved problem.Each SNCP should satisfy several important attributes.For example, SNCP should have high performance and reliability.So the attributes performance and reliability are two important attributes of SNCP.Besides, it is very comprehensive that the investment on designing and developing SNCP should be as little as possible.So are the attributes implemented difficulty, development period and maintainability.As a result, when we try to develop a SNCP, we should focus on the protocol attributes mentioned above.
However, different design schemes (protocol alternatives) of SNCP have very different protocol attributes.For example, one protocol design scheme has higher performance, but need much more investment than another scheme; Or, one scheme has lower reliability, but need more future maintainability.In such situation, it is difficult to select a proper protocol solution from so many solution alternatives with quite different protocol attributes, especially some mutual conflicted attributes.Thereupon, it is necessary to provide a quantitative method to evaluate these SNCP design schemes, so as to make a wise decision on how to select protocol solution.
To solve this problem, this paper proposes a Quantitative Evaluation Method of satellite network control protocol (SNCP-QEM) based on Multi-Attribute Utility Theory (MAUT) [1] [2].As Fig. 1 shows, this method introduces several critical protocol attributes, including performance, reliability, implement difficulty, development period, investment and maintainability.The attributes performance and reliability are predicted by employing a SNCP quantitative Generalized Stochastic Petri Nets (GSPN) model [3].The other four attributes are estimated by analyzing the protocol design schemes.And then, these protocol attributes from different protocol solutions are aggregated into a decision matrix, which will be utilized as the input of quantitative evaluation algorithm.Based on MAUT, the algorithm employs user decision-making related attribute weight vector to achieve quantitative evaluation values and corresponding solution selection ranks.Consequently, it offers fundamental support to choose SNCP solution.At last, the paper presents effectiveness and performance evaluations of SNCP-QEM and some discussions.

Related work
Multi-attribute utility theory has been widely applied in the domain of communication.In this section, we try to present some typical approaches in related domain.
On one hand, MAUT is used for network selection, handoff selection, scheme selection and so on.For example, a network access selection algorithm based on multi-radio parallel transmission is proposed by Zhang etc., to make full use of radio resources in heterogeneous wireless networks.In this algorithm, the optimal scheme is determined based on cost function value of every network scheme by using TOPSIS method with user preferences [4].Dimitris etc. propose a framework to address network selection problem by the employment of multi attribute decision making methods and by using certain Quality of Service (QoS) indicators acting as figures of merit which influence the decision process [5].Multi attribute decision making algorithm is also applied to vertical handoff decision in a WiMAX-WLAN environment to facilitate user with better quality of service, by N. P. Singh etc [6].Zhang etc. present a hybrid QoS-clustering web services composition algorithm based on multi-attribute decision making theory, to solve the web service selection problem with hybrid QoS.This algorithm takes all possible QoS expression types, such as real numbers, interval numbers, triangular fuzzy numbers, into consideration [7].In addition, Guo etc. generalize and formulate multi-service oriented devices aggregation problem based on QoSmetric with multi-attribute decision making.A multiservice oriented devices aggregation mechanism, which concludes two processes dimensionless processing and device aggregation approach, is introduced [8].
On the other hand, MAUT is also employed for quantitative evaluations.A multi-attribute decisionmaking method is provided by Yu etc., to identify the key nodes in complex networks.Each node is regarded as a solution, and each importance evaluation criterion as one solution's attribute [9].Li etc. introduce a dynamic trust quantization model with multiple decision factors based on information entropy.The multiple decision factors include direct trust, risk function, feedback trust and so on, which are incorporated to reflect trust relationship's complexity and uncertainty [10].Liu etc. propose a multiattribute ranking method based on the technique for order preference by similarity to ideal object, so as to evaluate the node importance comprehensively [11].
However, there is little detailed information in current literature that the MAUT has been applied to address the problem on how to quantitatively evaluate and select satellite network control protocol.As a result, it makes sense to provide a quantitative evaluation method of SNCP based on multi-attribute utility theory.

Satellite network control protocol attributes analysis and acquisition
This section devotes to introducing critical protocol attributes and the ways on how to acquire these attributes.

Protocol attributes
Satellite network control protocol attributes conclude many aspects.However, for the reason of evaluating the protocol, SNCP-QEM concerns on the protocol attributes listed in Table I, such as performance, reliability, implement difficulty, development period, investment, maintainability.
These protocol attributes are classified into two groups.One is 'qualitative attribute', and the other is 'quantitative attribute'.The 'qualitative' attributes cannot be precisely measured by values.It comprises implement difficulty, maintainability.On the other side, the 'quantitative attributes' can be assigned with precise values.Attributes performance and reliability belong to the category 'quantitative attribute'.Where do these attributes come from?It is about how to get the values of these attributes.In our method, some protocol attributes values reside on scheme evaluation, which means that the protocol design scheme determines the attribute values.For example, implement difficulty, development period and investment are evaluated by protocol design scheme.Meanwhile, the other attributes lie on model prediction.We propose a quantitative model to predict some critical protocol attributes, such as performance, reliability.
Besides, Table I shows the importance extent for each attributes.Importance extent reveals how important the protocol attribute is for satellite network control protocol.It is thought that the attributes performance, reliability are the most important attributes, so they are assigned with five stars.And the implement difficulty and investment are less important.In the following part of this paper, the importance extent of protocol attributes determines the weight vectors in the quantitative evaluation method.

O T P u
).By utilizing the rules of this modeling method, a typical protocol example 'establishment procedure of communication connection' is mapped to GSPN model N CC shown in Fig. 2.This model describes the whole control flow and protocol signaling interaction behaviors.Signaling places S P in N CC are marked with gray rounds.And sub-models which are indicated with dashed frames respectively correspond to ground user terminal 1, satellite 1, satellite 2 and ground user terminal 2. We assign transition weights of this quantitative model N CC with specific values according to the design scheme of SNCP, in order to get quantitative analysis results.The quantitative analysis procedure relies on GSPN tools PIPE2, which can give out the predictions of protocol performance and reliability.The detail of quantitative analysis is presented in section IV.

Scheme evaluated protocol attributes
Scheme evaluated protocol attributes primarily depend upon SNCP design scheme.For example, the protocol attribute investment indicates that how much money the design scheme will cost during the whole development phase.Attribute implement difficulty is estimated by analyzing the development scheme on the difficulty extent of protocol development.Development period shows how much time the design scheme needs to finish the protocol development task.

Introduction of protocol quantitative evaluation method
The SNCP quantitative evaluation algorithm employs the MAUT that the selected protocol solution shall have the least disparity with the best protocol solution A + (ideal solution) and the largest disparity with the worst protocol solution A -(anti-ideal solution) [1].The utility value of each protocol attribute mentioned in section III is monotonic.For example, if attribute i j x x d , the utility function f(x) has the relation ( ) . Moreover, the best protocol solution is composed of all the best protocol attributes, and the worst protocol scheme is composed of all the worst protocol attributes.The essential of this algorithm is to regard the SNCP quantitative evaluation and solution selection method of m protocol solution alternatives and n protocol attributes as a geometry system constituting of m points in n-dimensional space.Meanwhile, the key point is how to select a protocol solution which owns the least Euclidean distance with the ideal solution., , , , , (1 i m d d ) as theses m protocol solution alternatives.Thereupon, protocol alternative i is expressed with , , , , , , where , , , , , are the n different attributes in protocol solution alternative i.In addition, define these n ( , , , , , , where , , , , , are the attribute values of one protocol attribute in different protocol solution alternatives.
In this protocol quantitative evaluation method, the protocol attributes are classified into two categories: efficiency attributes and cost attributes.The more the values of efficiency attributes are, the better the quantitative evaluation result is.For example, performance belongs to efficiency attributes.Whereas, the cost attributes are reversed.The more the values of cost attributes are, the worse the evaluation result is.Attributes implement difficulty; investment and development period are cost attributes.The quantitative evaluation algorithm consists of the following four steps.
STEP 1. Build a weight normalized decision matrix V.
Build m protocol solution alternatives and n protocol attributes decision matrix D.
Then, do the operation of attribute weight normalization to the decision matrix D. This step completes that the original attribute values are transformed to dimensionless attribute values, in order to lay the foundation for comparing protocol attribute values.Calculate new normalized decision matrix R. Where ¦ .So each protocol attributes have the same vector length.Furthermore, make matrix R weighted, and build weight normalized decision matrix V. Let ( , , , , , ) , , , ) , ,  , , ,  , ,  , , , , as attribute weight vector, where STEP 2. Define protocol ideal solution and antiideal solution.
Suppose that collection J is efficiency attributes collection, and J' is cost attributes collection.Define satellite network control protocol ideal solution A and anti-ideal solution A .
Thereby, protocol ideal solution A and anti-ideal solution A stand for the most preferable and the least preferable protocol solution.
STEP 3. Compute the distances between protocol solution alternatives and the two protocol ideal solutions (ideal and anti-ideal solutions).
In this step, compute the Euclidean distances between each protocol solution and ideal solution.Euclidean distance means the real distance of two points in n-dimensional space.The distance of each solution i A is measured by n-dimensional Euclidean distance.As a result, the distance between protocol alternative and ideal solution and the distance between protocol alternative and anti-ideal solution are presented as follows.) ) Next, calculate the relative distance between each solution alternative i A and ideal solution A .Let i C be the distance between i A and A .
,0 1 According to the above definition, if i A A , then

The protocol solution collection { i
A } is ranked in descending order according to the distances i C .Then the solution preference order of protocol solutions is achieved.In accordance with the order result, it is helpful for users to select the most appropriate protocol solution from many alternatives to meet some specific requirements or conditions.O m computing time.This operation repeats for all of the n protocol attributes.The computational complexity of STEP 2 including formula ( 4) and ( 5) is (2 (6) generates the computational complexity ( ) O n .Formula (7), that computes the distance between each protocol solution alternative and ideal solution A , has the computational complexity of

Computational complexity analysis
. Thereby, the computational complexity in STEP 3 is ( ) O m n u .In SETP 4, if we choose high efficient fast sort algorithm, the computational complexity is ( log )  O n n .Above all, the computational complexity of the whole protocol quantitative evaluation method is . It means that the proposed method owns high execution efficiency.

Experiment and discussion
This section centers on protocol attributes evaluation and algorithm effectiveness evaluation.The typical protocol example establishment procedure of communication connection, mentioned in section III is taken for case study.

Protocol attributes evaluation
Assume that there are 6 protocol solution alternatives, which are represented by A 1 , A 2 , A 3 , A 4 , A 5 , A 6 .Table II and III show the values of model predicted protocol attributes and scheme evaluated protocol attributes.The predicted attribute values are achieved by use of protocol quantitative GSPN model N CC .The model employs different protocol solutions as model inputs, and gets the predictions of protocol attributes performance and reliability.Meanwhile, the scheme evaluated attribute values mainly depends on protocol design schemes.
As a result, we can get the conclusions as follows: (1) By employing Single Attribute Decision Making (SADM) method, the solution rank result is A 4 >A 6 >A 5 >A 2 >A 1 >A 3 according to the efficiency attribute performance; Meanwhile, the rank result is A 1 >A 3 >A 2 >A 5 >A 6 >A 4 according to the cost attribute reliability.(2) Different solutions have contradictory protocol attribute values.For example, protocol solution A 4 has the highest performance but the worst reliability; Protocol solution A 1 has the best reliability, but worse performance.(3) It becomes troublesome, when we have to select one protocol solution from a large number of solutions that own great protocol attribute diversity.These protocol attributes conflict with each other, that increases the difficulty of evaluating protocol solution and even solution selection procedure, as Fig. 3 shows.For example, if we need a protocol solution which has high performance and reliability, it is obvious that more investment and development period are needed.If the foundation is limited, the solution decreases the performance and reliability as a sacrifice.So it is difficult to find out an appropriate solution from these tanglesome protocol solutions.In one word, it is necessary to propose a quantitative evaluation method for several protocol solutions with several mutual-conflicted attribute values.
On the basis of QEM method introduced in Section IV, we apply the above weight vectors to evaluation procedure, and get the evaluation results as Table IV shows.In Table IV, 12 weight vectors are adopted for quantitative evaluation.The larger the QEM value is, the better the protocol solution is under the same weight vector.Thereupon, the solution prefer rank results under corresponding weight vectors are: A 6 >A 5 >A 3 >A 4 >A 1 >A 2 , A 1 >A 3 >A 6 >A 5 >A 2 >A 4 , A 6 >A 1 >A 3 >A 2 >A 4 >A 5 , A 3 >A 1 > A 6 >A 5 >A 4 >A 2 , A 3 >A 1 >A 2 >A 6 >A 5 >A 4 , A 6 >A 1 >A 3 >A 5 > A 2 >A 4 , A 6 >A 3 >A 1 >A 5 >A 2 >A 4 , A 6 >A 3 >A 1 > A 5 >A 4 >A 2 , A 3 >A 6 >A 5 >A 1 >A 4 >A 2 , A 3 >A 6 >A 1 >A 5 >A 4 >A 2 , A 3 >A 6 > A 1 >A 5 >A 4 >A 2 , A 6 >A 5 >A 3 >A 4 >A 1 >A 2 .
1. Algorithm Effectiveness Evaluation a) SADM vs. QEM The evaluation contrasts the effectiveness of SADM algorithm and QEM algorithm.The prefer rank results are shown in Fig. 4. By contrast with SADM rank results according to single attribute performance or reliability (real line in Fig. 4), QEM rank results (dash line in Fig. 4) are quite different, because QEM method considers not only the single attribute performance or reliability but also the other five protocol attributes.However, SADM method focuses on only one attribute and does not put any other protocol attributes into consideration.For example, the solution A 4 rank of SADM(X 1 ) is 1, and its rank of QEM(X 1 ) is 4.Although solution A 4 has the best performance, the other protocol attributes such as reliability are not good.So the solution A 4 rank in QEM method decreased to 4 for the reason of considering other protocol attributes.
In conclusion, the QEM is much more balanced than SADM method, by putting all of the protocol attributes into consideration on the basis of emphasizing one or more attributes.b) QEM vs. QEM (different categories of weight vectors) Fig. 4 illustrates the effectiveness comparisons of SADM and QEM methods, and Fig. 5 shows the QEM evaluation effectiveness of different weight vector categories, such as Fig. 5(a) indicates the QEM value comparisons between prefer one attribute category and prefer two attribute category.We can draw the conclusions that: (1) The QEM values are quite different between different categories, because the weight vectors depend on these different preferences of users.And the user preferences represent different concern points on SNCP.(2) From Fig. 5(b)(c)(d), the QEM method not only focuses on the protocol attributes that user prefers, but also puts all the other attributes into all-around considerations.For example, in Fig. 5(c) the QEM value of solution A 2 in QEM(X 1 , X 2 & X 3 ) under weight vector presents mixed and disordered status, while Fig. 5(e) (prefer all of the attributes) presents a kind of unified status.When all of the protocol attributes are evenly put into consideration (QEM(X 1 , X 2 , X 3 , X 4 , X 5 & X 6 ) with 18 w 18 w 1 ), the QEM values reach a steady state.As a result, such steady QEM values can be adopted as the basis for the protocol solution selection.
Above all, this evaluation vividly exhibits the effectiveness of SNCP-QEM method.Different values reside on the different weight vectors, which reflect the concern points of users.And SNCP-QEM supplies a theoretical solution selection way.

Algorithm Performance Evaluation
The performance test environment is listed as follows: Intel Core i7-2600 CPU 3.4GHz, 8GB RAM, Windows 7 Ultimate 64bit SP1, and JDK 7u21.
Fig. 6 illustrates the QEM performance evaluation results by using a 3-dimention mesh diagram.This mesh diagram uses the numbers of protocol solution alternatives (m) and protocol attributes (n) as independent variables, and uses average execution time (t) as dependent variable.In the diagram, X axis stands for solution alternative number; Y axis stands for protocol attribute number; and Z axis stands for average execution time (ns).The average execution time of QEM is from 9.39 u 10 2 to 7.75 u 10 3 ns, when 3 ≤ m ≤ 10 and 3 ≤ n ≤ 10.It is concluded that the algorithm of SNCP-QEM owns high execution efficiency.Meanwhile, the average execution time grows up as the numbers of solution alternatives and protocol attributes increase.As a result, it is verified by this performance evaluation that the computational complexity of QEM satisfies O(mun).
In one word, the QEM method, proposed in this paper has high enough performance to satisfy the quantitative evaluation requirements.

Conclusion
To sum up, this paper proposes a MAUT-based quantitative evaluation method of SNCP, in order to provide some theoretical and quantitative evidence for SNCP solution selection.The method involves several critical protocol attributes, such as performance, reliability, implement difficulty.Some protocol attributes performance and reliability are predicted by use of a satellite network control protocol quantitative GSPN model.Then, QEM aggregates all of the protocol attributes from different protocol solutions into a decision matrix.By utilizing the matrix as the input of quantitative evaluation algorithm, it employs user decision-making related attribute weight vectors to achieve quantitative evaluation values and corresponding solution selection ranks.Consequently, it offers fundamental supports and suggestions for selecting SNCP solution.In the end, the effectiveness and performance evaluation of SNCP-QEM give the conclusion that SNCP-QEM has a remarkable effect in quantitatively evaluating SNCP and a efficient execution procedure.Protocol Solution

Figure 2 .
Figure 2. GSPN Model NCC of Protocol Example

4. 2
Quantitative evaluation algorithm Suppose that there are m ( 2 m t ) protocol design solution alternatives.We have to carefully choose one of them as the final solution by evaluating protocol solution with multiple attributes utility.Define vectors 1 2

.
Matrix V is acquired by multiplying each column of matrix R with related weight vector j w .Thereby, weight normalized decision matrix V equals as follows.

C 4 .
is regarded as the quantitative value of QEM, which is also symbolized with v(A i )( j w j w j ) in this paper.Depend upon these values, we can determine how to quantitatively evaluate satellite network control protocol in different situations.STEP Rank preference order according to the distances i C of each solution alternative.

Formula ( 1 )¦
is an ergodic process of all the elements ij x in decision matrix D, and the computational complexity is about ( ) O m n u .In formula (2), the decision matrix D is normalized into decision matrix R. The computational complexity of computing for each protocol attribute is ( ) O m .Then, for the n protocol attributes, the , attribute costs around ( )

Table 1 .
Satellite Network Control Protocol Attributes

Table 2 .
Model Predicted Protocol Attributes

Table 3 .
Scheme Evaluated Protocol Attributes