Risk analysis with the usage of IT system TURAWA

In the time of globalization and the continuous travelling of people between different parts of the world, air transportation is becoming one of the most important modes of transport. Nevertheless, it is crucial to continuously improve the level of safety and reduce the absolute number of accidents and their victims. The risk analysis with the usage of IT system TURAWA, which was developed to collect the aircraft’s accidents informations, has been discussed in the presented article. Calculation of risk is based on the data sample collected in the operation process. The implementation of the risk of potential failure event in TURAWA is presented and compared with the prediction of such situation by the decision trees. It has been concluded that models should be developed, which cause in the elimination of the human factor from the decision making chain.


Introduction
All industrial sectors have been under increasing cost pressure in recent years. The policy for the aviation industry is being progressively replaced by companies controlling Quality Management Systems with the use of state regulations. The International Civil Aviation Organization (ICAO) has developed the Safety Management System (SMS), which is applicable to various aviation organisations, including airlines and airports [1,2].
Air traffic services is one of the most important elements in the process of performing aviation tasks regardless of the type of flight. Priority shall be given to the process of safe flight due to its consequences [3,4]. The risk management system is a fundamental element of the safety management approach. In this system all operational processes and procedures, that are acceptable for safety requirements, are described. An operational system consists of aeronautical organisational structures, processes and procedures, people, equipment and devices [5].
Since 2011, IT system TURAWA has been used in the Polish Air Force, which brings together the various users of the institutions and military units in a coherent operating system on a computer network. IT system TURAWA has the ability to enter preventive flight safety documents and to gather the information of flight crew, flight operations, and occurrences [6]. * e-mail: marta.woch@itwl.pl * * e-mail: j.tomaszewska@wsosp.pl * * * e-mail: jaroslaw.wojcik@itwl.pl * * * * e-mail: mariusz.zieja@itwl.pl

Formulation of the problem
Safety risk management is one of the key component of a safety management system. The term safety risk management is meant to differentiate this function from the management of financial risk, legal risk, economic risk and so forth [7,8]. Safety risk is the projected likelihood and severity of the consequence or outcome from an existing hazard or situation. The process of controlling safety risks starts by assessing the probability that the consequences of hazards will materialize during aviation activities performed by the organization. Safety risk probability is defined as the likelihood or frequency that a safety consequence or outcome might occur.
A typical safety risk probability table, in this case, a five-point table, is presented in the table 1. The table includes five categories to denote the probability related to an unsafe event or condition, the description of each category, and an assignment of a value to each category. Once the probability assessment has been completed, the next step is to assess the safety risk severity, taking into account the potential consequences related to the hazard. Safety risk severity is defined as the extent of harm that might reasonably occur as a consequence or outcome of the identified hazard [7].
The severity assessment should consider all possible consequences related to an unsafe condition or object, taking into account the worst foreseeable situation. Table 2 presents a typical safety risk severity table. The safety risk probability and severity assessment process can be used to derive a safety risk index. The index created through the methodology described above consists of an alphanumeric designator, indicating the combined results of the probability and severity assessments. The respective severity/probability combinations are presented in the safety risk assessment matrix in figure 1.

IT system TURAWA
Lack of an effective and reliable information sub-system, practically excludes any sensible actions, was one of the reason to give grounds for a complex system to analyse and assess flight safety. The system intended for the aviation of the Armed Forces of the Republic of Poland will be made operative under the name TURAWA. Efforts to develop this system were started in ITWL (Air Force Institute of Technology) in 2003. Prediction and expectation The TURAWA system is fully functional within a computer network. It has been designed using a three-layer architecture, with the following layers to be distinguished: • the customer's layer that includes the system users' computers, each of them furnished with the www browser, • the applications layer, i.e. a server that contains the system's logic, • the database layer accomplished by means of the database server that stores the recorded data.

Airworthiness Risk Acceptance Matrix
Risk is the potential for losses and rewards as a result of a failure event. Risk is a characteristic of an uncertain future, and not of either the present or past. When uncertainties are resolved, or the future becomes the present, risk becomes nonexistent. Risk does not exist for historical events or events that are currently happening [9]. Risk is evaluated in terms of both the probability of occurrence and the impact of the occurrence. The United States Air Force (USAF) uses a matrix to determine the risk in terms of a Hazard Risk Index (HRI) as shown in figure 2 [10]. In Polish Air Force the Airworthiness Risk Acceptance Matrix is used in the form presented in figure 3. The Airworthiness Risk Acceptance Matrix is implemented in the IT system TURAWA, which support the decision process. The Airworthiness Risk Acceptance Matrix method is insufficient to assess the risks of safety aircraft operation, therefore other, statistical methods are proposed. Figure 4 shows the implementation, together with the interface, of the Airworthiness Risk Acceptance Matrix.

Decision trees
The breakthrough of this contribution was the application of methods, such as decision trees, applied in other fields of science.
Decision trees are a class of predictive data mining tools which predict either a categorical or continuous response variable. They get their name from the structure of the models built. A series of decisions are made to segment the data into homogeneous subgroups. This is also called recursive partitioning. If presented graphically, the model can resemble a tree with branches [11].
A decision tree is composed of nodes and splits of the data. The tree starts with all training data residing in the first node. An initial division is made using a predictor variable, segmenting the data into 2 or more child nodes. Divisions can then be made from the child nodes. A terminal node is the one where no more divisions are made. Predictions are made based on the behaviour of terminal nodes [12]. Year of the promotion of the first pilot Continuous Subsequent departure of the first pilot on a given day Continuous Decision trees offer many advantages. One important advantage is the ease of interpretation of a decision tree. While the tree can be complex, involving a large number of splits and nodes, users can interpret the model [13]. Additionally, making model predictions does not involve mathematical calculations as in General Linear Models. The predictions are based on decision rules. In classification problems, the user can specify misclassification cost. Decision trees tend to give good predictive accuracy and can allow for missing data in deployment [14].
On the other hand, decision trees have such disadvantages as: most of the algorithms require that the target attribute will have only discrete values, or as decision trees use the "divide and conquer" method, they tend to perform well if a few highly relevant attributes exist, but less so if many complex interactions are present.

Decision trees results
In order to carry out experimental studies, flights performed on a third generation fighter aircraft were considered. The data were derived from the last 6 years of operation exploitation process in Poland. Flights were analysed in terms of incidents or undesirable events occurrences [15,16]. Table 3 presents variables, which represent the examined flight parameters.
Evaluation of the performance of a classification model is based on the counts of test records correctly and incorrectly predicted by the model. These counts are tabulated in a table known as a confusion matrix [17]. Although a confusion matrix provides the information needed to determine how well a classification models perform, summarizing this information with a single number would make it more convenient. This can be done using a performance metric such as accuracy, which is defined as follows: In this analysis, 80% of the cases were selected as the testing samples. Table 4 and 5 shows the percentage of correct decisions for the training sample and the learning sample, respectively. X X X X X X X X X X
X X X X X X X X X X In the failure prediction matrix the correct prediction was in more then 70 % for training sample (Table reft2) and in almost 65 % for testing sample (Table reft3). Additionally, it should be taken under consideration how to reduce case "no incident -incident", the mistake in this case has the worst effects. However, if the model predict wrongly "incident" as negative result can be one more technical check. Figure 5 shows the obtained results with the use of decision tress.

Conclusions
In order to properly assess the risks, information about aviation occurrences and their consequences is needed. In Polish Air Forced such information are stored in IT system TURAWA. This system gives the possibility to assess the severity and consequences of an air accidents. Thus the Airworthiness Risk Acceptance Matrix may be created thereafter.
With real data available from the operation process, quantitative and qualitative risk estimation methods, that complement each other, can be used. Airworthiness Risk Acceptance Matrix method based on the expert knowledge. In order to reduce the human factor error, statistical method has been proposed in this contribution.
There are some practical and theoretical issues that need to be taken into account in the future work. The use of tools such as fuzzy logic and statistical reasoning may in the future allow to obtain more accurate results.