Measuring the efficiency of air traffic management systems

: Global demand for more capacity in air traffic management systems is increasing. At the same time there is growing demand for quality of the service provided by the Air Navigation Service Providers as well as for lower charges. This has led to efforts by the European Union to increase the efficiency of Air Traffic Management in Europe. An analysis and evaluation of the metrics that are currently used to measure the performance of Air Navigation Service Providers and to develop a system of performance metrics for the Air Traffic Management System is presented.


Introduction
The performance scheme and network functions introduced by the European Council contribute to the sustainable development of the air transport system by improving the overall performance of air navigation services (ANS). The European Air Traffic Management System (ETS) is so complex and sophisticated that the smallest improvement in efficiency saves aviation operators millions of euros.
For every air navigation service provider (ANSP) it is necessary to measure the performance of its processes and business activities in order to be able to assess whether the ANSP is achieving its goals and heading in the right direction towards its future vision for the ANSP. Performance measurements are also used to evaluate how an ANSP is performing with respect to similar ANSPs. It is necessary for ANSPs to be able to know their strengths and weaknesses in comparison to other ANSPs.
The objectives of this research are: • to analyze and measure the indicators used to assess the performance of an ANSP; • to make a selection of the main indicators to measure the performance of the ATM system; • to process data on existing performance measurements that are being used or offered for use by ANSP in Europe.
The scope of the study is limited to examining the key areas of safety, capacity and environmental protection as well as their performance indicators. Statistical data from the Eurocontrol database is used for the purpose of the survey.

Literary Analysis
Air Navigation Services (ANS) includes five broad categories of services provided to air traffic during all phases of operation (area control, approach control and aerodrome control). These services are: Air Traffic

Management
(ATM), Communication services, Navigation services and Surveillance services (CNS), Meteorological services for air navigation (MET) and Aeronautical Information Services (AIS) and Search and Rescue (SAR) (ICAO, 2001) [1]. ANSPs provide a package of aeronautical services, excluding SAR, which can be performed by the government or private organizations. The main focus of this study is on ATM with a further focus on service activities for Air Traffic Control (ATC) including CNS services. One of the tasks of the ATM system is to develop a quality indicator, which should quantify the quality of the system functioning.
The study of the literary sources has shown that there is literature for assessing the efficiency of ATM processes and optimal management, according to a previously adopted optimization concept. It comes down to the compilation of a quantitative quality indicator, most often in a mathematical form, which is called the criterion of optimality of ATM system. The various allowances can be quantified in the "better-worse" scale at the value of this quality indicator determined for them.
The study shows that the definition of the concepts of "efficiency" and "effectiveness" of processes in the ATM system in Bulgaria is ambiguous.

Efficiency, effectiveness and performance of the systems
The ambiguous definition of the concepts of "efficiency" and "effectiveness" stems from the difference in the English translation of the words and what is popular in Bulgarian. Efficiency reveals the relationship between the resources used and the result achieved. As an indicator of performance measurement, the productivity of the ATM system is used.
Effectiveness is a concept that reveals the degree of achievement of the objectives set in the ANSP and, above all, whether the services offered are adequate to the needs of the aviation operators.
The usefulness of a system is determined by the concepts of efficiency and performance. Both terms have common features but are not identical. The performance of the system is the full set of its features related to its ability to meet established and presumed needs. Each of these features is called a measure of performance (MOP). They represent quantitative performance measures that can be measured and dimensioned. Only the most representative features reflecting the main properties of the system are selected. They combine into a comprehensive performance index. While performance characterizes the system itself, effectiveness is a feature of usage (action, operation). Effectiveness characterizes the system from the point of view of its users. Effectiveness can be considered to measure the external characteristics of the system, while the performanceinternal. Efficiency as well as qualities can be presented in a mathematical form and have Means of Efficiency (MOE).
In Commission Implementing Regulation (EU) No 390/2013 of 3 May 2013, the English term "Performance Scheme" is called "efficiency". Therefore, for the purposes of this study, it is assumed that the performance scheme introduced by the European Commission is used in the ANSP on both criteria -efficiency and effectiveness. This allows both concepts to be used at the same time to characterize ANS.

Process efficiency in the ATM system
The Single European Sky (SES) performance scheme was introduced with the second Single European Sky (SES II) package. The aim of the scheme is to improve the effectiveness of ANS and network functions in the Single European Sky (European Parliament and Council of the European Union, 2009) [2].
The following key performance areas are identified: safety, environment, capacity and cost-efficiency [2]. For each key area there are goals and one or a limited number of Key Performance Indicators (KPE) as shown in Fig. 1.
Evaluation of the overall ANS performance can be done using a quantitative performance indicator. One way of forming an indicator is to benchmark the system or process with requirements that require a unified approach to assessing efficiency [3]. For each key performance area, indicators are set. Recorded in analytical form are called KPI and can be represented as Is (s = 1, 2 ... μ), where s is the performance area and the μ is the KPI.

Measurement of efficiency in air traffic management systems
The efficiency framework has been expanded into seven areas: capacity, cost-efficiency, efficiency, flexibility, predictability, safety and environmental protection.

Safety
Review of Safety ANS performance in terms of accidents, ATM related incidents and the level of safety occurrence reporting in the EUROCONTROL area [4].
Safety is of upmost importance in aviation. For every change that is made in ATM systems, safety must always be considered as it cannot be jeopardized in pursuit of greater efficiency or effectiveness. However, it is not possible to optimize safety as such. If one seeks to ensure 100% safety in ATM that could only be achieved by grounding all aircraft. The Performance Scheme introduces the following Safety Measurement Indicators shown in Fig. 1.   Fig. 1. Key performance areas and indicators for their measurement. The application of the severity classification is done using the Risk Analysis Tool (RAT) based on Eurocontrol methodology for all operational events classified as SMI (Separation Minima Infringements) and RI (Runway Incursions) and 80% for specific events [4].
The safety levels set by the Competent Authority for 100,000 served aircraft are determined on the basis of statistical analysis of empirical data using Poisson probability distribution and are shown in Table 1.

Productivity
Productivity can be defined as the ratio of the useful result (output of the system) to the resource input (input into the system). When looking at the ATM system, productivity is the output of the system divided by all inputs into the system and can be adopted: • the number of controlled aircraft; • flight time for controlled flights in hours; • kilometers passed by controlled aircraft; • number of flight paths required by ATCOs providing area control service. Input data can be personnel, facilities, equipment and systems. When examining the input to the ATM system, it is logical to consider the contribution of the work of the ATCO. This contribution may be measured in number of ATCOs (in full-time equivalent) or in working hours (hours worked) [5]. Hourly ATCO productivity measures how many flight hours (output) per working hour in the ATCO (input) will the ATCO produce, so the goal is to increase this indicator.
Productivity can be defined as follows: .

ATCOs
In this case, the productivity of the ATCO per hour will be defined as: (4)

Flight efficiency and environmental factors
For each flight, the goal is to define a trajectory with a minimum deviation from the optimal one [5]. The aviation operator (AO) has determined the optimal trajectory and it is included in the flight plan. There are two main types: • horizontal inefficiency where the trajectory is longer in kilometers than in the optimal trajectory; • vertical inefficiency where the height at which the aircraft flies is not optimal. The horizontal flight efficiency H can be represented as: The track extension is therefore a number without unit with the optimal value is zero.
Vertical flight efficiency V can be represented in the form of: where T is the total flight time; T S -the flight time of one flight level; L -the cruising level; L S -the actual flight level. The optimal trajectories are modeled using standard Eurocontrol aircraft databases [5]. The actual trajectory was simulated using the same aircraft performance data and atmospheric assumptions as the reference trajectory. The fuel burn calculations were performed using a model called KERMIT (Kerosene Emissions Research Model): where H is horizontal inefficiency, V CL -vertical inefficiency of the climb, V CR -vertical inefficiency of the cruise, V D -vertical inefficiency of the descent and β are constants. The constants, which were found by regression analysis, are given in Table 2.  3 1,1095 β 7 2,9618 β 4 1,6788 -- The components of φ are additive, φ is therefore not limited from zero to one. As the 3Di Score calculates fuel inefficiency its optimal value would be zero.
The following annul target levels are accepted on the Functional Airspace Block (FAB) level for the Key Performance Indicator "Average horizontal efficiency of the actual trajectory in a flight along a route" (KEA) are shown in Table 3.

Capacity
When the capacity for ATC service does not meet the required number of aircraft, they are delayed at the airport. For the key performance area capacity, the indicators are defined as the average delay in minutes, as shown in Fig. 1.
The delay in the ATFM route is the delay expressed as the difference between the take-off time requested by the aircraft operator in the last flight plan submitted and the estimated take-off time determined by the ATFM central authority. The maximum values for route delays for the "Capacity" are shown in Table 4. In order to ensure the necessary capacity of the airspace, it is allowed to revise the target levels in the increase of the air traffic flows and forecast for their lasting preservation.

Conclusion
The present work is an analysis and selection of the main indicators for measuring the performance indicators of ATM system. The set target levels are determined on the basis of statistical data analysis. The research is a subtask of the single task of identifying and measuring performance indicators. It is of practical importance to determine the productivity of the strategic resource planning system. The results obtained can be used to assess the performance of the ATM system.