ATM change management: An evolutionary and agent-based approach

. Air Traffic Management (ATM) is a complex socio-technical system, whose behaviour depends on a combination of various subsystems of different nature: societal, technical, and human. Due to such aspects, it is difficult to understand which could be the part to be changed in order to improve performances, or which is the impact of a change on the overall performances. Such tasks are though issues, and cannot be easily performed. In this work, a new approach for the ATM change management process is proposed. It aims to introduce an innovative multidisciplinary process by combining the following different paradigms: the agent-based paradigm for the modelling of a change solution and the assessment of the achieved performances; the evolutionary computing paradigm for the tuning of the change: the sensitivity analysis to understand which part of the ATM system should be changed in order to match the targeted performance levels.


Introduction
Despite the excellent services supported by present Air Traffic Management (ATM) system, present architecture does not provide an efficient mechanism to adapt the capacity to the demand by exploiting the flexibility of the air transport system.To enhance ATM and meet the future traffic demand and environmental requirements, present ATM system is under evolution.The Single European Sky Air Traffic Management Research Program (SESAR) aims to transform European ATM into a more modular, automated interoperable, flight and flow-centric system.
The key objectives are to restructure European airspace as a function of air traffic flows; to create additional capacity; to increase the overall efficiency of the ATM system.In order to fulfil these objectives, the European Commission set some performance driven high-level goals on capacity, safety, environment, costs for airspace users.Hence, the change management process becomes crucial in order to design the new solutions to achieve the expected performances.In addition, the design and the validation of the changes are tough issues due to the fact that the evaluation of the impact of a possible change on the overall ATM system cannot be easily performed.Indeed, the global behaviour of the system is not easily predictable, since ATM is a socio-technical complex system joining three major interacting subsystems: the societal subsystem that contains the society constraints such as airlines; the technical subsystem that contains all technical infrastructures supporting the functioning of the ATM system; the human subsystem that includes all human components of the whole system, from flow managers to supervisors, pilots and controllers.This work proposes an innovative process to perform the change management by combining the agent-based paradigms with evolutionary computing and sensitivity analysis.It discusses preliminary considerations on the approach itself, the Agent-Based modelling paradigms to adopt, the sensitivity analysis, and the possible Evolutionary algorithms to use; the referring case study to validate the approach; the performances to apply.

Related work
The ATM change impact assessment is addressed by other SESAR exploratory research projects in several ways.For example, PACAS project is structured around a collaborative platform and expert assessments on different performance areas by using different formalisms, like fault trees and security networks.Other research projects have been focused on the performance evaluation of an ATM solution.APACHE project proposes a new framework to assess European ATM performance based on simulation, optimization and performance assessment tools that will be able to capture complex interdependencies between Key Performance Areas (KPA) at different modelling scales (micro, meso and macro).AURORA project proposes advanced metrics to assess the operational efficiency of the ATM system.These new metrics will be developed with the aim of encapsulating the airspace users' operational objectives, considering fuel consumption, schedule adherence and cost efficiency of the flights.User-preferred trajectories are defined as references for performance analysis purposes.
Other research projects investigate the performance evaluation by using the agent-based modelling and simulation technique, in order to take advantage of its high flexibility in terms of level of abstraction.In these projects, the agent-based simulation is used (usually in combination with other techniques) either as an analysis tool to understand emergent behaviours of the ATM system, or to study resilience and disturbance propagation (see projects CASSIOPEIA, ELSA and MAREA).For what concerning the Agent-based Evolutionary Search (AES), it is an emerging research paradigm.Depending on the research area taken into account, the combination of EC and ABM is still relatively unexplored, especially if it is considered the study of socio-technical systems in which the interaction between individual/cognitive, social, technical and environmental factors, generates complexities that are difficult to predict and dealt with [1].A recent edited volume by Rennard [2] describes numerous applications in economics, but applications of EC to other social sciences (anthropology, political science, sociology, psychology) have been relatively fewer.Even if the first attempts to exploit evolutionary approaches in the analysis of air traffic assignment date back to the early 1990's, most interesting works about ATM and ABM have been published only very recently in more limited contexts and with exploratory goals limited in scope [3].

Change management approach
The change assessment process is made up of different steps: understand which are the targeted performances, which element has to be changed to obtain such performances; design at high level the change and choose on some decision variables which could be the best tuning to match the targeted performances; implement the change and verify and assess the impact, that is, the obtained performances.The proposed work starts from the current state-of-the-art in agent-based modelling for ATM.It aims to extend the state-of-the-art on ABM in ATM and the respective simulation methodology, by introducing: 1. a full agent-based model of the direct routing operations; 2. possible different paradigms according to the nature of the specific agent; 3. the evolutionary computing approach for the optimal tuning of the modifications; 4. a sensitivity analysis strategy for the identification of the most critical parameters from a performance-oriented point of view (see Figure1).The simulation may "discover" the emergent (i.e., unpredictable) aspects of the change, also in terms of failures and performance degradation of the global system.
As a consequence, a set of KPIs for the ATM domain shall have to be rigorously established in order to include all the relevant performance dimensions.They may be directly measured or they may be calculated from supporting metrics.In addition, the agent-based simulation may be used as a feedback to the design process of the new ATM solutions for their optimal tuning and calibration, in order to comply with the requirements for integration.The calibration of the modification implies the fine-tuning of the model that integrates the modification.The parameters that are related to the modification (a new or modified component, a new procedure, etc.) are chosen so that the performance of the designed system is optimal in some way, e.g., with respect of some reference KPIs.In detail, the introduction of evolutionary computing approaches for the optimization allows for a proactive approach to the design of a modification enabling the emergence of the most suitable solution.The optimization shall be performed by means of evolutionary agents, i.e., agents whose behaviour rules are processed by means of evolutionary computing paradigms.The provision of feedback to design relies in exercising the system after the insertion of the modification by means of evolutionary agents and running the optimization algorithms in order to find the proper constraints to the inserted modification in order to fulfil the expected KPIs of the overall modelled system and other possible constraints.

Agent-Based modelling
One of the main difficulties in the modelling of the ATM system architecture is the coexistence of human, technical and information managing components that must be described considering a holistic approach in which the relationship between components and actors can lead to different behavioural states.Despite most of these relationships rely on information flows that can be traced, there are several intangible flows difficult to identify (trust, knowledge sharing, experience, training, competences) that influence drastically in the overall system behaviour.Thus, the modelling framework to be used in the proposed work, to get a deep understanding of the behavioural dynamics of component/actors relationships must be enhanced with mechanisms to support complex relationship dynamics described by multiple loops and chains, loops within loops, mutual cross feed relationships connecting them, inhibitory connections and preferential reactions in front of different events.
Despite there are relatively few scientific publications of ATM modelling [5] considering ATM as a sociotechnological system that could support the holistic approach mentioned, there are well accepted formalisms (see Figure 2) that could provide the right framework considering different abstraction levels.Among these dynamic modelling approaches, in the proposed work it is chosen to combine the use of Agent Based Modelling (ABM) and discrete event formulation.The first one will model the tactical layer (i.e., Aircraft and ATC) because it supports different types of tangible and intangible relationships connecting the actors of the ATM architecture and in consequence provides an excellent quantitative analysis tool.The second one it is proposed for the strategic layer (Network Manager and Air Operator) to be able to support a better understanding of the traffic behaviour at the tactical layer by suggesting new roles or changes in the architecture component and services.

Evolutionary computing
Agent-based Evolutionary Search (AES) is an emerging research paradigm lying at the intersection between Evolutionary Computation (EC) and Agent-Based modelling (ABM), two methodologies that pledge to contribute together to the analysis and the solution of challenging and complex problems in in areas that span from online trading to disaster response [6].In general terms, the idea behind AES is to exploit a bio-inspired framework to explore the dynamic adaptation of complex systems as it evolves over time, so as to discover the significance of critical factors of stability and change.A major challenge in this field is the exploration of the operating conditions that separate sustainable and stable from fragile or unstable configurations of the target system.
The main objective of the application of EAS is to improve the AB model performance by varying it.The integration of ABM and EC usually works according to a schema that may be summarized as follows [7]: an evolutionary algorithm maintains a population of "individuals" that represent the potential ABM variations.
Each model variation is run in order to assess the quality of that particular model with respect to a given goal.Once a "fitness", based on the quality of the model it represents, has been assigned to each variation, a selection mechanism chooses the more fitting individuals to produce "offspring", representing the variations on the parents to be evaluated as part of the next generation.Less fitting individuals (model variations) are eliminated, so the average fitness of the population increases over time.In order to apply these notions, and to explore how ATM architecture, rules and procedures could be better designed, reformed or re-engineered it is obviously necessary to specify which parts of the ATM model are changing and what the selection pressure is, understanding at which level intends to apply the EAS: model variation can be represented at parametric, structural, and behavioural variations.Over the years, evolutionary algorithms have been shown to be particularly good at solving parameter optimization problems in which the parameters interact in highly nonlinear ways [8].In this sense the model variation will be specified by means of simulation parameters whose values are left unspecified within a range of pre-defined values and thus will be used for tuning the parameters.It could be chosen also to automate model refinement at a more fundamental level by allowing variations of structural elements of models like, just for instance, the network layout, or the communication patterns.The proposed for improving model performance based on this kind of structural variation.The most fundamental level of model variation will be the characterization of the behaviour of the agents that make up a system as even simple changes in these behaviours can have a huge impact on the emergent behaviour of a model over time.

Sensitivity Analysis
One of the main problems of ABM is the difficulties to trace the interactions between the agents that generates a global system behaviour from isolated component/actor behaviours.Thus, the sensitivity analysis in ABM should allow to identify the most critical parameters that affect some reference performance metric as a results of complex agent relationships formalized as a complex adaptive system.
Most of the time, in a stable condition, the variation or oscillation of metrics and agent attributes around a reference value remains within a variation range with very well defined boundaries, characterized by limit values.The variation of each metric depends on the interaction with other metrics and agent attributes that exist in the system.For a specific metric, the other agent attributes constitute the environment responsible for the noise, which represents the variability of this environment.Consequently, the functional resonance method supported by FRAM (Functional Resonance Accident Model) [9] can be considered as the detectable event that appears out of the non deliberate interaction of the weak variability of many metrics and agent attributes interacting with each other.In Figure 3, it is represented the agent relationships to be implemented in the ABM framework considering the FRAM approach for sensitivity analysis.

The case study
The validation of the approach finds its effectiveness when the a specific part of the ATM system is chosen to apply the proposed approach.The soundness of the validation relies on the representativeness of the modelled part of the ATM system in terms of complexity, involvement of human aspects, criticality and interdependencies among the subsystems.To these purposes, the reference scenario takes place in the execution phase, for en-route flights both in cruise and vertically evolving for cross Area Control Centers (ACC) borders and in medium and high complexity environments.The overall approach will be applied to a specific case study starting from the reference scenario and proposing an evolution in direct routing that introduces the revision of trajectory proposed by SESAR.Such a revision will be modelled involving a new (evolutionary) agent, airspace user, to negotiate the conflict resolution with Air Traffic Control (ATC).

The direct routing
Free Routing is intended to overcome the excess of distance flown by aircraft due to the constraints imposed by the current airspace network structure.Within Free Routing, the Direct Routing paves the way forward an environment where the user preferred routing is working, by allowing more direct route options so that the users can be closer to their preferred flight path.Direct Routing allows users to select a direct route between two published waypoints without the need of passing by the intermediate points of the fixed-route network.This operational solution is particularly relevant for crossborder operations in high and very high complexity environments.The impact for controllers is substantial, since Free Route increases the unpredictability of the traffic making human actors a critical source of its performance; , 0 • Direct Routing allows the comparison with the evolution to free route, since it is open to several architectural evolutions, such as the introduction of a new agent, airspace user, who will be involved in the negotiation of changes to the agreed trajectory; the inclusion of the short term planning timeframe with Demand and Capacity measures.

Human aspects
In order to support the design and development of evolutionary solutions to Free Routing, it is relevant to assume a wide perspective reviewing the behavioural and cognitive aspects of the controllers working in the Freeroute environment.In the presented work human aspects (pilots and Air Traffic Control Operators -ATCO) are investigated under three perspectives: learning from direct routing scenarios elements on how human agents behave, to model the transition to the free route simulation; describing at operational level the cognitive processes in ATM (desk analysis on literature review); depth analysis of the socio-cultural aspects through literature empirical data gathering and their subsequent heuristic validation form expert pilots and ATCOs.The human behaviour works as an elective adaptive system, by which we reply to needs, keep our balance, live in our environment.The chosen approach for the proposes work combines the human activity theory addressed by Leont'ev [11], the Hierarchical Task Analysis (HTA) [12] and the recent studies on the complexity in work environment [13].The human activity is a hierarchical system where each activity comprises a set of actions including a set of operations.Activities always reply to a motivation, that at the level of the actions refer to a goal and at levels of operations are conditions.This theory includes the tools, the equipment and systems that support the action execution; the sets of conditions determining how and why individuals may act; these conditions are expressed in rules that are a result of the social conditioning [14].The HTA enables to model the complexity cognitive levels that interplay in ATC implying at the same time knowledge-based skill-base and rule-based processes [15].The expected description of the HTA focuses on the structure of the main goaloriented tasks and subtasks in the free route control management, specifying the temporal and causal hierarchy and triggers such as operation procedures.Moreover, quantified variables on behavioural, cognitive (Visual search, Trust, Problem solving, Decision making, Judgement on take actions, Prioritization Situation awareness) and socio-cultural aspects (Group behaviour, verbal coordination, Shared situation awareness, Negotiation) are provided.The complexity aspects in the Free Routing environment, such as Distributed systems, Hazardous system, Automation, etc. will be considered too.

Applicable performance metrics
To the evaluation of the performances achieved by the proposed scenarios, the applicable metrics to assess will be: punctuality, fuel consumption and safety.Punctuality or timeliness can be understood as the difference between the planned and the actual departure and arrival times.It can be: (i) zero for timely arrival; (ii) positive for delayed arrival; (ii) negative for early arrival.Taking as a limit of acceptability a 6 minute delay.Fuel consumption can be very different say between full thrust at take-off and initial climb, on climb after cut-back, on approach at idle, or on cruise at different altitudes and speeds and on acceleration or deceleration.Thus fuel consumption needs to be considered by sector of flight and added for all sectors, unless all sectors refer to cruise at the same speed and altitude.The evaluation of safety can be performed as follows: (i) consider each pair of aircraft trajectories and determine the point of closest approach: (ii) if either the altitude difference meets or the horizontal separation meets the situation is safe; (iii) if both the assigned separation thresholds are violated then a potentially unsafe situation arises; (iv) the speed, heading or vertical rate of one or both aircraft must be changed until at least one of the thresholds is met.The key to this assessment is the calculation of the altitude and horizontal separation at the point of closest approach between two trajectories.This is not necessary in many cases when there is no doubt; e.g. if the aircraft fly horizontal at different flight levels separated more than the threshold or if their horizontal flight paths never come closer than the threshold.

Conclusions and future works
The proposed work defines a new approach in order to perform the change assessment in ATM system by supporting all foreseen steps, from the strategic thinking to the change impact assessment.The innovation potential of this proposal lies both in the innovativeness of AES application in the ATM field and in the strongly interdisciplinary vision inspiring our work.It leverages on the ABM formalism, enriching it with the EAS computing and the Sensitivity analysis techniques.The in-depth and interdisciplinary investigation of the potentialities of hybrid modelling frameworks, together with the validation results obtained by applying it to a real ATM subsystem and changed scenario will enable to get methodological and applicative results suitable of generalization.It will support the harmonization process by enabling the modelling of the ATM system according to different levels of scale, at least regional, national and trans-national.The approach has to be further specialized in the definition of the ABM paradigm and the kind of AES algorithms to be chosen, in applying the approach to the addressed case study and analysing the obtained results.

Figure 3 .
Figure 3. FRAM Agent relationships.The implementation and deployment of Direct Routing requires operational and airspace changes as well as the evolutions of the air traffic systems, and in particular of the Flight Data Processing ground system.The selection of Free Routing as Case Study responds to a number of considerations.• It is part of the SESAR Pilot Common Project.Free Routing will be mandatory in Europe above FL305 in 2022; • Direct Routing has been partially implemented in some parts of Europe and is still pending in many others.This uneven implementation level makes the scenario interesting because of the data availability, on one hand, and of the remaining open issues and unclear implementation aspects, on the other hand; •The impact for controllers is substantial, since Free Route increases the unpredictability of the traffic making human actors a critical source of its performance; This work has received funding from the SESAR Joint Undertaking with grant agreement No 783189 (EVOAtm project) under European Union's Horizon 2020 research and innovation program.