The Development of a Simulation Model for Assessing the CI/CD Pipeline Quality in the Development of Information Systems Based on a Multi-Agent Approach

. The need for the development of fast and efficient software requires the automation during the build, test and release stages. Thus, a methodology of the continuous integration and continuous delivery (CI/CD) emerges which using a continuous integration server which makes easier automatical build, test, and package of the new feature settings. Poor understanding and execution of CI/CD operations can slow down and even stop the pace of a software project. Many bottlenecks in the CI/CD pipeline can arise from misconfiguration, in other words, an inadequate level of automation, inappropriate agent characteristics, and a suboptimal queuing strategy. On the other hand, the finding a way to analyze and improve the settings of the CI/CD pipeline brings great management and economic benefits, since an optimal configuration ultimately implies high efficiency. In this regard, this study attempts to develop a simulation model that will provide a testing environment for a variety of factors that affect CI/ CD performance. The present study, therefore, aims to provide a pipeline simulation model based on the logic of a queuing system, which is based on a multi-agent approach and an experimental environment that allows you to test different settings and scenarios.


Introduction
The study is devoted to solving the problem of development of a simulation model for the CI/CD concept in the field of creating information systems (IS) based on a multi-agent approach.

Pipeline CI/CD
CI/CD is a technology for automating the build, testing, and delivery of the new modules of development project to the parties concerned. The Continuous Integration (CI) and Continuous Delivery (CD) are two aspects of the CI/CD technology, as well as a set of principles and practices that allow to speed up the process of building, testing, and deploying an application.
The Continuous Integration is a methodology of development and set of practices in which developers regularly merge changes to the application code in a central repository, after which the application is automatically builded, tested, and launched. The main goal of the continuous integration is to automate the process of building and testing of the application in order to quickly find and fix errors, improve quality, and reduce the time spent on checking and releasing new software updates.
The Continuous Delivery is a methodology of development that extends the practice of continuous integration by ensuring that all code changes after the building stage are deployed in a test and/or production environment. With the right implementation, the developers will always have a built-in software unit ready for delivery, which has passed a standardized testing procedure [1]. Together, these practices form the CI/CD pipeline, which is shown in Figure 1. All the stages presented in the diagram, except for fixing source code changes, are performed on a special server or group of servers that act as agents, and a special dedicated continuous integration server is engaged in the orchestration of the entire process. After changing the source code of the application, the developer sends it to the central repository and the continuous integration server automatically selects the first free agent from the pool and sends the build task to it. When the agent receives a task, it builds the application and notifies the continuous integration server of the build result. If the build was successful, the continuous integration server again selects the first free agent from the pool and sends the task to it. After completing the tests, the continuous integration server provides a report on passed or failed tests and deliveries the application to the required environment.

Tasks description
Due to the fact that the number of agents is always finite, there is a situation when, with a large flow of source code changes, we can see a queue of tasks for building, testing and deploying the application. In turn, this leads to a longer product lifecycle and slower development, which can negatively affect the commercial performance of the application. Queuing theory offers several ways of the effectively use it to avoid such queues. The main task of queuing theory is to establish an optimal dependency between the number of service channels and their performance. The optimal dependency involves minimizing the costs of service channels and losses associated with downtime of the technological equipment in the service queue and with downtime of the service channels [2].
In this case, each agent in the CI/CD pipeline is a service node, and the entire CI/CD pipeline acts as a queuing multi-channel system (QMS). The sources of requests are developers who submit changes to the application source code to the central repository. Due to the fact that each service node has a fixed cost, the task is to find the indicators for the queuing system and establish an optimal dependency between the number of service channels and their performance for a certain number of events per period.
One of the methods for calculating the indicators that ensure the effectiveness of the queuing system is the method of simulation modeling. It was proposed a simulation model of the CI/CD pipeline with sequential execution of each stage, which is presented in Fig. 2 and the pipeline model with parallel execution of the testing stage, which is presented in Fig. 3 to solve this problem. The design of various types of software based on simulation modeling is devoted to the following studies. [3][4][5][6][7]. Various aspects of software development are devoted to the following studies [8][9][10][11][12][13][14][15].
Agents have a set of the following characteristics: • maintainable task types -it defines what types of tasks can be performed on a particular agent; • service cost per month -the cost for agent operating; • accessibility -if the agent can accept requests; • probability of the agent failure -it describes the probability of the agent fail and requiring the service.

Fig. 2. CI/CD pipeline in simulation model
The input flow is tasks from the continuous integration server that have the following characteristics: • type -it defines the type of task (application build, unit testing, integration testing, UI testing, manual testing, validation, deployment to the required environment); • branch in the central repository -to download the necessary branch with the source code during the task execution; • the duration of the task on the agent; • error probabilities -it defines the probability of making errors during the execution of the task; • error severity -it describes the impact of the error on the task execution. The input flow of requests is random, since the time between the two requests is a random amount and depends on the submission of changes to the application source code to the central repository by the developers and on the completion of previous tasks on other agents. The intensity of the flow λ is the frequency of the occurrence of events entering the queuing multichannel system per unit of time. The number of input flows entering the queuing multi-channel system is equal to the number of agents. Depending on the type of tasks, it is possible to perform them in parallel on free agents. These tasks include testing tasks.
The Queue is the place where the received requests are waiting for the start of service. In this case, the queue has an unlimited length. In addition, the waiting time in the queue is also unlimited. Queue maintenance discipline is: first in, first out.
In the test environment taken for development of the simulation model, the following values of the starting parameters were determined: • λ = 20 requests/day; • K = 20 -number of channels in the system; • Tmean build time = 28 ±7 minute; • T mean unit-tests time = 6±2 minute; • T mean integration tests time = 183±12 minute; • T mean UI tests time = 47±9 minute; • T mean deployment = 52±6 minute. To assess the quality of the received QS, the following indicators were selected: • throughput of the system; • the probability of each channel being busy and all together; • mean busy time of each channel; • the probability of all channels being busy; • mean number of busy channels; • probability of downtime of each channel; • probability of downtime of the entire system; • the mean number of requests in queue; • mean waiting time for a request in queue; • mean of service time; • mean time spent by a request in the system.

Estimate and conclusion
The current study attempts to model the CI/CD pipeline in a simulation system using a multiagent approach in order to establish an optimal dependency between the number of service channels and their performance at a certain number of events per period. The research process will include building and testing of the model. Thanks to this model, it will be possible to calculate the amount of resources required to work on a project more efficiently and take into account many parameters.
To sum up, we can say that the final product of the study has the following applications and advantages: • Low cost: the research on the impact of the various parameters and attributes of the CI/CD pipeline is relatively cheap due to simulation modeling. However, conducting such a study in real life is quite expensive and time-consuming; • through simulation, you can not only test different scenarios, but also track the causes of errors in advance by examining the output data. This may provide insight into some issues that may have important managerial and economic implications.
The work was supported by Act 211 Government of the Russian Federation, contract no. 02.A03.21.0006.