A two-layer model to dispatch electric vehicles and wind power

In this paper, the optimal charging and discharging schedules of electric vehicle (EV) are studied considering wind power under the condition of distribution network. In view of the uncertainty of EV chargingdischarging demand and wind power output, the Markov decision process is adopted to model the randomness of supply and demand. Considering the dimensional disaster caused by dispatching a large number of EVs’ charging and discharging behavior in a centralized way, this paper proposes the twolayer dispatching model based on Markov decision process. First, the lower EV agents are responsible for collecting the real-time charging-discharging demands for EV and report to the upper dispatching center. Then the upper dispatching center gives the optimal charging and discharging power according to the real-time distribution operating status, wind power output and the EV information reported by each EV agent. Last, the lower agent gives the optimal charging-discharging sequence of each EV according to the upper optimal power. The goal of the upper dispatching center considers the power losses in the distribution network, load variance and the matching degree between EV charging-discharging and wind power output. The goal of the lower EV agent considers the EV charging-discharging fees and costs by EV battery losses. When deciding the optimal charging strategy, we design the two-layer Rollout algorithm to decide the optimal chargingdischarging strategy considering the impact on future strategy decisions by current strategy decisions. Finally, the optimal results under four different strategies are simulated on the IEEE 30-bus distribution network system. The simulation results show that the proposed model and strategy can effectively reduce the distribution network losses and load variance, and greatly improve the utilization rate of wind power. Compared to the cost of uncoordinated EV charging, EV charging-discharging fees and battery loss costs by the proposed strategy have been greatly reduced. * Corresponding author: zhifeng.qiu@csu.edu.cn CSCNS2019 MATEC Web of Conferences 309, 05015 (2020) https://doi.org/10.1051/matecconf/202030905015 © The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons Attribution License 4.0 (http://creativecommons.org/licenses/by/4.0/).


Introduction
In recent years, due to the limited reserves of fossil fuels and atmospheric pollution caused by energy problems, new energy sources have drawn wide attention. Because of its small pollution and limited resources, new energy has become a power generation method competing for development at home and abroad. However, the ubiquity of randomness due to the impact of the weather has had a significant impact on the reliable operation of the grid. Taking wind power generation as an example, the size of wind energy is greatly affected by time period, which may result in low output power at high load or high output power at low load of the power grid. This randomness will lead to anti-peaking. For the reason that the current dispatching mode of the power grid in our country is mainly deterministic dispatching, the wind curtailment in our country is relatively serious. According to statistics, in the first half of 2015, wind power generation in our country reached 17.5 billion kWh [1]. Meantime, a substantial increase in the number of cars in our country has led to the drastic depletion of fossil fuels and the aggravation of environmental pollution. Therefore, EVs have been rapidly developed due to their advantages of non-exhaust gas consumption. At the end of 2014, a total of 780 charging and discharging stations and 31,000 AC and DC charging posts were built across the country, providing more than 120,000 electric vehicles with charge and discharge services. By 2020, it is expected to build a charging infrastructure system to meet the charge needs of more than 5 million electric vehicles [2]. Therefore, it is a direction for the green development of the power grid in the future to comprehensively coordinate the conversion of new energy power generation and electric vehicle power conversion, make full use of wind power output, offset the randomness of both sides of supply and demand.
On the coordinated scheduling of electric vehicles and wind power considering the impact on the distribution network, scholars at home and abroad have done much research. Literature [3] studied the optimal random matching problem of electric vehicle charging and wind power generation in distribution network environment. The scheduling strategy adopted can effectively improve the wind energy utilization rate and reduce the system network loss. However, with the increase of electric vehicles, the calculation complexity of real-time grid scheduling will increase rapidly. In [4], a stochastic economic dispatch model based on the randomness of electric vehicles and wind power generation is established to minimize the expected total generation cost of the system. The results show that the proposed scheme can effectively reduce the cost of power generation. Literature [5] considered the electric vehicle charging behavior under different permeability on the distribution network, using Monte Carlo simulation method to give the EV charging and discharging power curve, the results show that the EV charging and discharging process in V2G mode Can play the role of peak load. In reference [6], considering that there will be a large number of EVs in the future, the direct control of each vehicle by the dispatch agency will greatly increase the complexity of problem solubility. Therefore, the concept of hierarchical and optimal dispatching of EVs is put forward. Literature [7] established the objective function that minimizes the network loss in the distribution network from the perspective of the economic operation of the power grid. The results show that the proposed method can effectively reduce the power loss of the distribution network and can also stabilize the load fluctuation and the role of improving voltage quality. Based on the analysis of the current EV charging schedule strategy, Ref.
Reference [8] pointed out that the EV charging scheduling strategy at the distribution network level should fully consider the uncertainty of charging needs. Reference [9] proposes a cluster-based charge-discharge optimization model to optimize the speed of solution in the light of the possible dimensionality disaster caused by large-scale real-time optimal scheduling of electric vehicles. Reference makes full use of the randomness of vehicle charging behavior and dynamically adjusts the charging and discharging power of EV to take account of the intermittent effect of distributed generation on the regulation capability of power grid, and reduces the impact of distributed generation intermittently on the grid.
In this paper, the main research work is based on the electric vehicle load aggregation effect and wind power output and electric double-random optimization of the individual vehicle charge-discharge behavior. With the increasing popularity of charging facilities, the optimal charge-discharge strategy for a single car needs to be studied urgently. Therefore, the charge-discharge strategy developed in this paper will be specific to each EV. Considering that the centralized real-time scheduling of each electric vehicle will make the calculation of the problem dramatically increase, in order to reduce the computational load of centralized scheduling, this paper will introduce the electric vehicle two-layer scheduling model based on the EV agent. First, the EV agent collects real-time charging and discharging needs of EVs in the region and reports them to the upper-level grid dispatching agencies. Second, the grid dispatching agency targets the network loss, wind matching and variance of load curve in the distribution network, taking into account the impact of the decisions made at the current moment in the future on the basis of the optimal charge amount of each electric vehicle agent at this moment. Third, according to the optimal charge-discharge amount, each agent takes the comparative charging costs of the electric vehicle and the battery loss costs as the goal, given the impact of the current decision-making in the future on the basis of the optimal charging and discharging sequence of electric vehicles.

Two-layer dispatching model for EV
The two-layer dispatching model of EV can be divided into three levels: grid dispatching agency, EV agent and EV. The double-layer model structure is shown in Fig.1. First, the agents of electric vehicles collect the driving information, parking information and chargedischarge plan of the electric vehicles in their respective regions and report the plan to the upper-level grid dispatching agency. Then the dispatching agency of the power grid gives the optimal electric charge-discharge power of each EV agent according to the real-time output of the wind power and the operation of the distribution network. Finally, the EV agent formulates the optimal sequence of charging EVs and discharging EVs according to the optimal charge-discharge given by the upper-level dispatching organization and issues a specific charge-discharge instruction to each EV. Considering that EV scheduling is a discrete stochastic system, and the future state of the system is related to the state of the moment and the action of the decision makers, it has no post-validity regardless of the state and behavior before the current moment [13]. Therefore, this paper uses Markov decision process to model the up-and-down scheduling models.

The EV charging and discharging model
Currently, individual electric vehicles use constant power for charging and discharging, and the use of constant power charging and discharging for electric vehicles is beneficial to the unification of standard of charging pile construction and the losses caused by the battery is relatively small. Therefore, the electric vehicles in this paper all adopt constant power to charge and discharge. The charging behavior of electric vehicles is scheduled by the decision-makers, that is, when the electric vehicle is in the parking state, the power decision right of the electric vehicle will be handed over to the decision-maker. This article refers to the power grid dispatching agency or the electric vehicle agent.
With the large-scale popularization of electric vehicles, the total capacity of EV batteries in a certain area can reach to the capacity range of the conventional grid energy storage equipment. Therefore, in the future, it is mentioned that the electric vehicle discharge will be dispatched by power grid agency. Based on this, this paper adopts the prospect theory as a method to determine whether EV can discharge to power grid in the prescribed time.
The revenue 1 f based on the EV discharge prospect theory comes from the certain electricity bill received by the electric vehicle through the discharge: (1) In which 1 Pr shows the peak electricity price of one day, 2 Pr shows the usually electricity price, d P shows the discharge power. The cost of EV discharge loss 2 f is due to the change of mode of charge or discharge which causes the influence of battery life: (2) In which C expresses the price of EV battery, n expresses the number of recycling for EV battery, cap E expresses the rated capacity of EV battery.
In judging the advantages and disadvantages of revenue and loss, according to the prospect theory, the subjective cognition of EV owners on the proportion of earnings and losses is different. Value function is introduced here to further describe the proportion of earnings and losses: (4) P represents the probability that the remaining capacity of the electric car will satisfy the travel demand. 2 P represents the probability that the remaining capacity of the electric car can't satisfy the travel demand. Here, 1 e indicates the current required charge of the electric vehicle and 2 e indicates the remaining charge of the electric vehicle. When 12 ee  , 1 P and 2 P is defined as: When 12 ee  , 1 P and 2 P is defined as: Based on the above definition, the prospect value V of the electric vehicle discharge behavior selection can be expressed as follows: In which, 0 V  represents the EV can get the relatively large value of discharge behavior in current case, 0 V  represents the EV can get the relatively large value of discharge behavior in current case.

Upper grid dispatch agency
The system state space of the upper grid dispatching agency is defined as n out t E  show the total number of EVs and total electricity leaving the system for the n th EV agent at 1 t  .

Lower EV aggregators
Since each lower-level electric vehicle group is independent from the others, the lower-level model establishes the Markov decision model with the n th electric vehicle group as an example. The lower state electric car group system state space is defined as , , , , nr t L shows the length of the remaining parking time for the r th electric car in the n th electric car cluster at t , , nr t E shows the required power for the r th electric car in the n th electric car cluster at t .
The system behavior space is defined as ,, 1 1 1 Among them, 1 g indicates the required number of t from the moment of parking to the end of parking. 2 g expresses the required number of t from the parking moment until the electric vehicle is fully charged.

Objective function of upper grid dispatch agency
The goal of the upper-level grid dispatching agency is to mainly consider the consumption of wind energy and the economy of the grid-side operation, and calculate the utilization ratio of wind power output by electric vehicle, the variance of load curve and the network losses within the distribution network.
1) In order to maximize the use of wind energy and reduce the abandonment of the wind, taking into account the matching between the charging power of the electric vehicle and the output power of the wind power, the matching degree is used to measure the matching effect.
The higher the matching degree, the more electric power needed to charge the EV at time t can come from the wind power more, which is defined as follows: (15) Among them, 1 EV t P shows the total charging power of all electric vehicles in the distribution network area at time t and can be defined as: 2) This model uses the active network loss in the area of the distribution network to represent t=he network loss, it can be expressed by loss t P [16]:  shows the total nodes.   ,  ,  ,  11   ,  ,  ,  22  ,  1 , , , In order to avoid the electric vehicle from accumulating and charging and causing the peak added to peak, which is harmful to the safe operation of the power grid, we introduce the load variance as the target to smooth the load curve, it can be expressed by F  : Among them, day t P indicates the load power of each node in the distribution network where the electric vehicle is located, 2 EV t P indicates the total discharge electricity of EVs at time t .
In summary, the objective function of the upper grid dispatching agency can be defined as follows (19) Among them, 1 2

Objective function of lower electric vehicle aggregator
In the lower model, the electric car dealer is responsible for the charge-discharge arrangement of the electric car fleet in the region. Under the background of time-of-use electricity price, electric vehicle users hope to save electricity expenses as much as possible without affecting the safe and stable operation of the power grid and extend the service life of the electric vehicle on-board battery. Therefore, the objective function of the lower model will fully consider the electric vehicle charging as well as the economy of battery loss.
1) Time-of-use electricity price in one day is defined as:  2) The total battery loss of the n th EV group at t is defined as: Among them, C indicates battery purchase cost,chooses 15000 C  , n t e indicates the total battery cycle life of the n th EV group at t [17],defined as: Among them, cap E indicates the rated capacity of electric car battery.
In summary, the objective function of the upper electric vehicle dealer can be defined as follows: (24) Among them, 12 1   .
As described in the upper model, for any strategy , n d n dt A   of the underlying optimization model, its performance evaluation index can be expressed as:

(25)
Among them, E represents the expected operator, 0 n d S indicates the initial state of the system of the n th EV group in the lower model.

The constraints of Bi -level markov decision model
(1) EV charge-discharge power constraints: In order to ensure that the battery charge electricity and discharge electricity of the r th electric car in the n th electric vehicle group in time t does not exceed the maximum capacity of the battery itself, the definition is as follows: In addition, the electric energy required for the r th electric vehicle in the n th electric vehicle group in period t must not exceed the maximum charging capacity that it can access within the parking time, and is defined as follows: (2) The uniqueness constraints of EV charge and discharge modes: In order to ensure that each EV uses only one mode during the same period, the rule is as follows: (28) (3) The charge and discharge constraints of Electric vehicle battery: In order to protect the battery of EV, the remaining battery electricity ,r n t ST of r th EV in n th EV group at time t is ruled as follows: In which max S and min S each shows the maximum and minimum charge capacity of EV battery.
(4) Power plant power constraints: In this model, the energy generated by the thermal power plant at time t is used to provide electric energy for electric vehicle charging when the wind power produced by the wind power station is insufficient to meet the electric energy needed for charging the electric vehicle. We use G t P to represent: The satisfaction of the constraint is:

Case study 3.1 Parameter setting
The wind power is generated by weibull distribution. The shape parameter k and position parameter c are 1.309 and 7.0576.
After generating the wind speed of all time quantum the value of wind power can be calculated as: In which, cap W represents the rated capacity of wind turbine, rated v represents the rated wind speed, cutin v represents the cut-in wind speed, cutout v represents the cut-out speed. As mentioned earlier, the EV parking event can be described by its start and parking time, which depend on the initial parking time and the required charging power. There we suppose that the length of parking time in this model all obeys the truncated Gaussian distribution. The Gaussian distribution adopts the parameter in reference [20], The probability We suppose that the travel distance of EV adopts the Chi square distribution and the degree of freedom  is 3.4875and the consumption of energy is proportional to the distance [22]. The battery of EV adopts the parameter of BYDe6 [23]. The wind generator adopts the parameter of Vestas [24]. And the specific parameters setting is as follows in table 2. min max  In which,  expresses the needed electricity each kilometer. Based on hypothesis earlier, the travel distance e of EV can be calculated by formula: In which, r n d expresses the travel distance of the r th EV in n th EV group.
In simulation platform, we set the parameter T in the model as 24 hours, namely to optimize the EV charging process in a day cycle. The electricity prices for each hour of the day are shown in Table 3.

Simulation and analysis
In this paper, the IEEE 30-bus distribution system is used to verify the rationality of the optimal scheduling model of double-decked electric vehicles. IEEE 30-bus structure shown in Figure 3, in which node 5,11,15,21,24,29 for the electric car charging station, each charging station can accommodate 150 electric vehicles at the same time charge by the agents at each node is responsible for coordinating the development of charging plans; nodes 4, 7, 12, 20, 27 are access nodes of the wind power plant. The electric vehicle parking events (including the initial parking time and the length of the parking event) and the wind power output of each time period are generated by the above probability distribution. The parameters of the wind power generator and the electric vehicle are set as shown in Table 2, and the load curve adopts a local area Distribution network typical working day load curve.

Analysis on network performance and wind utilization
Based on the above IEEE 30-bus distribution system, the optimization objectives of the upper-layer grid dispatch mechanism include the network losses within 24 hours of the distribution system where the electric vehicles are located, the average matching degree between the charging power of the electric vehicles and the output of the wind power within 24 hours, and the 24-hour load variance. Optimization goals for lower-level EV agents include optimization of single-and double-target EV charging and battery-loss costs over a 24-hour period. The simulation results are shown in table 4, table 5 and table 6.   From table 4 and table 5, it can be seen that when the upper-level grid dispatching agency adopts the optimized control strategy proposed in this paper, the total network loss in the distribution network area decreases by 171.5176kW in 24 hours, saving the operating cost of the grid. The charge power of electric vehicles and the average matching effect of wind power output in 24 hours has also been greatly improved. The average matching degree increases from 0.2827 to 0.6645 when the EV is out of orderly charging, and the matching effect diagram is shown in Figure 4. The load variance within 24 hours is reduced by 339.78kW, Different electric vehicle charging strategy 24 hours under load distribution network shown in Figure 5. As can be seen from Figure 5, compared with the disorderly charging of electric vehicles, the optimal scheduling strategy for electric vehicles can reduce the peak load of the power grid and reduce the peak-valley difference of the power grid by reasonably arranging the optimal charging amount of each agent at each moment, which is in favor of Grid side of the safe and stable operation.  Lower EV agents use the optimal scheduling strategy proposed in this paper, compared with the disorderly charging of electric vehicles, electric vehicle charging costs reduced by 1230.3551 RMB, the battery cost reduced by 75.9968 RMB. At the same time, compared with the single-objective optimization strategy, the optimization strategy of dual-objective optimization is higher than that of single-objective optimization, but the total cost is reduced. Both to help electric vehicle users to save the cost of charging, but also reduce the loss of electric vehicle batteries, extended battery life of electric vehicles. Figure 6 shows the optimal charging amount of each electric vehicle group within 24 hours under the optimal EV charge strategy. It can be seen that the charging process of electric vehicles is concentrated in the midnight and the lunch break at noon, which is in line with the daily charging demand habits of electric vehicle users and ensures the daily charging needs of electric vehicle users, which plays a positive role in the safe and stable operation of grid side. Figure 7 shows the optimal charge and discharge capacity of each period of 24 hours for EV agents under the optimal charging and discharging strategy. Compared with Figure 6, EV agents' charge-discharge electricity is negative in the 10 to 13 o'clock and 18 to 22 o'clock, which indicates that the total absolute discharge electricity is greater than the absolute charge electricity at each EV agent area. At the same time, this period is generally the peak period of the daily load. At this moment, the electric vehicle reversely discharges to the power grid and can be regarded as an energy storage device to adjust the frequency of the power grid to reduce the peak-to-valley difference and smooth load the load curve. Figure 8 shows a 24-hour charging and discharging sequence of 30 electric vehicles in a certain electric vehicle group under optimal charge-discharge strategy.   The solid black square in figure 8 represents the EV which is charging at this time. The hollow black square in figure 8 represents the EV which is discharging at this time. It can be seen that the overall charge-discharge behavior of each electric vehicle is relatively decentralized and can better match the optimal scheduling goal of the grid side and its own optimization goal. For the reason that a lot of EVs' charging behaviors happens in the lowprice period and EV can discharge to power grid in the peak load period, the EV users can save their comparative charging costs which is advantageous to EV users' participating in the agent's unified scheduling.

Conclusion
In this paper, we construct an optimal model of EV charging and discharging scheduling in distribution network. In view of the uncertainty of EV charging demand and wind power output, a model of randomness of supply and demand is established by using Markov decision process. As the scale of electric vehicles increasing, the centralized scheduling method of electric vehicles may lead to dimensionality disaster. Therefore, this paper proposes a two-layer optimal scheduling model for electric vehicles: that is, the electric vehicle agent collects the total charge-discharge information of the electric vehicle fleet in the area and reports to the upper power grid dispatching agency. Secondly, after obtaining the charging demand and discharging information of each node, the upper power grid dispatching agency aims at optimizing the network loss, the matching degree with wind energy and the variance of load in the distribution network area. Finally, according to the optimal charge-discharge electricity given by the upper agency, the lower agents give the optimal charge-discharge sequence to each EV in their responsible area taking into account the EV comparative charging costs and battery loss costs. In the process of formulating the optimization charging and discharging strategy, we use the Rollout method to simulate the impact of the current moment on the future moment, and ensure the rationality of the optimization strategy at the current time in the multi-time scale. The simulation results on IEEE 30-bus show that the proposed optimization strategy can reduce the network loss in the distribution network, reduce the operating cost of the grid, increase the utilization of wind energy and reduce the peak-to-valley difference of the load. At the same time, the comparative charging costs and Battery loss costs get lower which prove the effectiveness of two layer optimization charge-discharge scheduling strategy mentioned in this paper.