Measuring urban public transport performance on route level : A literature review

In this paper, we conducted a systematic literature review of empirical studies that focus on measuring urban public transport performance on route level using frontier methods. 22 papers were identified and included for full text review and were classified according to paper category, year of publication, country context, performance dimensions discussed, methodological approach used, sample size, data type and inputs, outputs and other variables used in each study.


Introduction
The measurement of public transport performance is a crucial tool for transport operators.It generally allows them to verify whether the service is provided efficiently and effectively, to identify areas where performance improvement may be needed, to ensure that community and users are satisfied; and to support decision-making bodies, such as transport authorities and funding institutions (etc...) , to make decision about where, when, and how service should be provided .
The performance measurement in public transport is commonly carried out from four different perspectives.The first perspective is based on the user's perception and satisfaction, or aspects of service, such as reliability, frequency, fares, comfort, cleanliness, etc, are noted by users through surveys satisfaction (See reviews of [1]; [2]).The second perspective consists to measure the objectives achievement expected by the community in which public transport is served.In this sense, the objectives of the community are mainly social and environmental nature such as mobility of aged and disabled people , accessibility of precarious people to employment, reduction of air pollution , reduction of the congestion ... etc.The third perspective is about measuring and comparing the public transport performance from a service provider point of view.A fourth emerging perspective is one that tries to combine the aforementioned perspectives into a single analytical framework (See [3], [4], [5]).
From the third perspective, substantial efforts have been made to develop various methods and models to evaluate and compare public transport performance from system-level, operator-level and route-level's.At the system level, index measures are normally employed to produce a single value to reflect the combined and weighted result covering various kinds of transit activities [6].At the operator level, there is a large and extensive literature on measuring and comparing the performance of urban public transport firms using, namely, frontier methods (See literature reviews of [7], [8], [9]).
In their comprehensive literature review about efficiency and effectiveness of public transport, [8] have found that 76.7%, 20.9% and 1.6% of studies treat performance measurement from Operator-level, Systemlevel and route-level respectively.The measurement of public transport performance on route level is an emerging research field where researchers consider transit routes as production lines.Therefore, the aim of this paper is to review empirical studies that focus on measuring urban public transport (UPT) performance on route level using frontier methods.
The remainder of this paper is organized as follows.Firstly, a brief overview about performance concepts and measurement methodologies is presented.Secondly, the literature review procedure used is described.Thirdly, the results of literature review are presented and finally some concluding remarks are outlined.

Concepts and methodologies
The public transport literature generally distinguishes two performance dimensions, namely efficiency and effectiveness [10].Efficiency is the relationship between the inputs (resources) and the outputs (production) of what is called "productive" or "technical" efficiency in the economic literature.On the other hand, effectiveness refers to the use of products to achieve goals or the consumption of services [11].Due to the non-storable MATEC Web of Conferences 200, 00021 (2018) https://doi.org/10.1051/matecconf/201820000021IWTSCE'18 characteristics of public transport services, these two measures should be considered separately in the evaluation of public transport systems [10][11].
To accommodate non-storable characteristics , [12] have proposed three performance measures that reflect efficiency and effectiveness dimensions.The Costefficiency is defined as the ratio of service inputs (Labor, Capital, and Fuel) to service outputs (Bus-Hours, Buskm, Seat-km).The Service-effectiveness is defined as the ratio of service consumption (passengers, passenger-km, operating revenue...) to service outputs.The costeffectiveness is defined as the ratio of service consumption to service inputs.However, if input factor prices are not available, it would be more appropriate to use the terms of production efficiency, service effectiveness, and operational effectiveness instead of cost-efficiency, service effectiveness and costeffectiveness respectively [13].
In term of measurement methodologies, the most used methods in the literature over the past decades are frontier methods.These methods are classified into two groups: Parametric frontiers and Non-Parametric frontiers.The Stochastic Frontier Analysis (SFA) and Data envelopment analysis (DEA) are two most important methods used respectively in this regard.Due to lack of space for our article, we will present these two methods without mathematical details .For more details, the reader may refer to [14], [15], [16], [17] and [18].
The Stochastic Frontier Analysis (SFA), which is proposed independently by [19] and [20], is a parametric approach and is based on regression.SFA requires the specification of functional forms (production, cost or profit functions) for the technology.This approach takes into account, in addition to technical inefficiency, another random term that encompasses any errors in the observation or output measurement.
The most used version of Non parametric approach is the DEA (Data Envelopment Analysis) method which does not require an assumption of a functional form and it can handle multiple inputs and multiple outputs.DEA is firstly developed by [21] and used to evaluate the relative efficiency of organizational units that transform resources (inputs) into services (outputs).These units are called Decisions Making Unites (DMUs).The DEA technique involves the use of linear programming methods to construct a nonparametric piecewise surface (or frontier) over the data.The efficiency measures are then calculated relative to this surface [17].DMUs located at the frontier have a score of 1 (or 100%) while those below the frontier score have less than 1 (or 100%) and therefore have a scope for improvement of their performance.
Note that no DMU can be greater than the efficiency frontier because it is not possible to obtain a score greater than 1 (100%).DMUs at the frontier serve as peers (or benchmarks) for inefficient DMUs.These peers are associated with observable best practices.
The two basic models of the DEA method most used in the literature are CCR and BCC.The CCR model developed by [21] assumed constant returns to scale, (a model also named Constant Returns to Scale-CRS), whereas the BCC model, developed by [22], assumed variables returns to scale (model also named variable returns to scale -VRS-) Finally, a DEA model (CCR or BCC) can be oriented towards inputs or outputs.In an inputs-oriented approach, the DEA model minimizes inputs for a given level of outputs.In an outputs-oriented approach, the DEA model maximizes outputs for a given level of inputs.

Literature review procedure
The aim of our paper is to present a systematic literature review about empirical studies that focuses on measuring urban public transport (UPT) performance on route level using frontier methods (namely DEA and SFA methods).This literature review is devoted solely to papers published in specialized scientific journals and conference proceedings (from electronic sources).
To find these papers, a search was carried out on the databases like, ScienceDirect, SpringLink, Scopus, JSTORE ... etc, using expressions like " bus route performance", "efficiency of bus routes", "transit route performance"....After executing the procedure, we found 29 papers, 6 of them were excluded because of their methodologies which does not full within frontier methods and one which is a project report.
In the end, 22 empirical papers were included for full text review.They are classified according to their category, year of publication and country context, performance dimensions discussed, methodological approach used, sample size, data type and inputs, outputs and other variables used in each study.

General considerations
Among 22 examined papers, 18 are published in scientific journals and only four are published in conference proceedings.Moreover, the majority of examined studies are published after 2010 (figure 1) which means that the measurement of performance of urban public transport on route level is an emerging field in transit performance searches.
According to the examined papers, we observe that the majority of studies come from developed countries (figure 2).These may be explained by the fact that the intelligent public transportation has reached a higher level of maturity and so route-level data collection has been made easy with GPS and AVL (automatic vehicle location) technologies.

Performance dimensions:
In addition to the traditional dimensions of performance that relates to efficiency and effectiveness, namely technical or operational efficiency, cost efficiency, operational effectiveness, technical efficiency or service effectiveness, some authors have explored other new dimensions of performance, namely quality efficiency, planning efficiency and financial efficiency.Furthermore, these dimensions have been measured separately or jointly.The most studied performance dimension in the examined studies is efficiency (figure 3).

Methodological approach adopted:
From the figure 4, we notice the dominance of studies based on the non-parametric DEA method.The basic or the modified conventional DEA Models are most used.In addition, we notice the emerging trend of using DEA method with other techniques (GIS, Panel data analysis, multi-objective spatial optimization techniques, AHP…).
In contrast, only one study has adopted the parametric SFA method.Finally, we observe that some studies have adopted a mixed approach (use of DEA and SFA methods).

Sample size and data type:
Concerning the sample size (figure 5), the majority of studies are based on a sample size that ranges from 10 to 100 bus route, a few studies are based on a sample size less of 10 bus route or sample size more than 10 bus routes.
When it comes to data type (figure 6), the majority of studies are based on yearly or weekday data.A very few studies are based on monthly data and the remained studies did not specify the type or their data.

Inputs, outputs and other variables
The three most used inputs are related to geographic characteristics of route namely route length and number of stops per route, and those related to service supply on route namely service duration or operation time and The most used output are those related to service consumption namely number of passengers/ridership and passengers-km (miles) and those related to quality of service namely indicators related to service reliability (on-time performance, running adherence, regularity adherence) and finally vehicle speed.
Finally, a very few authors have used some variables that are considered as neither inputs nor outputs.Some of them have used some variables that reflect environmental factors of routes as population density, parking, number of vehicles, intersections, lanes, traffic condition…and others have used variables that reflect externalities on the route as accidents, emission, and pollution.

Conclusions:
In this paper, we have carried out a systematic literature review of empirical studies that focus on measuring urban public transport performance on route level using frontier methods.22 papers were identified and included for full review and were classified according to paper category, year of publication and country context, discussed performance dimensions, methodological approach used, sample size and data type and inputs, outputs and other variables used in the study.A summary of examined studies, following the aforementioned classification criteria, is given in appendix 1 (At the end of this paper).
Finally, it is important to outline the following remarks: -An important set of studies have explored a new performance dimensions in urban public transport (quality efficiency, financial efficiency, spatial effectiveness….)rather than the traditional ones discussed on operator-level or system-level performance measurement studies (technical efficiency, cost efficiency, scale efficiency...) .
-A varied use of inputs and outputs reflecting new aspects (geographical characteristics of the route, quality and characteristics of service, virtual input/output...) instead of traditional inputs/outputs ((Labour, capital, fuel)/ (passenger-km, vehicle-km) used at system-level or company-level performance measurement studies.
-An important set of studies have used daily and monthly data instead of annual data mostly used when evaluating the operators and systems performance.
-The majority of studies are based on DEA than SFA approach.Only one study has adopted SFA and limited studies that adopted a comparative perspective, or mixed for two DEA and SFA methods.
-The absence of other urban transport modes (Rail, metro, tram....) (All identified papers focuses on Bus transport mode).
-Very few studies that have incorporated variables reflecting user and community perspective's ( Corrected fares+ Waiting time cost + In-vehicle time cost+ Reliability penalty cost Total income+ Passenger-kilometers [42] total operation time per day+ number of operating buses per day+ total mileage per day the average number of passengers per day [43] Operational efficiency : capacity and number of stops /Service efficiency : frequency+ stops per km+ service hours average unlinked passenger trips per day for both models

Table 1 .
Most used inputs/outputs and other variables