A comparative study of performance measurement standards of railway operator

The European standard (EN 13816), is one of the widely accepted standards for measuring the quality of public passenger transport (PPT) service. EN 13816 indicates 8 measurement criteria, 29 sub-criteria and 193 Key Performance Indicators (KPIs) to be used to measure the performance of railway operators. Nowadays, there are other addition criteria beyond EN13816, developed by various organisations. This research firstly aims to explore the service performance measurement of railway operators used by actual railway operators at international level and in Thailand. After an intensive review of performance measurement standards, 9 standards are compiled and compared in terms of criteria, subcriteria and KPIs using a cluster analysis methodology. The result found additional performance measurement aspects at 2 sub-criteria and 91 KPIs in addition to EN 13816. This research summarized and compared different performance measurement standards to measure service quality of metro rail line.


Introduction
The first section of the Chaloem Ratchamongkhon Line (Blue Line) running between Hua Lamphong Station -Bang Sue Station was officially opened in 2004 as Bangkok's second public transit system. The Metropolitan Rapid Transit (MRT) is the first underground rail line in Bangkok Metropolitan Region, with the total distance of 20 kilometers, totaling 18 stations. The second section, the MRT Purple Line Project running from Khlong Bang Phai Station -Tao Poon Station, with the total distance of 22 kilometers, totaling 16 stations, was later opened in 2016 [1]. The MRT is operated by the Bangkok Expressway and Metro Public Company Limited (BEM) under a concession granted by the Mass Rapid Transit Authority of Thailand (MRTA).
In 2015, Thailand had planned to expand the metro-railway network. The MRT Blue Line extension will compose of 1) Hua Lumphong -Bangsue section and 2) Bangsue -Tha Phra section, and will connect with the MRT Purple Line Project, Tao Poon -Rat Burana Section [1]. Construction of the Red Line section from Bang Sue to Rangsit was started in May 2013 and will use some of the concrete infrastructures from the Hopewell project. In addition to the main route of the Red Line between Bang Sue and Rangsit, there are several extensions planned. These extensions include Bang Sue -Taling Chan, Bang Sue -Hua Lamphong, Rangsit -Thammasat University, Taling Chan -Salaya and Don Mueang -Phaya Thai (Improvement of Railway Passenger Services, 2014). In 2017, MRT Pink Line and Yellow Line projects are planning to be built. MRT Pink Line Project will start from Khae Rai Station to Min Buri Station, with the total distance of 35.5 kilometers, totaling 30 stations. MRT Yellow Line Project from Ladprao Station -Samrong Station, with the total distance of 30.4 kilometers, totaling 20 stations is planned to be built as a monorail line [2]. Along with the MRT, the Bangkok Mass Transit System (BTS), and the Suvarnabhumi Airport Rail Link (SARL), are also a part of Bangkok's railway transportation network. Figure 1 shows the Bangkok Metro-railway map. Due to the fact that the 3 systems, namely MRT, BTS and SARL are operated by different companies, therefore performance measurement standards of these railways operators are somewhat different. Some of the operators develop their own performance measurements of service quality prioritising on market-driven and customer-oriented satisfaction. However, some of these operators' performance measurements criteria are still based on standards, such as European Standard (EN 13816), CoMET and Nova.
Other international measurements standards might have different viewpoints of measuring service quality. For example, Sweden developed a robust performance monitoring tool for Performance-based Railway Infrastructure Maintenance (PBRIM). Service quality for rail operators in Northern Italy consists of seven main characteristics of a railway service and 33 service quality attribute. In order to improve public transport and increase the use of public transportation, Boston Consulting Group (BCG) developed the Railway Performance Index (RPI) to provide a tool for measuring railway performance. The RPI is the most comprehensive benchmarking widely accepted by European railway operations.
The purpose of this research is to develop and compare performance criteria measuring service quality of metro rail lines operators at international level and Thailand.

Related Literature
Clustering is the unsupervised classification of data into groups of objects that share common characteristics [3]. Clustering can produce either disjointed or overlapping partition. In an overlapping partition, it is possible for a document to appear in multiple clusters whereas in disjointed clustering, each document appears in exactly one cluster [4]. Clusters are mainly categorized into partitioning and hierarchical methods. A partitional cluster is a method for grouping the data objects into non-overlapping subsets. A hierarchical clustering method works by grouping data objects into a tree of clusters. These methods can further be classified into agglomerative and divisive hierarchical clustering depending on whether the hierarchical decomposition is formed in a bottom-up or top-down fashion [5]. In this paper, we only focus on hierarchical clustering.
The performance measurement standards for measure quality of service that are internationally accepted are for example; European Standard EN 13816: 2002 [6], CoMET and Nova benchmarking groups [7]. Thailand also established a performance measurement standards for measure the quality of service but they are varied according to each operator such as; Suvarnabhumi Airport Rail Link (SARL), Mass Rapid Transit Authority of Thailand (MRTA). Most of performance measurement standard, used by Thai Railway Operators standards often adapted from international standards. Their main focuses of the performance are punctuality and reliability, by choosing the appropriate. Currently, some of the Thai Railway Operators are aiming at International arena since they are registered members of The CoMET and Nova group. However, the measuring system should be of a high standard and comparable among operators [8].

Data Collection
The researchers reviewed other performance measurement standards of railway service used by international railway operators and Thai's operators, then compared performance measurement criteria. Standards for measuring service quality of railway operators are as following;

European Standard EN 13816: 2002
European Standard (EN 13816) is the most comprehensive performance measurement aspects of service quality in transportation sector. It is also the international standards accepted worldwide. It is composed of 8 main criteria which are Availability, Accessibility, Information, Time, Customer Care, Comfort, Safety, Environmental Impact, 30 sub-criteria and 193 KPIs [9].
One result of Quality Approach in Tendering Urban Public Transport Operation (QUATTRO) is the quality loop of public transport system [10]. The diagram in Figure 2 illustrates the Quality Loop set out in EN 13816, which defines a clear process to ensure that the service provider can meet the needs of existing and potential users most effectively, and therefore be as convenient as possible [11].

CoMET and Nova Benchmarking Groups
The Community of Metros (CoMET) was established in 1994. The original members of the CoMET were Hong Kong's Mass Transit Railway Corp, New York City Transit Authority, Berlin U-Bahn, London Underground and Paris Metro. CoMET is a system of international railway benchmarking and consists of large metro systems from around the world. Following the success of the CoMET, the Nova group was later set up in 1998 as another benchmarking association for the more medium and small-scale metro systems [12]. Nowadays, CoMET and Nova comprised of 34 large and medium-sized metro systems from 32 cities around the world [7].
CoMET and Nova provider a confidential forum to share experiences, compare performance, and identify best practices to improve performance. It consists of 3 indicators, 6 criteria and 32 KPIs [12]. The data set is collected according to the standard definition set by all the metros of CoMET and Nova, on an annual basis [13].

Customer Satisfaction Surveys at Istanbul Railway Systems (IRS)
The Istanbul Transportation Corporation, an enterprise of the Istanbul Metropolitan Municipality was established in 1988 for operating the public rail transportation systems and for managing all other public transportation.
To assess the user satisfaction level, Istanbul Ulasim (IU) has been conducting annual passenger satisfaction surveys since 2005. The passenger-oriented performance index (POPIX) can be determined through the Passenger Satisfaction Surveys by classification that measures the indicators from the questionnaire, determination of each measure indicator satisfaction. The performance collected from the 6 criteria in the surveys are fares, reliability, cleanliness, comfort, security and security and 24 indicators [14].

The European Railway Performance Index
Boston Consulting Group (BCG) developed the Railway Performance Index (RPI) to provide a tool for measuring railway performance. The Railway Performance Index (RPI) measures railway systems' performance in three dimensions for both passenger and freight traffic. The three performance dimensions are Intensity of Use, Quality of Service and Safety. Each dimension comprises at least two sub-dimensions, and all were given equal weight percentage for each criterion because it's easier to understand and analyse as shown in Figure 3 [15].

Maintenance in Swedish Transport Administration (STA)
The Swedish railway network system has approximately 20 companies that are now using the Swedish state's rail infrastructure. The maintenance of the railway network (rolling stock and infrastructure) is also managed by many different companies. Morant refers to Espling (2007) studied the maintenance strategy for a railway infrastructure in a regulated environment by implementing benchmarking techniques to compare different case studies from the Swedish railway network. The result of the analysis identified 4 risk areas, which are the risk of losing control over maintenance costs, asset condition, safety demands, core competence and asset knowledge [16].
Swedish Transport Administration (STA) is the standard developed to monitor the outcome of Performance-Based Railway Infrastructure Maintenance (PBRIM). It is composed of 4 objectives, 5 strategic and 19 measures. The outcome measure could be divided into strategic areas representing the basic maintenance objectives. [17].

Railway Service Operating in the North of Italy (RSI)
The number of passengers in the Northern Italian railway system (comprises of 32 regional lines, 9 suburban lines, 2 express lines) are increasing due to its benefits of alleviating the problems of traffic jams especially in metropolitan areas. In order to improve public transport, railway operators developed the appropriate tools for measuring and monitoring service quality. RSI developed service quality comprising 7 main characteristics of a railway service and other 33 Service quality attributes to measure customer satisfaction and service quality [18].

The National Rail Passenger Survey (NRPS)
The National Rail Passenger Survey (NRPS) provides a network-wide picture of customers' satisfaction with rail travel. Passenger opinions of train services are collected twice a year from a representative sample of passenger journeys. It composed of 2 indicators from station facilities, train facilities, and satisfaction with 33 specific aspects of service [19].

Suvarnabhumi Airport Rail Link (SARL)
The Suvarnabhumi Airport Rail Link (SARL) is a special mass rapid transit rail system project, linking the center of Bangkok to Suvarnabhumi international airport operated by State Railway of Thailand (SRT). The structure of the Suvarnabhumi Airport Rail Link is both underground and elevated, with the alignment accommodating journeys from the eastern and northern outskirts, covering a total of 8 stations, with a distance of 28.6 kilometers [1].
SARL moderately adopted from the guidelines of the European Standard EN 13816: 2002. This service quality guideline considers the needs and expectations of the customer in a comprehensive manner to ensure that remedial action can be taken to improve the quality of service. It is composed of 8 criteria, 27 sub-criteria and 97 KPI [20].

Mass Rapid Transit Authority of Thailand (MRTA)
The purpose of MRTA standard is to establish a framework for service performance targets that are intended to set the service quality levels to be achieved, particularly as regards the provision of train services and the operation besides maintenance to achieve continuous practicable improvements in the quality.
MRTA will use KPIs for measuring performances of the Concessionaire. Methods of measuring the performance are generally divided into 3 criteria, 5 sub-criteria, and 35 KPI. Firstly, direct performance measurement method of monitoring the actual performance of the service from operational records or physical observation. Secondly, MRTA'S mystery shopper measurement of each of the MRTA Survey KPIs shall be carried out by MRTA personnel undertaking a physical inspection of stations, trains, and related buildings. Thirdly, MRTA will carry out the customer satisfaction surveys during the operation services period and use the results of the customer satisfaction surveys to be attributes for making decisions to adjust the performance targets of service quality [8].

Data Integration
To find the suitable performance evaluation, the researcher reviewed the secondary data of the performance measurement standard at international level and Thailand from various sources; such as the database, documents, books, websites, researches, international journals, user manuals, etc. The researchers intensively studied the details of performance measurement criteria indicators, and then compared and analysed using cluster analysis method.
Cluster analysis is a method of evaluation or grouping things of the same characteristics or condition. Researchers would receive new criteria and indicators from integrations of EN 13816, CoMET and Nova Metros, SARL, MRTA and other international performance measurement standards. Researchers will then grouped the standards by clustering and comparing the key performance criteria.
EN 13816 is the main system for integration (criteria, sub-criteria and KPIs) because it has most comprehensive performance measurement aspects of service quality in the transportation sector. It is also the international standard used to measure service quality worldwide. The results of clustering analysis still display the same main criteria and condition as EN 13816 standard, which consists of 8 main criteria as shown in Figure 4.

Implementation and results
After reviewing both international and Thai performance measurement standards, a comprehensive list of performance measurement criteria will be obtained after cluster analysis method.
The result of clustering analysis found 2 additional sub-criteria of performance measurement aspects in term of Availability, Customer Care and new 91 KPIs in addition to EN 13816 which are 22 KPIs in Availability, 10 KPIs in Accessibility, 14  Thailand as a newcomer in Metro Railway system, has tried to comply with EN 13816 standard. This is especially for SARL who has the highest number of KPI at 91 KPIs complied with EN 13816 standard. On the contrary, MRTA might have only 8 complied KPIs but is in the process of intensively developing their own performance measurement model and guideline.

Conclusions
This research introduces a performance measurement standard and service quality of railway operator that is developed on the cluster analysis methodology by reviewing several international practices and Thailand's practices. Starting from the development an integrated performance measurement model from EN 13816, CoMET and Nova, SARL, MRTA and other international performance measurement standards in terms of service quality developed to match with a case study performance. On the part of clustering analysis is evaluated with a total 8 measurement criteria, 31 sub-criteria and 284 KPIs. The additional performance measurement criteria and KPIs are the effort to make the system effective replenishment and to help solving problems that have occurred in the past. The development of current performance measurement standard however, needs supporting technology and regulation of government in order to achieve the defined quality. The outputs of this study will be an early phase of producing a comprehensive list of performance measurement criteria which will be the beginning of the next phase, to gather opinions of the expert panel and develop suitable performance measurement criteria for Thai's railway operators.