Analysis of assessment criteria for selected systems of transport means operation

The problems discussed in the study are connected with evaluation of complex technical systems functioning, in particular, transport systems. It was assumed that evaluation of their functioning depends on fulfilment degree of selected criteria. Therefore, it is important to determine a set of criteria including their type, number and importance. Since the research object is a public city transport system considered to be a sociotechnical system of the type: human (driver) –machine (vehicle) and the environment <H-M-E>, the criteria to be used for the assessment must include behavior of humans, operation of transport means, and the environmental impact. Thus, selection and determination of importance of significant, time variable, measurable and independent characteristics whose values, in a given time moment or a given time interval, define fulfillment degree of the criteria provide the basis for evaluation of such systems functioning. The quality of technical systems functioning in time is assessed on the basis of the criteria fulfilment degree or comparison and classification of different systems of the same type. It needs to be emphasized that the choice of optimal methods for selection of relevant criteria and determination of their impact on the analyzed system functioning is the research subject of study.


Introduction
The choice and specification of assessment criteria for complex operating systems, including the analysed transport ones, from the point of view of their functioning quality evaluation, is an important research problem.A criterion is defined as fulfilment degree of assumed requirements.At the same time, selection of the most important assessment criteria, whose fulfilment degree are values of independent and measurable features describing a given system, was carried out.
The notion of a transport system quality covers a set of its features expressed by means of numerical values, in a given time t or fixed time interval (t+t), determining the fulfillment degree of the specified criteria (its users requirements) [1].
In connection, with the above, in order to build an assessment model for a given system, it is necessary to establish the number and significance of the assessment criteria to be used for determination of characteristics describing the system in terms of its functioning quality evaluation.The assessment process involves referring each criteria from set X to the characteristics they describe and determining, on this basis, whether and to what extent the criteria have been fulfilled in given time t (during assessment).
Having in mind adequacy of the constructed resultant models, it needs to be emphasized that identification and determination of the assessment criteria significance is an important issue in diagnosis and analysis of the system functioning.Therefore, this study uses selected methods for evaluation of the criteria significance and analyzes them in terms of resultant approach to the set of the most important criteria.
A research method and a resultant model for transport systems operation quality assessment has been developed on the basis of literature and the author's own study The developed method enables assessment and comparison of operation quality of different transport systems of the same type.It is also one of rational diagnostic factors of the processes carried out within the discussed systems, depending on value changes of the characteristics describing actions of operators, controlled by them objects, and the environmental impact.For this purpose, metrics to be used for transport system operation quality assessment, have been developed.Values of the metrics described by dependence (1), are determined basing on the values of significant features describing the system, taking into account the values of weights attributed to particular features.A random process is defined for the analyzed system: where: It should be emphasized that determination of a set of the most significant assessment criteria to be used for identification of significant characteristics consistent with the accepted criteria and establishing their significance is an important task for determination of a given system functioning quality assessment [2][3].
Further, in the next section of this paper, methods applied in the process of selection of the most significant criteria to be used for development of a resultant model for functioning quality assessment of a transport means operation system, will be presented.

Research object
The research object is a real, complex, transport system, providing passenger transport services in an urban agglomeration and suburban areas.This is a socio-technical system of the type < H-M-E> (Human -Machine -environment) in which the operation quality assessment is carried out in dependence on changes of values of features describing behavior of operators, controlled by them technical objects and the environmental impact [4].
The basic goal of a transport system operation is providing passenger transport services in a given area, in a given quantity and in a given time.Providing the systems with the required operation quality including efficiency, reliability, safety and economy is the main task in the process of their operating [1, 5-10].
The considered system consists of major subsystems such as: management subsystem, subsystem ensuring operation continuity (providing vehicles with usability, diagnostic, fuel supply), and executive one consisting of elementary subsystems of the type <H -TO> (driver -bus) and the environment as a cooperating system which has a direct impact on the technical system and its subsystems functioning.the one hand, we try to describe the obtained data in the most complete way.On the other hand, we are limited by the survey costs.While analyzing the survey, different problems can be encountered.One of them is the question how to make an optimal reduction of the set of variables without significant loss of information.Popular statistical multidimensional methods do not provide ideal statistical procedures for the best possible choice of a subset of variables.One of such methods is the principal component analysis.However, application of this analysis requires fulfillment of certain assumptions.
This method is carried out on measurable variables (quantitative), though according to literature, it is possible to use it for variables of order type.The examined variables should be linearly dependent on each other and the correlation between features should be measured by Pearson coefficient.If the analyzed variables are not related to each other, then application of the principal component method is not advisable.
Literature provides [4,11] that when all coefficients of correlation are smaller than 0.3, application of the principal component method is not effective.The higher correlation coefficients, the more justified application of this method.
At the beginning of the statistical analysis it is necessary to he use the test of Bartlett [12].Bartlett test answers the question whether all correlation coefficients are equal to zero.Application of the method of principal component analysis requires a test with an adequate quantity.Literature [13] suggests that if the correlations are strong, it is enough for the statistical test to have the quantity equal to 50.An assumption about the distribution normality is not necessary for a description of relations between the variables.However, when statistical tests are used to define significance of the components, the assumption about multi-dimensional normality of the studied features distribution is necessary.

The analyzed data sets are numerical matrixes with dimensions n x p, where n stands for the number of surveys, p denotes the number of criteria. A matrix of correlation coefficients is determined for a given matrix X(n x p). Matrix of correlation coefficients undergoes Bartlett test which decides about advisability of application of the principal component analysis. If R denotes matrix of correlation coefficients, then Bartlett test involves verification of a statistical hypothesis form:
H0 : R = I, (2) where: I − is a unit matrix of dimension p x p.
Hypothesis Ho means that all coefficients of correlation contained in matrix R are equal to zero.Test statistics for this hypothesis has the form: This means that it is justified to use the principal component method.

The principal component analysis enables determination of linear transformation of the form: Z = A X, (5) where:
A is a matrix of p x p dimension linear transformation, X is a column matrix of X T = [X1, X2, …,Xp] T .
Z is a column matrix containing dependent variables Z1, Z2, …, Zp called components.
The principal component analysis determines the first row vector of matrix A in such a way that component Z1 has the maximal variance with the form: Next, the second component is determined so that variance of variable Z2 will be maximal for proper limits.
One of the main reasons for using the principal component analysis is verification of hypothesis of the form: H0 : λk+1 = λk+2 = … = λp, (7) In relations to the alternative hypothesis H1 : not all λk+1, λk+2 , … , λp are equal.Testing statistics for H0 hypothesis has the form: where: χ2 has distribution, χ2 z df = q(q+1)/2 -1, q = n ─ k degrees of freedom for the assumption that the hypothesis is true.Statistical hypothesis described by dependence (7) was successively verified for k=0,1,2,…,p-2.The analysis of data contained in table 2 shows that all the proper values differ significantly statistically, for significance level p < 0.033.Whereas, an analysis of the assessment results obtained from set (N=100) shows that all proper values are statistically different.On this basis, it can be said that the considered set of criteria should not be reduced to their subsystems.Distinct differentiation in the speed of the avalanche curves stabilization, confirms advisability of further study on the differences between the criteria and their statistical dimensionality.

Application of Analytic Hierarchy Process (AHP)
In this study a set of 16 criteria is defined according to scale from 0-10, where grade 10 refers to the most significant criterion.In order, to determine the resultant weight of a particular criterion, a table with comparison of criteria pairs (each with each), has been presented, according to AHP method [14].The number of the verse and column of each element denotes the number of the compared criteria, whereas, a comparison between criterion i and j is, according to the reasearch, a difference in significance degree between criterion i and j, in the element of verse i and column j.
where: aijdenotes value in element i and column j of the comparison table, sijsignificance degree defined in the surveys for criterion j, siisignificance degree defined in the surveys for criterion i.The calculated values change in the range from -10 to 10, and according to the applied method, they should change within the range from 6 to 6 [15].Thus, normalization was carried out on elements of the tables, according to dependence (10), and the results were rounded off to integral values.Comparisons of criteria with each other were not taken into account in further calculations.With such an assumption, a matrix of coefficients and absolute terms describing the system of normal equations was created for the comparison tables.These matrixes are of the following form: where: ilijdenotes the number of grades in elements of verse i and column j for all the tables, iljdenotes the number of grades in elements of verse i for all the tables.According to AHP, the obtained systems are undetermined.In order, to solve them the weight of one of the criteria with value is assumed to be 0, and its corresponding verse is removed.Acceptance of zero value results also in removing an adequate column.For the purpose, of calculation, the weight value of criterion 1 was accepted to be zero (also analogical operations were performed with acceptance of the last criterion weight to be zero, obtaining an equivalent solution).
The reduced systems of equations were solved with the use of 'scilab' program.In this way, these solutions have been received.The obtained solutions can be normalized, by value 10 being added.Assessment of the criteria significance was performed according to the SMART method.This assessments can be interpreted according to the table 3.For calculations of a transport system functioning quality it is better to use the weight of criteria established for the AHP method as their values approach to 1, which reflects relative degrees of the criteria significance in relation to each other.For this purpose, a transformation was made, according to the dependence described in work [ Whereas, it seems that a better method for reduction of the criteria number is to take into consideration only three criteria whose weight is larger than the half of weight of the most significant criterion.The considered criteria are presented in the tables in italics.In further research on the subject, of a given transport system operation quality, the determined criteria will provide basis for its assessment whereas, their fulfillment degree will be crucial for the analyzed transport system operation quality.
In connection, with the above, for further research on the analyzed transport system operation quality, the following criteria have been accepted: safety, efficiency, availability, ergonomics, punctuality, damageability and reliability.

Conclusions
In this study, methods for selection and determination of significance of criteria necessary to carry out the assessment process of the research object functioning quality, have been presented.On the basis, of the obtained results, it can be said that, each particular, method yielded divergent data -criteria sets with different significance.
It should be noted that sets of the most significant criteria have been distinguished on the basis, of applied statistical analysis and the AHP method.However, the results obtained with the use the PCA method show that all eigenvalues are statistically different.On this basis, one can conclude that the considered set of criteria should not be reduced to their subsets.A distinct differentiation of the avalanche curve stabilization speeds (Fig. 1) shows that it is advisable to carry out further research on particular criteria and analyze their statistical dimensionality.
However, it is possible to determine one subset of significant criteria, representative for this study, by analyzing particular results and comparing them with each other.
Thus, on the basis of the obtained results, it was possible to identify from a set of 16 criteria, the most significant ones: safety, efficiency, availability, atmospheric factors, timeliness, reliability, external factors and punctuality.
In connection with the above, it should be emphasized that in order to evaluate usability of methods it is advisable to compare different ones, as using only one method does not guarantee that it is the most suitable one, and the obtained results are the most optimal from the research point of view.

Fig. 1 .
Fig. 1.Chart of avalanche for the analyzed data set.An analysis of the above chart shows that it gets stabilized along with the growth of the proper value number.However, the stabilization degree is different which has been shown before in statistical tests.Whereas, an analysis of the assessment results obtained from set (N=100) shows that all proper values are statistically different.On this basis, it can be said that the considered set of criteria should not be reduced to their subsystems.Distinct differentiation in the speed of the avalanche curves stabilization, confirms advisability of further study on the differences between the criteria and their statistical dimensionality.
the value in verse element i and column j after normalization.
order to analyze the obtained results, a relative weight drop was calculated for each criterion, according to the dependence: denotes a relative drop in weight of criterion j, in relations to j+l.