Homogeneity of the Test Variants in Entrance Exams

The problem of homogeneity of test variants in mathematics, which are used in admission process, is solved in present paper. The analysed data are the results of entrance examination in mathematics at University of Economics in Prague. Different statistical methods (test of hypothesis, correlation, regression) were used for this analyse. These results will be used to further improve of the preparation of test variants in the coming years.


Introduction
For entrance examinations are frequently used the multiple choice question tests. Advantages of these tests: the multiple choice question tests are objective (there is clearly no impact of any subjective factor in evaluation) and results can be evaluated quite easily for large number of students. On the other hand, a student can obtain certain number of points in the test purely by guessing the right answers. This problem was analysed in [7], [16] and [18]. In [7] it was shown that risk of success of students with lower performance levels at University of Economics in Prague is negligible.
The difficulty of test variants for students is poorly measured. If we use the multiple choice question tests, the difficulty of test variants depends also on variants of answers from which the correct answer is chosen, e.g. the example "combinatorial number This problem shall be analysed for the test variants in mathematics which are used for the entrance examinations at University of Economics in Prague.
Test variants in mathematics are generated from a database created by the Department of Mathematics.
These tests are used to the three faculties of the Prague University of Economics (Faculty of Informatics and Statistics, Faculty of Finance and Accounting, Faculty of Business Administration ). Analysis of the entrance examinations in mathematics in 2013 is provided in this paper (similar problems are solved in [2][3][4][5][6], [9][10], [12][13] and [15]. These results will be used to further improve of the preparation of test variants in the coming years.

Entrance examinations at University of Economics
The analysed data are the results of 4614 students in mathematics in 2013. Seventeen test variants were used for the entrance examinations in mathematics in 2013 (these test variants we can find in [8]). Part of one of them is on Figure 1. The test variants have 15 questions, each question has 5 answers -see Figure 1. Between these answers is the answer "e) another answer". If e) is correct answer, to mark the letter e may be for the students more complicated than to mark the letters a, b, c, d. For example solution of the question 1 in the test variant A7 (see Figure 1 7 7 i.e. the correct answer is e). The question 1 in the test variant A7 with variants of answers e.g.
e) another answer may be easier for students. Therefore we shall study the dependence of the number of points in the test in mathematics (denoted Y) on the number of answers e) in tests variants (denoted X).

Results
The number of answers e) in tests variants which were used for the entrance examinations at University of Economics in Prague in 2013 is 1, 2, 3, 4 and 5. Now we shall compare distributions of number of points in the test in mathematics in test variants with 1, 2, 3, 4 and 5 answers e) -see Figure 2 and Table 1.
where P 1, P 2, P 3, P 4, P 5 is mean number of points in mathematics in test variants with 1, 2, 3, 4 and 5 answers e), i.e. mean number of points in test variants with 1, 2, 3, 4 and 5 answers e) is the same.
To verify the validity of the hypothesis we use ANOVA. In the first step we verify assumption of this method by Bartlett's test (see e.g. [1]). The hypothesis "variance of number of points in mathematics in test variants with 1, 2, 3, 4 and 5 answers e) is the same" is not rejected at 1% significance level, assumption of ANOVA can be considered to have been met.
Results of ANOVA we got with MS Excel (e.g. [14])see Table 2 ! F the null hypothesis (1) is rejected at 5% significance level. There are some differences between the test variants with 1, 2, 3, 4 and 5 answers e), the differences between average number of points in test variants with 1, 2, 3, 4 and 5 answers e) in Table 1 are statistically significant.  From results of ANOVA it seems that Y=the number of points in the test in mathematics depends on X=the number of answers e) in tests variants. From Figure 3 it seems that dependence is linear, therefore we shall compute the correlation coefficient r and regression line . bx a y

Source of variability
Using MS Excel we have 112 . 0 r From this result follows that the dependence of the number of points in the test in mathematics on the number of answers e) in tests variants is weak or the dependence does not exist. Therefore we shall test null hypothesis Ho: U=0, i.e. the number of points in the test in mathematics does not depend on the number of answers e) in tests variants. We shall compute -see e.g. [17]

Conclusion
The differences between number of points in mathematics in test variants, which were used for the entrance examinations in mathematics at University of Economics in Prague in 2013, were analysed in present paper. The differences may arise due to the varying difficulty of the test variants, but also because of the different level of knowledge of students who write these variants. From [11] it follows that differences between results of tests variants may arise due to different level of knowledge of the students who wrote these variants. On the other hand, the difficulty of test variants for students is poorly measured. This problem was solved in the present paper. From results of the paper it follows that the number of points in the test in mathematics depends on the number of answers "e) another answer" in tests variants, but the dependence is very weak.