Credibility Evaluation of Operational Test Simulation Data under Small Sample Circumstance

It is highly necessary to study how to analyze the reliability of simulation data under small sample circumstance when the number of times live operational test is strictly limited. Based on the analysis of existing test ideas and methods, combined with the characteristics of sequence statistics of uniform distribution, a new method of consistency verification is proposed by constructing the variable-scale differential quotient sequence statistic. The research shows that this method is not limited by the sample size, and the credibility of the simulation data can be quickly judged by MATLAB programming.


Introduction
With the constant development and perfection of modern modeling and simulation technology, it has gradually become an important class of methods for operational testing that the combat effectiveness and operational suitability test of weaponry and equipment are carried out on the distributed networked computer system acquiring the complete solution of the entire process by constructing a real-life weaponry and battlefield environment [1].But the model is, after all, an approximation of the system, or even a simplification of the system.The resulting data must be validated by real test data before it can be used for subsequent analysis and evaluation.However, due to the constraints of the test period, environmental conditions and funding guarantee, the times of live operational tests actually carried out on some large-scale weapon systems is often strictly limited and the resulting data show obvious features of small sample.Therefore, it is highly necessary evaluating the credibility of operational simulation data under small sample circumstance.
The common idea of credibility verification is to compare the difference between simulation and live test results under the same input (test) conditions, and verify the consistency of simulation data and live data, which is called consistency test.According to the probability theory and mathematical statistics theory, consistency test can be understood as testing whether two sets of data obey the same distribution.The commonly used classical significance test methods, such as the Kolmogorov-Smirnov method and rank sum test, generally requiring adequate experimental data, are not suitable for use under small sample circumstance.This paper will improve the test ideas and methods based on the data consistency research results [2] [3] [4], and analyze the credibility of the simulation data in detail.

Suppose
is the data sample obtained through the simulation operation ( M is generally relatively large),


is the data sample obtained by the live test ( n is usually relatively small), and now it is required to test whether the two sets of data obey the same distribution.If the simulation data and the live test of small sample data mixed together, then tested, the results often turn out to be accepting assumptions, which cannot reflect the difference.Because many of simulation information overwhelmed the live test information, even if there are differences, it is difficult to reflect.
In Literature [2], After that, the consistency between the simulation data and the live test data is verified one group by one group.Then the reliability of the simulation data under the small sample circumstance is transformed into the hypothesis test problem of whether the small sample data are subject to the same distribution.Let


will be mixed and sorted from small to large to form a sequential subsample, denoted by . The probability density of x can be proved to be: is the distribution density.In the case that the distribution parameters of ) (x F are known, the sequence of random variables corresponding to the sequential sub-sample is denoted as The random variable k  can be seen as the sequence statistic of the sub-samples taken from the uniform distribution in the [0, 1] interval.Therefore, the consistency problem between the two sub-samples  is transformed into checking the problem of whether the random sequence k  is subject to (0, 1) uniform distribution.
Literature [3] questioned the method of checking that the random sequence k  obeys the uniform distribution of (0,1) in Literature [2], and proved its irrationality by analysis, and then used the statistic ) to analyze the compatibility of the small sample capacity of normal distribution.The physical meaning of ijk K lies in that it uses variable-scale measurements to extract large amounts of information from small samples, measures tiny differences between data, and amplifies these differences to identify different features between them.When the sample capacity

N
, the accurate analytic expression without the integral of the statistic ijk K distribution cannot be obtained, and only the critical values of the test can be obtained through a large number of computer simulation operations, which greatly reduces the efficiency of the test, So it is necessary to improve the above method, giving a simple and feasible method of consistency test.

Improved test ideas and methods
It can be seen from the above that the irrationality of Literature [2] lies in the idea and method of checking that k  is subject to the uniform distribution of (0, 1).In this paper, we propose a new way to test whether k  is subject to the uniform distribution of (0, 1).
Firstly, the theorem proved in Literature [4] is introduced: Suppose the population X is subject to uniform distribution in the interval ) , ( b a , and The statistic with the parameters of i j − and j k − .Then, the distribution density of ijk K can be expressed as: According to the above theorem, the  is sorted from small to large to form a sequential subsample, denoted as , to construct the variable-scale differential quotient sequence statistic ).

So: if
At this point, the test of whether

Conclusion
This paper studies how to evaluate the credibility of the operational test simulation data under small sample circumstance when there is a small sample size of the live operational test data.By analyzing the characteristics and deficiencies of the existing research ideas and methods, a new judgment idea is proposed, which is not limited by the sample size.Using MATLAB programming can quickly determine the credibility of the simulation data, mainly including the following Step: ① Randomly group the simulation data; ② Solve the sequence sub-sample

K
can be solved.Then for a given significance level ,  -quantile, the upper  -quantile or two-sided  -quantiles can be obtained according the rejection region of the testing.Finally the consistency between the data can be determined by comparing the values of 123 K and the quantiles.However, when the sample capacity 4 

k
obeys uniform distribution of (0, 1), then the statistic function of ijk Z can be expressed as:

.
Take the two-sided test as an example, given the significance level  , find the two-sided  -quantiles

Table 2 .
Part of ijk Z