Expert poll in complex system projects – scientometric view on the trend of redundancy research

When carrying out research works in the economic, political, sociological fields, as well as in the sphere of governance and in management, we widely apply the integral characteristics of a plenty of factors affecting the existence, functioning and further effectiveness of the existence of the object being studied a social phenomenon, the development and extinction socio-political trends, the existence and stability of specific clusters of economic projects or some large technical projects, etc. Since the influence manifestations of different factors are different, for estimation of their influence complex (vector. To assess the impact, an analysis of expert opinions is carried out, often with multiple redundancy. Such an approach can lead not to an increase in reliability, but to the opposite result, certainly leading to an increase in the cost of preliminary analysis.


Introduction
The impact of various factors on the effectiveness of a project, system, or complex object is diverse. To assess their impact, complex (vector) assessments are often used, including multiple attributes, for example, the weight of the impact on project output parameters from this or another factor, the possible duration of exposure or manifestation of the exposure at different stages of the project, the risk of losses caused by one or another factor, etc. [1,2]. In most cases, these weighting coefficients are determined on the basis of an expert judgment with further processing according to certain algorithms.
Different scales (depending on the nature of the object) are used in the formation of an assessment of a particular object [3,4]. Expert evaluations are often given (and claimed) on a ordinal scale [5]. For example, the tasks of ranking and classifying objects, indicators that are taking into account for making a choice of equipment, methods of analysis and so on.
In our work, we also regularly encounter expert surveys -both in the diploma papers of students and (sometimes) when conducting our scientific research. Moreover, the assessment of the consistency of experts ' opinions and, consequently, the validity of the decision made on their basis, is carried out on the basis of the Kendall concordance coefficient [6,7]. Indeed, with a direct rank estimate of the parameters, the formula can be used: where n, m, S are the number of factors, experts and the sum of the squares of the differences in the ranks (deviations from the mean).
With the assumption of equivalent ranks, the formula becomes more complicated, but the fact is that not always the assessment of parameters can and should be carried out by ranking. It is often necessary to evaluate the values of the studied factors, and to form a result on the basis of the resulting "expert sample".

Comprehensive assessment of expert opinions.
The assessment of consistency of experts ' opinions, given not in the form of ranks of significance within a group of several factors, but in the form of an arbitrary digital evaluation of these factors (as in the discrete setting of this assessment with some acceptable step, and in the case of admissibility to set an arbitrary numerical value for each factor), can be carried out by calculating the degree of dispersion of expert's opinions around the averaged value of this estimate [8]. In the case of an estimation on each individual factor (one-dimensional FDVH WKLV YDULDWLRQ FDQ EH GHILQHG DV WKH VWDQGDUG GHYLDWLRQ ı DQG LQ WKH FDVH RI FRPSOH[ estimation we have to determine the value of the mean square distance from the midpoint (the center of mass for the points -experts' opinions in the n-dimensional space variablesestimated factors) to the point-opinions in the n-dimensional space. In the one-dimensional case, either the coefficient of variation is estimated, (i.e. the ratio of the obtained value of ı to the mathematical expectation), or (as a special case of the multidimensional factor for n = 1) the fraction of all points that fall in the n-dimensional sphere of radius ı (or ı with some coefficient) with some normative restrictions on these criteria. Here, the standard deviation of the estimates of the n-dimensional factor in relation to the n-dimensional math expectation is defined as: where D i is the variance of expert estimations for the i-th factor. When specifying the radius of the sphere, it should also be taken into account that, depending on the dimension of space (i.e., on the number of factors), the proportion of points falling into the sphere of radius Nı, decreases.
In the multidimensional case, the analog of the coefficient of variation can also be usedthe ratio of the root-mean-square value of the deviation vector modulus to the length of the vector to the center of mass (from the zero point -the origin.) This approach is illustrated in Fig. 1 with 3 factors and 9 experts (the demonstration is carried out in 3D graphics). The ILJXUH VKRZV WKH VSKHUH RI UDGLXV ı IRU ILQGLQJ WKH QXPEHU RI SRLQWV-indications falling within the acceptable zone around the center (average) point, as well as expert points of opinion that are included and not included in the ı-sphere.
Thus, by analyzing the spread of points (the vector value of expert opinions for the entire set of factors in the group), it is possible to remove those data that deviate significantly from the group's core view (with /often/ a small number of experts, it is possible to admit the absence of clusters of opinions) In addition to this method, it is possible to produce analyze of the variation of each factor assessment by different experts, representing points in the so-called expert space, j coordinates of which are the values for each i-th factor named by individual experts (direct values if all factors are evaluated within the same ranges values, for example, risks -from 0 to 1, otherwise it is more convenient to present in normalized form). At the same time, factors that have received the same values by all experts will receive a point-object on the bisector of the m-dimensional angle, where m is the number of experts, otherwise deviations from the bisector are formed to a greater or lesser extent.
With this interpretation of the data, the coefficient of variation of the expert' estimations on the i-th factor is graphically represented as the tangent of the angle of the i-th vector's deviation from the bisector vector. In the continuous assessment of factors by experts, each individual factor can be considered as unreliably estimated (inconsistency of experts), or unreliably associated with the problem under consideration when the generalized estimation of this factor goes beyond the m-dimensional cone constructed from the reference point {0,0,..., 0}.
However, experts do not actually carry out an accurate numerical assessment of a parameter, but an interval estimate, with a certain step of changing the readings. The error (variation) of expert opinions with its normal distribution and with a small (relative to the average generalized value of this opinion) value will significantly depend on how close this average value is to the boundaries of the specified intervals within which it is proposed to evaluate the parameter under consideration.
For such conditions, the function of the angle ß between the vectors leaving the point shifted from the origin to the unit vector {-1,-1,-1,...,-1} can be used, thus providing an acceptable possibility of spread at small averages (required due to the discrete step of data change). Due to the displacement of the beginning of the vector, the estimate will be more weighted, taking for the evaluation criterion not the coefficient of variation, but the ratio of the standard deviation to the maximum possible value of the parameter. When the pivot point of the vectors is displaced, the resulting tangent of the angle between the vectors indicated earlier passes from the value of the coefficient of variation to the value of half from the average harmonic between the coefficient of variation and the standard deviation: This approach is visually illustrated in Fig. 2 with 3 experts and 7 factors. The figure shows a cone (green) from the point {-1, -1, -1} (with a divergence angle determined by ı and ȣ at the maximum value of the factor estimates equal to 1) to find the number of readings that fall in the allowable area around the bisector, as well as the point-evaluation of factors by experts who fell (blue) and not (red) in an acceptable zone.
The development of a various solutions on both large-scale and small projects is based on the current understanding of the state of society' development, social reserves, social potential in the scientific, ideological, technical, social and other spheres.
Determination of the necessary project reserves, designed to ensure the mandatory achievement of the project objectives, is performed by estimating the probabilities of achieving certain components of the project's objectives vector to varying degrees, with allowable variations of the various supporting components, such as: economic condition of the target area of society, the success of achieving the necessary scientific and technical saturation of both the project and society, and so on. A numerical estimate of such probabilities is made by so-called risk factors.

Primary analysis of excessively detailed expert surveys
In the case of unstable, variable socio-economic base, the calculation of risks would require taking into account a very significant set of parameters with high accuracy of values and given relationships both with each other and with the desired result. In this regard, in practice, developers and owners of complex projects go through expert risk assessment, often dividing them into several components, determined by analysis of the state of certain properties, characteristics of the modern (for the project) society The number of "splits" of risks, set as one-dimensional, linear, and matrix, 2 or more dimensionally, is determined by the authors of such expert surveys quite arbitrarily, based on For example, when implementing projects in the field of transport infrastructure, several groups of risks can be selected out: financial, political, economic, social, fiscal, monetary, commercial, design and construction risks, business event risk, force majeure, etc. And for each group, a request is made for many components -the risks of influence from local and global communities, administrations, underestimates of the hodograph of the global development of the country, including poorly predicted inflation, changes in legislation, etc.
Without going into a detailed description of the components, consider the problem of the quality of the result of the expert survey on the other hand. Let us ask ourselves -how is it better to conduct an expert survey -to request from experts as much data as possible or less (within reasonable limits)?
The answer to this question is determined not only by the fact that a qualitative expert study requires funding related to the amount of work carried out by the expert.
Let's introduce the concept of conceptual-semantic space, within the framework of which analysis and decisions are made.
The fact is that experts who make some evaluative decisions work within the uncertainty of both the parameters of the estimated base and the boundaries of the areas of individual concepts with a focus on which the risk assessment is conducted. For General reasons, it is believed that the more evaluation coefficients, the values of different risks will "give" the expert Advisor, the better and more accurate will be the final integral evaluation.
However, certain gnoseological considerations show that this is not always correct. The qualification of experts may be different, and the more necessary to estimate the number of (connected) parameters and below civilized society's potential, the more blurred will be the resulting estimates, the variation of which may lead to the fact that the integrated assessment will have less credibility than in the case of shortened list of estimated risks. The reason, in addition to the qualifications of experts and the indispensable system error of the method, lies also in the fact that the conceptual boundaries within which an assessment of a particular risk is conducted can also have a significant variation, and the smaller the covered area, the greater the degree of uncertainty, the intersection concepts and, as a result, mistakes in determining the estimated risk.
As an example, we can consider the number of main named colors in the rainbow at different levels of civilization, in different countries -from 3 to 7 colors (ancient Russia -4, some tribes in Indonesia -3, ancient China -5, the modern world -7, etc.). Another example is the gradual conceptual division of the science of society, some science directions. The splitting off for separate concepts-sections: Politics, Economics, Sociology... For example, the term "sociology" was introduced into scientific circulation by Politics, as a term and concept, emerged from the philosophical knowledge of the world earlier, but later than physics or mathematics, which in ancient Greek times were united by a common notion of philosophy. Figure 4 shows the approximate timing marks for the formal separation of sciences and the base countries at the conventional time axis from left to right. At the same time, with a small number of zones, they (it) may not be located accurately in the conceptual and semantic field of the expert or not completely cover the entire field of concepts, and therefore the reliability of the integral risk assessment may also decrease.
To demonstrate this, let us represent the above-mentioned conceptual and semantic space in the form of a plane (shown in figures in brown). With one coverage area (Fig. 5), incomplete or inaccurate coverage is possible and, consequently, underestimation of possible consequences. At increase in number of splinters (Fig. 5), the number of zones of risk decision-making increases the degree (coefficient) of the coating, increasing the reliability of the integrated risk assessment (orange column on the right), but the reliability in each of the zones is gradually decreasing and increasing degree of overlapping of concepts, "centering errors"which reduces both the reliability of individual components of the complex risk vector, and the generalized integrated risk assessment.

Conclusion
Based on the above, it can be stated that the trend of conducting very expensive and lengthy expert interviews with super-detailed elaboration of the levels of questions is not always justified, even if the projected event involves the allocation of significant resources, both financial, time and others. Determining the rational level of elaboration of the conducted surveys with subsequent analysis is a separate task.