Holistic assessment of intelligent technologies: Top management’s perceptions

Due to the beginning of the Fourth Industrial Revolution based mainly on intelligent technologies used in the production area, rational assessment of their functioning has become the subject of interest of many scientists. The main aim of the article is to present the method of intelligent technologies assessment and to determine the possibilities of its adaptation by manufacturing companies. The following methods and research approaches were used in the construction of the method: integrated method, controlling, desk research, observation of the functioning of intelligent technologies, questionnaire interview. The research procedure assumes the adoption of a three-stage assessment process (measure – module – dimension). The holistic approach adopted in the study is achieved through horizontal aggregation of obtained values of measures and modules. The method was verified for the assessment of an intelligent technological line operating in a large metallurgical company. The substantive analysis of the achieved results allowed to develop a list of recommended activities within modules that have not been assessed positively. It has also become possible to assess the implementation of strategic and operational goals realized through the use of intelligent technological line. The developed final assessment formula will help determine the effectiveness of the activities undertaken (comparative analysis).


Introduction
A number of studies have reported or suggested that Industry 4.0 is a concept characterized by the integration of intelligent machines and systems, as well as introducing changes in production processes aimed at increasing production efficiency and enabling flexible assortment exchanges. Industry 4.0 refers not only to technology but also to new forms of work and the function that workers will perform in a company [1][2][3]. However, it should be clearly underlined that an indispensable element of industrial development is its rational and factual assessment, therefore the issue of the assessment of intelligent technologies discussed in this paper is extremely important and still actual.
Based on the review of the literature on the subject of the assessment of intelligent technologies used by manufacturing companies, there are some gaps in this area. Available publications mainly deal with issues such as the evaluation of the potential of these technologies [4] and to quality assessments of specific production processes using intelligent technologies [5]. Many publications also address the issue of assessing intelligent technologies in manufacturing companies and its impact on the development of Industry 4.0, but without proposing specific solutions (methods, models, tools) that could serve this purpose [6,7].
After analyzing the classical methods used for technology assessment [8,9], it can be concluded that they are characterized by certain limitations, imperfections and that do not fully take into account the specificity of intelligent technologies used in the production area. Existing methods allow for the assessment of technology only in some areas of its functioning (mainly economic and technological) and are based mainly on financial data, they reflect limitedly the specificity of the environment and management needs to be determined by the environment. Available methods do not provide the opportunity to take into consideration the complexity and new features of intelligent technologies used in the production area. After the literature review, not identified the method for assessing intelligent technologies in realtime, which would consider the implementation of operational and strategic objectives of the company.
The research problem formulated in the article is to construct a method that allows for carrying out the most comprehensive assessment of intelligent technologies in real-time.
A thesis has been also formulated, that the application of the assessment method of intelligent technologies used in the production area will allow obtaining information enabling companies to make rational decisions regarding the improvement of the functioning of intelligent technologies.

Method of assessment intelligent technologies
The basis for the elaboration of the solution proposed in the article was: the integrated method [10] and the controlling tool [11].
In previous publications [12][13][14] the author developed a model for the assessment of intelligent technologies based on a two-step approach (dimension -measure). In this paper, its development assumes creating functional modules (Table 1) within the adopted dimensions, consisting of a specific group of measures ( Figure 1). The measure is understood as an indicator specifying the size, quality or value of something. It may have a quantitative or qualitative form. The module is an element of the evaluation system performing a specific function in it. It is easy to use as part of a larger whole.
In order to develop a list of measures, and then to create functional modules, it is necessary to collect key information about the research subject. In the initial phase of this process, an interview questionnaire and participant observation are recommended, which will allow the preparation of an assessment questionnaire. Its final structure will depend to a large extent on technical documentation of intelligent technology, analysis of the external and internal environment of the company as well as operational and strategic objectives implemented by the technology. The questionnaire should be completed by employees inside the company, possessing substantive and practical knowledge about intelligent technology. Thanks to this, it will be possible to determine the value of individual measures and then entire modules and dimensions. It is recommended to select both quantitative and qualitative measures, while their total number should not exceed 100. Each measure should have the most adequate method/tool to determine its value.  Each of the created modules should have selected (e.g. in an arbitrary manner, based on the knowledge and experience of the person/team conducting the assessment or based on the literature on intelligent technologies) specific weights (values in the range from 0 to 1, where 1 is very important). The weights can depend on many factors, such as the type, characteristics and technical specification of intelligent technology, the significance of the measures included in the module for achieving company objectives, etc. Benchmarking can also be used during selecting weights and if it is possible to analyze the processes of functioning of such intelligent technologies in other organizations. The weighting may be done by using heuristic methods or by using the available research tools (e.g. a questionnaire to be completed by the evaluation team). The final value of weight can be, for example, the arithmetic average obtained by completing the questionnaire by all members of the evaluation team.
The big advantage of assigning weights to the created modules is also the fact that if the evaluating team have decided about the slight impact of a particular module on the final result of the assessment, it is possible to disregard it by giving a weight equal to "0". Using the weight approach, it is also possible to obtain a greater objectivity of the assessment (it is useful, for example in a situation where there are differences of opinion regarding the meaning of a given module), also selection of only those modules that are particularly important from the point of view of the operational and strategic goals of the company thanks to intelligent technology.
After creating a list of measures within each of the functional modules (functions of modules should be related to the operational and strategic goals of the company), the method of determining their value should be chosen. Different methods and tools (e.g. financial, statistical, comparative analysis, etc.) can be used for this purpose.
After making a detailed analysis of each measure, determining its value and determining the evaluation (positive, neutral or negative), proceed to calculate the assessments of individual modules. Horizontal aggregation of measures takes place here. In the further stage of the assessment process, the assessments (numerator) and weights (denominator) of the individual modules within a given dimension should be summed up and multiplied by the relevance level of this dimension. In this way, a complete assessment can be obtained using the following formula 1: Referring to modern concepts of management and technology assessments, the author postulates the pursuit of the assessment comprehensiveness (holism). However, it is not possible, when adapting the method to the conditions of a chosen company, to include all possible groups of measures in the assessment. A lot depends on the internal and external environment of the company, intelligent technology being evaluated, as well as the company's priorities regarding the achieving of operational and strategic objectives.
It is also worth noting that a given dimension may also manifest itself in an indirect way in other dimensions, which is consistent with the idea of a holistic approach. For example, in the environmental dimension, there are measures related to the emission of pollution that have a big impact on society (social dimension).
In order to verify the method proposed by the author, it was decided to assess the intelligent technology functioning in a large Polish manufacturing company -Pruszyński Ltd. The research object that served the assessment is a line for forming profiles for steel structures (Figure 2) of the Italian company STAM S.p.A Ponzano Vento (TV). It has been designed for the production (starting from the semi-product -coils) of sheet profiles of various widths and lengths. The length of the line is 82m, it consists of several machines (straightener, punching machine, cutter, guillotine, profiler, feeder).

Fig. 2. The line for forming steel structure profiles
Functions that indicate that technology can be considered as intelligent include: − thanks to equipping the line with sensors and devices, it is possible to collect very large amounts of data and based on carrying out analysis processes -optimization of the production processes; − the line is able to learn from the collected data; − energy saving as a result of the adaptation of the line to energy needs in the production of a given type of product; − in case of any irregularities or failures, the line informs the operator (also telediagnosis option is possible), but if it is able to solve this problem by itself, it does not stop and continues production; − working of the line takes place without human intervention (fully automatically), so it is safe -the operator is only responsible for delivering the raw material and uploading the required order data to the main computer; − while executing the same orders (which have already taken place in the past), the line is able to independently adjust the optimal technical and operational parameters. According to the developed method, the process of assessing intelligent technology in a particular company was carried out. For this purpose, it was necessary to thoroughly familiarize with the characteristics and technical specification of the research facility. This was possible thanks to the management of the company providing technical documentation of the intelligent production line and the author's interviews regarding its operation with a member of the top management, the main technologist and operators who work with technology on a daily basis. The author's participation in the direct observation of working technology was significant, which made it possible to understand what the entire production process looks like while using it.
After conducting an interview with a member of the top management, the collected information was sorted out, and a questionnaire for further research was prepared. The questionnaire consisted of five dimensions within which measures have been contained together with units and were grouped into functional modules. In addition, there are fields for entering weights for individual modules as well as relevance levels for the adopted dimensions. The questionnaire was completed by a member of the top management, the main technologist and two employees serving the line. Taking into account the answers contained in the questionnaires (mainly the qualitative measures), as well as the measurements made by the author, the size of the quantitative measures has become possible to analyze and evaluate them. The list of all measures with units, as well as a description of the research by which their values have been evaluated has been described in detail in the publication [15]. Synthetic results of the research are presented in Table 2. To perform a parametric assessment for the measures evaluated positively ("+") a value of 1 was assigned, for measures evaluated negatively ("-") the value of 0, and for measures evaluated neutrally ("0") the value of 0.5. Afterwards, the measures with the values assigned to them were added together within each module and the sum was divided into the total number of measures in the module. The obtained value multiplied by the module weight is its final assessment (last column). For example, in the module "Linework efficiency" we have 4 measures evaluated positively, 2 neutral and 2 negatively. Thus, the total score is: [(4×1 + 2×0.5 + 2×0) ÷ 8]×1 = 0.63.
Due to the fact that some modules contain quantitative measures that require regular control of deviations, the way in which they were measured, analyzed and evaluated was presented in Table 3.
Based on interviews with company employees, as well as getting acquainted with the aggregate data stored in the IT system, the author was able to determine the base value of the measures. Mostly they were the arithmetic mean of the values obtained from all correctly executed production orders from the beginning of the line functionating. Similarly, acceptable deviations were determined (mainly using statistical methods). Table 3 presents the results of the evaluation for one selected production order lasting 1 hour (production of 134.71 lm trapezoidal sheet). The units of measures are given in brackets. Thanks to the controlling tool, it was possible to compare the measured values with the base values and compare these values with an acceptable deviation. This helped in the analysis and evaluation of the measures, which in turn enabled the calculation of the assessment of entire modules. According to the adopted form of evaluation of measures: "A" means a positive evaluation, "CA" -neutral, and "NA" -negative. In a brief summary, it can be stated that in the environmental and social dimensions all measures were evaluated positively, in the technological dimension, 3 measures were evaluated neutrally, and in the economic dimension, 2 measures were evaluated negatively and 2 neutrally. By regular control the value of deviations of the measures, it is possible to focus on those not evaluated positively and to develop remedial/corrective actions that may contribute to the better functioning of intelligent technology.
Using the formula 1, a complete assessment of intelligent technology has been made: The value obtained (2) can be interpreted similarly as the OEE indicator [16]. It can be an internal measurement tool used to assess the functioning of intelligent technology and help identify areas (modules) in which corrective actions are indicated. It can also contribute to improving cooperation on the operator's line -intelligent technology, as well as better adapt technology to the expectations of the society (employees, recipients, etc.).

Discussion
During the realization of the intelligent technology assessment process in Pruszyński Ltd. no particular difficulties were encountered. Thanks to the good cooperation of the author with the member of the top management, the main technologist and operators servicing the technology, it has become possible to obtain all the necessary data to carry out the whole research procedure. It should be emphasized strong involvement in the process of assessing workers employed in the company, as well as the evaluation team's professional approach to the tasks entrusted to it (e.g. preparation of appropriate documentation). Thanks to this, the author could obtain information interesting for him without much difficulty, as well as determine the value of particular measures. As a consequence, it became possible to develop a complete, holistic assessment of the STAM's intelligent technological line and to familiarize the persons concerned with the results obtained. The author discussed in detail the determined values of individual measures, modules and dimensions, and also indicated the areas that in his opinion, based on the conducted assessment, should be improved in the near future (e.g. employees responsible for serving technology). After a substantive discussion with the members of the evaluation team, as well as a careful analysis of the results achieved, the author recommended actions within the modules that did not achieve the maximum possible assessment.
The main practical advantages of the assessment process include primarily its speed, flexibility and no need to incur additional costs. Therefore, it has been adapted to the requirements of industrial practice, and also allows the assessment of intelligent technologies in a different time horizon.
It has also become possible to connect strategic and operational goals with the functions implemented by intelligent technology. The main strategic goals of the analyzed company were: increasing market share, acquiring new customers and expanding the offer. The operational objectives included: increasing the production volume of profiles for steel structures and increasing the unit profit (per one lm of steel). All of the above objectives were indirectly included in the developed modules. Strategic goals were included in the social dimension, in the "Recipients of products" module and in the economic dimension, in the "Expected economical results" module. Both modules have been highly assessed, so it can be concluded that Pruszyński Ltd. accomplishes the strategic goals set by it thanks to intelligent technology. In terms of operational goals, they were included in the economic dimension in the "Linework efficiency" module, which was assessed at 63%. It can, therefore, be concluded that the company should pay more attention to the execution of current orders, and strive to improve the organization of work related to the preparation of a line for the production of particularly new products (previously not manufactured).
It should also be emphasized that the economic dimension (relevance level: 5) and technological (relevance level: 4) were deemed the most important by the member of the top management and the main technologist. This confirms the classic approach to technology assessment, in which these two perspectives are considered the most important by many authors. In the case of the analyzed technology, the low significance of the environmental dimension (relevance level: 1) is fully justified, as production results in trace emissions of pollutions and the number of wastes are at a low level. Legal aspects are set at the stage of preparing the offer and their fulfilment is a condition for the purchase of technology. The social dimension is playing an increasingly important role in the methods of technology assessment, which was indirectly confirmed by the evaluation team, which defined its significance level as 3.5.

Conclusion
In summary, it is worth emphasizing the validity of the interdisciplinary approach in the assessment of intelligent technologies, which leads to an increase in the level of essential rationality. The need to include in the assessment of intelligent technologies many dimensions, as well as paying attention to the role which played in this process consideration of the operational and strategic objectives of the company, enabled obtaining comprehensive information on intelligent technology and contributed to improving the rationality of decisions making by Pruszyński Ltd. In addition, it is worth mentioning that the applicability of the method was also confirmed as part of the author's work in the European Project [17], whose main goal was to create a service-oriented middleware that could be used, among others, in integrated production systems. This is proof that the thesis formulated in the article is correct.
The importance of the subject matter discussed in the publication seems to be so significant from the point of view of the development of technology assessment methods that it is fully justified to continue research aimed at constructing a universal, comprehensive and coherent method of assessing intelligent technologies. To achieve this, it is advisable