A TRL Assessment Method Considering Reliability Requirements

. In addition to considering the functional performance characteristics requirements of the product, reliability requirements should be also considered in the current product design. However, the reliability requirements of products have not been fully considered in the assessment method of TRL. This may lead the result of TRL assessment distortion and cannot help developers identify project technology risks accurately. Aiming at this, this paper improves the traditional TRL assessment method, and proposes a new TRL assessment method that considers reliability requirements. First of all, we expound the method of CTE recognition. Secondly, the TRL definition which incorporated with the reliability requirements has been proposed. Thirdly, the criteria of TRL has been put forward. Then the assessment method of the TRL has been given. Finally, the influence of reliability requirements on TRL has been described.


Introduction
Technology Readiness Levels(TRLs) are a systematic metric/measurement system that support assessments of the maturity of a particular technology and the consistent comparison of maturity between different types of technology [1] . There are several advantages by assessing technology readiness levels. First of all, it is possible to assess the state of technology development more objective. Moreover, TRL assessment can also help developers to identify project technology risks more scientifically and prevent the resulting adverse consequences.
When people develop and produce the product, they consider both the functional performance characteristics requirements and the reliability requirements. The reliability in this article includes reliability, maintainability, supportability, testability, safety, and environmental worthiness. However, in the assessment of the TRL, the current formulation of TRL criteria always considers the technical status, technical carrier, verification environment, process design, manufacturing process, equipment and facilities, cost management and so on [2][3]. They do not fully consider reliability requirements [2]. As a result, the reliability of the system entities integrated by the technologies to be evaluated may not be ensured. It will make the result of TRL assessment distortion and cannot help developers accurately identify project technology risks. Because of this, the system faults will increase easily and its service life will decrease and the actual needs of the product will be not satisfied.
According to the problems above, this paper first expounds the method of Critical Technology Element (CTE) recognition. Secondly, we adjust TRL definition which is incorporated with the reliability requirements. Thirdly, on the basis of the TRL definition above, the relevant contents of the reliability elements are added and the TRL assessment method was proposed. Finally, the influence of reliability on the assessment results of TRL is described.  When constructing the Technology Breakdown Structure(TBS), the Work Breakdown Structure(WBS) should be determined firstly. Based on the above WBS, people should identify the technologies used to implement products, establish the relevance of products, and finally determine the TBS [4]. The schematic of TBS is shown in Figure 1.

Identify CTE
On the basis of the TBS, the technology should be screened and then the CTE can be identified. The CTE should have both the importance and the high risk [5]. Importance means that the technology has a significant impact on the product's functional performance, development progress, life cycle cost, or it plays a significant role in promoting the independent innovation and overall level of science and technology.
High risk refers that the technology must meet the requirements specified in the agreement in the actual use environment.
After forming a list of CTE, experts should be organized to review them to finalize the CTE.

TRL definition
The definition of TRL always presents the degree which the CTE maturity degree meets the project's expected goals. It is usually used in the process of TRL assessment to determine the level of technological maturity of CTE.
In this paper, we supplement the traditional TRL definition [1,2,4,6]. The definition of TRL in this paper adds the factors related to the reliability requirements based on the definition of the original TRL definition, so that the designers can consider the reliability requirements when they determine the TRL of the CTE in initial judgement stage of TRL assessment through the definition of TRL shown in this paper. According to this, designers can effectively reduce the distortion of the assessment results caused by the failure to consider the reliability during the TRL assessment of the CTE. It can make the designers control the project's technical risks more scientifically.
The 9-level division of TRL adopted in this paper is defined in the table1.

The criteria of the TRL
When measuring the TRL of aviation products, people should consider either the design and verification of technologies or the circumstances of manufacturing and management. Therefore, when designers establish a TRL criteria, the evaluation elements are refined to three dimensions: technology, manufacturing, and management. The TRL criteria of CTE can be seen in figure 2.
(1) Technology This dimension mainly includes technical design, reliability design and analysis, technical test and evaluation, reliability test and evaluation, technical compliance and improvement, reliability compliance and improvement.
Technical design refers to the completion status of the research and design work that should be completed in Propose technology concept and application assumption. According to this, establish preliminary life profile and mission profile. 3 Finish the concept and the feasibility verification of the requirements in the basic functions, performance and the reliability of the application assumption.

4
Verify the basic functions, performance and the reliability of the intended system in laboratory environment using the breadboard.

5
Verify the basic functions, performance and the reliability of the intended system in simulated environment using the brass board.

6
Verify the basic functions, performance and the reliability of the real system in simulated environment which can sufficiently represent the actual operating environment using the system or subsystem.

7
Fully demonstrate and verify the basic functions, performance and the reliability of the real system in typical application environment which can sufficiently represent the actual operating environment using the prototype of the system. 8 Tests in the simulated use environment with the actual system as a carrier show that the actual system can meet the functional, performance, and reliability requirements by the specifications. 9 The actual system is used as a carrier to verify the actual use of the task and all functions, performance, and reliability can meet the requirements. technical research and development, that is, the coverage of the technical design work, the quality of the work and the progress of the work ("The completion of the work" in epilogue all includes the above three points). Reliability design and analysis refers to the completion status of the work that should be done in the design and analysis of the reliability when researching and developing the technology.
Technical test and evaluation refers to the contents (including the test and evaluation object, the test environment, the test and evaluation results) of the test and evaluation of the technical research and development results and the completion status of the work.
Reliability test and evaluation refers to the contents (including test and evaluation objects, test environment, test and evaluation results) of the test and evaluation of reliability design and analysis results and the completion status of the work during the technical research and development.
Technical compliance and improvement refers to the completion status of technical compliance and continuous improvement work that should be completed during technical research and development.
Reliability compliance and improvement refers to the completion status of the work that should be completed when the reliability is achieved and continuously improved during technical research and development.
(2) Manufacturing This dimension mainly refers to the materials, process design, process reliability design and analysis, process testing and evaluation, process reliability test and evaluation, and manufacturing processes which are involved in the production phase.
Material refers to the extent to which the material meets manufacturing requirements and its availability.
Process design refers to the completion status of research and design work on the processes involved in the manufacturing phase that should be completed during trial production.
Process reliability design and analysis refers to the completion status of research design work for process reliability that should be completed during trial production.
Process test and evaluation refers to the content of testing and evaluation of the processes involved in the manufacturing stage and the completion status of the work that should be completed during trial production.
Process reliability test and evaluation refer to the process reliability test and evaluation of the process reliability that should be completed during trial production and the completion status of the work.
Manufacturing process refers to the reflection status of the manufacturing process of the product on the design intent (reproducibility and economic affordability) of the critical characteristics.
(3) Management This dimension mainly refers to organization, equipment and facilities, customer's focus, cost management, and reliability management.
Organization refers to the number of people whose work are involved in the function, performance, and reliability during the process of technology maturity, the satisfaction status of the workers' technical level and the degree of the organization setting perfection. Equipment and facilities refer to the level of development and application of various equipment which are used in various work of functions, performance, and reliability during the technology maturity process and the construction progress of each facility.
Customer focus is the ability to clearly identify and remember user needs and he ability to target customers [6].
Cost management refers to the completion status of management work of the products' life cycle cost.
Reliability management refers to the completion status of reliability management work during the technological maturity process.
On the basis of the TRL assessment system, designer should formulate the criteria for the level division of each underlying element that is in accordance with the overall hierarchy according to the actual situation of the system or the project. These rules for the level division of each underlying elements will serve as a basis for evaluating technology maturity.

The TRL assessment method
The TRL assessment process of each CTE includes two stages: initial judgment stage and detailed judgment stage.
In the initial judgment stage, experts judge an initial level based on the TRL definition. After that, the assessment enters the detail judgement stage.
In the detailed judgment stage, experts will compare the parameters of the technology with the criteria of the initial level. If all the criteria of the initial level are met, developer need to assess the criteria of the higher level until the CTE meets all the criteria of the level N and does not completely satisfy the criteria in N+1 level. As a result, the TRL of the CTE of is N. On the contrary, if the criteria of the initial level are not completely satisfied, experts should determine whether all the criteria in the lower level is satisfied until the criteria of the level are completely satisfied or when the criteria for the level 1 are not completely satisfied. Then the TRL the CTE can be determined.
The judgement process of each CTE is shown in figure 3.

Influence of reliability requirements on TRL assessment
In this paper, the relevant contents of the reliability requirements are added in the definition and the criteria of the original TRL. The content to be considered in the TRL assessment are increased. Because of it, the severity of the TRL assessment are also increased. Therefore, the TRL which are proposed in this paper will be lower than the original TRL in most cases. At the same time, because TRL provides an important reference for technology development and transition, the increased severity of assessment of TRL can effectively reduce technology risks.
It can be assumed that there are a underlying elements in the TRL criteria. Among these underlying elements, there are b underlying elements related to reliability requirements.(a>b) For this reason, there are a-b nonuniversal quality characteristic elements exist.
It can be assumed that the weight of the i th underlying element in all underlying element of the TRL criteria is ω i . Besides, ω 1,..., ω b refers to underlying elements related to reliability requirements; ω b+1,..., ω a refers to other underlying elements.

Conclusion
Traditional TRL definition and its criteria do not fully consider reliability requirements. This article first describes the CTE recognition method. Moreover, based on the maturity of traditional technologies, the factors of reliability requirements are added to form a definition of TRL. Furthermore, based on the original technology readiness level criteria, the relevant contents of reliability requirements are added and the assessment method of TRL is proposed. Finally, the influence of the factors that add the reliability requirements on the assessment method of TRL is described. Through the analysis, it can be seen that the TRL proposed in this paper can increase the severity of the assessment of TRL, which reduces the risk of the project.
The work we should do in the future is to further refine the TRL criteria and measure the influence of reliability requirements on the assessment of TRL more reasonably.