Getting started procedure of a NC machine simplified by the use of a mixed-reality training scenario

. To allow increased manufacturing quality and integration in Industry 4.0, machines have become increasingly complex, resulting in increasingly difficult operating procedures and therefore a longer and more expensive operator-training period. Moreover, to be competitive in a global market where competition is sometimes distorted by local aid, European companies must be innovative and flexible. They must therefore be able to count on competent and responsive staff capable of adapting to the various workstations. The initial and continuous training of personnel is therefore a crucial need today. The arrival on the market of AR and VR technologies makes it possible to imagine new training models generally taking into account the technical possibilities, without rethinking the educational scenarios. The work carried out in this study consists of offering novice users a set of educational scenarios and an augmented reality device for handling a 3D printer. A first work carried out on a small group of students tests the autonomy of the users with this new material. A second exper-iment carried out on 80 first-year engineering school students made it possible to quantify usability using a standardized SUS questionnaire. The results show that the level of usability varies from good to excellent, regardless of whether the user has used a VR headset before. They also validate the transmission of technical skills. To obtain this result, the observed criterion is the effective printing of a part in an autonomous manner. The global work in progress aims at providing relevant training scenarios for the use of machine tools.


Context
Nowadays, competition between companies requires them to produce parts more quickly and with a smaller tolerance range than in the past. Deadline and cost requirements prevent companies from making pre-series and correcting the defaults observed before starting the production. In order to enhance the quality while reducing machining time, machines have become increasingly complex, leading to an increase of purchase and running costs. As a result, the long operator learning period combined with higher hourly rates makes operator training particularly expensive.
To further exacerbate matters, companies in this competitive field suffer from recruitment problems due to the decrease of application for Science and Technology degrees. In France, training in Mechanical Engineering suffers from a real image problem, despite the fact that there are good job opportunities in this specific field. To answer a real social demand, it is urgent to ensure that training evolves towards innovative and effective teaching aid products compatible with the high level of technical knowledge.

Terminology
Following Patel & al [1], we will consider in this paper that Virtual Reality (VR) is a completely computer-generated simulation. We will also consider Augmented Reality (AR) as an actual environment overlaid by simulated objects and Mixed Reality as an actual environment with an overlay of simulated objects with which you can communicate.

Bibliography
A significant amount of studies covers the use of Virtual Reality, Augmented Reality and Mixed Reality in a training environment. They showed that using these new tools can create student-centered zones of development. Nikhitha et al. [2] particularly insist on the increase of motivation for students while Gandolfi [3] mention that these tools can provide a significant opportunity for immersion and will play a key role in future educational settings. However, Miltenoff [4] indicates that « Following the "novelty" impact, only methodologically well-constructed and pedagogically wellapplied VR and AR curricula will continue to retain patrons' attention ». Nowadays, Digital Learning Environments (DLE) question current knowledge and make us reconsider analysis models for teaching-and-learning situations. The description of the didactic phenomena specific to the DLE led to the observation of digital phenomena for the understanding of the learning processes implemented by the learners [5].
Among the tools proposed in the literature, none to our knowledge allows to train students on manufacturing machines. In this study, we will therefore develop a proof of concept (POC) of a tool that helps students to get started with a machine. This tool, called Holoprint, combines a mixed reality headset and an educational scenario based on feedback from teachers.
Holoprint was created using professional situations as an answer to the assessment of expected competency profiles [6]. For this test phase, we focused on studying the terms of use [7] of this device to give answers to fundamental issues that comes with using Mixed Reality. Therefore, we seek to understand how the user takes this new way of behaving and perceiving in a virtual space and how they can accomplish tasks, targeted work-related tasks [8].

Experimental setup
The aims of this study are to evaluate the use of Mixed Reality applied to a NC machine tutorial and also to estimate the benefits gained from using Mixed Reality.
To evaluate the benefits gained, a System Usability Scale (SUS) was used and is presented in §2-1. The NC machine used is a 3D Printer Ultimaker3 Extended and the device is a head-mounted display Hololens2 ( Fig. 1-a). The targeted population is composed of 81 first-year students at the National Institute of Applied Science (INSA) in Toulouse, a French engineering school. The population can be considered homogenous, as it was chosen with specific and well-defined criterions. The experiment took place in a single area, during a practical work class. 8 classes composed of 10 students each took part of this class. The principal characteristics of the students participating to this experiment are presented in Table 1.
First time 3D Printer-user 89 To create the scenario, we considered that students were at most 20 years old, that they knew how to use new technologies and that they all owned a mobile-device. This hypothesis was confirmed as more than half had already used a VR head-mounted display for video games or drone flight control. However, none of them had already used a Mixed Reality head-mounted display.
A game aspect is inevitably associated with Mixed Reality. This aspect is interesting as it accentuates the involvement of users in the task to perform. In order to avoid mixing the attraction due to the novelty of the HoloLens glasses and the interest in the scenario, it was decided at first to integrate the game aspect in an acculturation step and make students try a 3D drawing application ( Fig. 1-b). The students were able to familiarize themselves with the HoloLens glasses and in a second part they used the scenario, in which the game aspect had been removed, to learn how to use the machine.
The scenario developed is overall linear and offers tasks following several items. For example, you are asked whether or not you want to change the spool (Fig. 1-c) and the answer associated can be either yes or no. If the task has already been completed (answer no), you are brought back to the main menu.

Acculturation and assessment
Since the scenario could not work without the head-mounted display and the headmounted-display could not be a teaching tool without the appropriate scenario, it was decided to consider the headset and the scenario as one single tool. The tool is first evaluated on the student's autonomy. The test will be a success if the student is able to start printing an object without any help. The preliminary acculturation step happens with the use of the 3D drawing game and allows students to familiarize themselves with the HoloLens headset. The steps of the experiment are in the following order: 1. Acculturation step using a 3D Drawing Game ( Fig. 1-b 2. Use of the 3D Printer with the Mixed Reality device ( Fig. 1-a) 3. SUS Questionnaire The SUS questionnaire [9] created by John Brooke in 1986 is used to evaluate the usefulness and usability of a developed tool. First, we can note that we used the French version of the questionnaire [10] to avoid any language ambiguity and that the questionnaire SUS is standardized. This "quick and dirty" survey is supposed to -as its name suggests-rapidly and easily get an idea of the usability of our product. It consists of a 10 items questionnaire with five response options for respondents; from "strongly agree" to "strongly disagree". Each question will alternatively have positive and negative connotations, the goal is to get the higher scores from the positive questions, and the lowest for the negative ones. The 10 items are the following: 1. I think that I would like to use this system frequently 2. I found the system unnecessarily complex 3. I thought the system was easy to use 4. I think that I would need the support of a technical person to be able to use this system 5. I found the various functions in this system were well integrated 6. I thought there was too much inconsistency in this system 7. I would imagine that most people would learn to use this system very quickly 8. I found the system very cumbersome to use 9. I felt very confident using the system 10. I needed to learn a lot of things before I could get going with this system The score of each participant is calculated using the SUS method with a 0 to 100 scale (Fig. 3). The overall assessment is made by observing the average scores of all the participants.

3
Results and discussions

Overall experiment results:
The result is positive since most of the students (80 out of 81) managed to start printing an object without exterior help but the headset, which corresponds to a success rate of 98.8 %. This high number can be justified by the fact that a large number of students had already used a 3D printer during previous practical work classes [tab. 1].
However, there is no difference in terms of success between the students that had already used a 3D printer and those who had not. The number of students that had never used a 3D printer is too low to be able to draw a general conclusion, nonetheless in this experiment, the assistance provided by the Holoprint tool (headset + scenario) allows novice students to achieve the same level of autonomy as experienced students.

Tool's usability:
The overall score calculated for the tool has a value of 79.69. We can see, according to Bangor et al. (2009) that the tool acceptability is considered "good" and is close to being "perfect". It is possible to extract a distribution of the answers that led to this overall score (Fig 4). To get into details, we can review two important questions. To the question, "I think I would like to use this application frequently", 76.5% of people chose "Strongly agree" or "Agree", and none chose "Strongly Disagree" or "Disagree". This shows that the students' interest remains existent after using the application for the first time, and so this interest might not be linked to the novelty of the tool.
Concerning the item "I needed to learn a lot of things before I could get going with this application", 95% of people chose "Strongly disagree" or "Disagree". It points out that the helping potential of the application is real, as they don't have to learn anything else to use the product. However, we can ask ourselves, what exactly did they learn, which skill did they gain?
The use of a VR headset prior to the experiment doesn't alter significantly the score obtained, since on average the score of first-time VR users has a value of 80.06 and for non-first-time users it has a value of 78.25. We can therefore deduce from the previous results that the use of the Holoprint is not based on VR prerequisite.

Emerging issues
The usefulness and acceptability of the tool presented is proven by the result of the questionnaire SUS. Following this first work, several questions arise concerning the transmission of skills and more generally the dynamics of transmission: -What have the students learned, what are the skills transmitted? Is additional work required to quantify the skills transmitted? -How to introduce a game dimension (gamify) the scenario to engage the player even more? -Can this work be extrapolated to another type of CNC machine tool? Initial work has been done and an experiment will be carried out on a CN Mécanuméric milling machine of the Charly Robot type. -Is it possible to support students with a cheaper device? A similar application was developed with a smartphone, which is inherently less efficient than the Hololens glasses, to assess whether the reduction in performance linked to the tool used reduces the quality of support. These questions show that it is necessary to continue the study to identify more precisely the advantages and limits of the approach and more generally its potential use with other equipment used in Mechanical Engineering teaching.

Conclusion
In this study, a use of mixed reality in support of the implementation of a numerically controlled machine was presented. The Mixed Reality tool that has been developed consists of a scenario and a HoloLens2 Mixed Reality headset. The experiment presented consists in using this tool with a homogeneous population of 81 first-year students from INSA Toulouse, divided into 8 groups of practical work. This experiment took place through a practical work session during which the students practiced first an acculturation to the tool through a 3D drawing game. In a second step, they used the tool to learn how to use a 3D printer. They ended by individually answering a SUS questionnaire.
The results show that the tool is very well received and that it is easy to learn with it. It should be noted that all except one of the students succeeded in launching a 3D print, which validates the individual test. These results also show that it is not necessary to have already used a virtual reality headset beforehand to use the tool.
The distribution of SUS scores around the mean value of 79.69 shows that the results of the SUS questionnaire are homogeneous and normally distributed with a low standard deviation. This observation confirms the result and also validates the method.
The experiment revealed many ways to improve the application, some of which are already being studied. The work in progress concerns first the possibilities of using a smartphone and the transfer of skills using this type of tool. To complete the study that has just been presented, the use of a similar tool to use a CN milling machine model Charlyrobot will be tested in the beginning of the 2022 academic year during practical work classes.