A case study of robotic simulations using virtual commissioning supported by the use of virtual reality

Virtual Commissioning (VC) is a method and tool for verifying and testing the PLC control program on a virtual digital model of the manufacturing system. It allows to visualize and test the control system before the real commissioning of the production systems. The aim of the research is to implement virtual reality (VR) into the VC method and to verify the mutual interaction of signals between the simulation in VR environment, the digital model of the production system and the control system. The introduction of VR in VC increases the concept by adding more realistic visualization and tracking, which extends its validation capabilities. The changes made in VR virtual environment are transferred to the simulation model and can be validated in a real production system. The real production robotic system transformed into a virtual form will be a case study with its verification. Also will be tested security protocols and proven human interaction with the system to control the system through the virtual HMI (virtual user interface) using VR.


Introduction
Virtual Reality (VR) is used to display digital computer simulation. M. A. Gigante said that The VR depicts an illusion in the digital environment and makes it possible to observe this as in reality. VR technology uses 3D stereoscopic and immersive imaging of space using displays where a person is embedded into the environment by sensors for tracking the position of the head and hands [1]. The virtual twin is used to connect a true production system with a digital world that is displayed in VR. So the most important step is to create, test and validate this virtual twin in an immersive virtual environment [2]. VR is used in the research field and used in the engineering field for designing manufacturing systems to optimize the deployment of various workplaces [3,4]. It is also possible to use VR in programming industrial robots [5]. One option is to use and test collaboration between an operator and a collaborative robot in a virtual environment [6]. VR is used extensively for training in moving in a dangerous environment [7] and testing strategies for evacuating people and escaping from dangerous places [8]. Several digital twin experiments have now been carried out using a real flexible manufacturing system iCIM3000 which have at our laboratories of the Institute of production technologies. Research is implemented and supported by the national project KEGA 021STU-4/2018. The experiments were performed in the ABB RobotStudio software environment, which allows displaying of various CAD models, robotic workstations, as well as creating robotic simulations with a supportive function of displaying such studies in an immersive VR environment using HTC Vive Pro VR glasses [9].
On the Fig. 1A shows the real flexible manufacturing system (FMS) iCIM 3000, Fig.  1B is the digital twin of the real system. The result was an interaction between the embedded VR environment and the classic OLP robot programming (Offline method Programming). Research has shown that this method allows for an immediate interaction between VR and conventional OLP, thus shortening the time for the programming of industrial robots. The experiments have shown that through a VR it is possible to interactively control the digital model of the production system and the actual production system, which is the basis of virtual commissioning (VC) designed to test control systems. Fig. 2. Thus, this testing can be performed not only on the digital model in the software environment, but also in the VR model display environment.

Objectives
The scientific goal is to motivate the continuing research into the VC of production systems into immersive environment of VR and to create a virtual twin [10,11]. Generally, in mechanical engineering, VR is used in product design, production process simulation, workplace system training or new systems design, either from software or hardware perspective [12]. The aim of the paper is to use proven hardware and software solutions (HTC Vive Pro + ABB RobotStudio) and use the virtual environment through VR and immersive virtual commissioning environment (VC) as a concept for visualizing, programming and validating a manufacturing system in a virtual environment. Authors Lee et al. said that VC allows to design a complete production system in a simulation environment including the validation of the control program by HiL or SiL methods [13]. By authors Niehorster et al. VR allows you to test your virtual manufacturing system in a secure environment without the risk of damage with multiple prototypes [14]. The aim is to achieve a true VR experience from the user's perspective in the manufacturing system. One way is to use the VR implementation method in the VC for active collaboration between the VR user and the virtual production system. In the current state the virtual environment is often used only as an observation environment without the possibility of interacting and influencing the production system itself, which runs in the form of a time simulation. The VR uses technologies to provide visual, audio and haptic feedback in a virtual environment [15].
One way to test the VR implementation into VC tools and to have feedback is to use safety equipment and related control logic. The aim is to evaluate the interaction between PLC controlled devices in VR such as break the safety light barriers, opening a security door without any collision or physical injury and controlling the system using a virtual HMI (human machine interface). The result should be validation of security equipment through VC in VR.

Methods
The research method is focused on the testing of the collaboration between the operator and robotics system in the VR system. We would like to quantitatively observe the behaviour of the security system and the control system of the robotics system in VR in the frame of the experimental studies. As already mentioned, the main motivation is to verify the feasibility of this method to shorten the overall development time in the production systems design. One method is the VC tool, which allows the PLC control to be tested on a virtual model in the development phase before the physical design of the manufacturing system. The assumption is that by simulation it is possible to detect unwanted behaviour of the production system, including collisions, well in advance. The real production system will be used for the experiments and its simulation models will be verified in simulation software supporting VC and VR (ABB RobotStudio). VR offers one of the interesting alternatives because it provides visual stimuli that are more similar to the real world. In the experiments, we will explore how virtual reality mediates the interaction between human and the production system. VR is a powerful tool capable of simulating such complex systems with a high degree of immersion. The following section will focus how to solve the problem.

Simulation digital twin model
The first step is to create a CAD model of control elements (Start + Stop + Semaphore) and fence of the whole system. Fig. 3 shows the integration of a virtual model of a real production system into a simulation environment with a new designed safety systems and fences. Fig. 3. New designed safety system with fences, push buttons and semaphore around the FMS.

Logical behaviour digital twin model
Creating the necessary signals to control simulation and security components features and creating control logic is important for the proper system logical behaviour. The production robotic system is designed to run its simulation, where the assembly robot grabs the part and creates a Pick and Place operation. The logic was completed according to safety requirements, when the door is opened, the sensor detect this change and the robot stops immediately. Also, the process is in stop mode and the traffic light is red. When the door is closed, the orange light appears and it is then possible to start the operation with the start button, the green light turns on at the traffic light. The entire operation can be started and stopped by controlling the created start-stop button and assigning the logic to these buttons, Fig. 4. On the Fig. 4 is showed developed and created the logical behaviour of the control system, consist of the Joint sensors, mathematical comparers and Boolean operations. The inputs and outputs from the logical behaviour are generated, transformed and assigned to the digital card of the robotics control system.
First of all, we have addressed the problem of how to transform a virtual realitycontrolled by the jogging function to signal into a real industrial robot control system. One of the options that worked after research and testing is to create a mechanism and import an analogue sensor into the mechanism itself. The principle of operation is in monitoring the position of the created kinematic mechanism through the encoder. The instantaneous changes in value are compared with the required values and are immediately evaluated accordingly. On the Fig. 4 at the right side, it can be seen that the created mechanism works and can be tested in the creation phase of the workstation through offline simulation too.

Virtual reality digital twin model
At this point, an overall visualization of the manufacturing system for displaying in the immersive VR environment has been created by supporting ABB RobotStudio software and testing by the VR glasses HTC Vive Pro. Subsequently, the method of testing through experiments was chosen, where the interaction between the digital twin of the production system iCIM 3000 and the control system for security equipment was monitored, thus verifying and validating the virtual model itself, as shown in Fig. 5.

Results
Testing the interaction between operator and production system can be evaluated as successful. The interaction between the human "avatar" and the virtual workstation of production system took place in a VR simulation process, where the human factor affects the running simulation by interrupting the safety circuits, by the open the door or pushing the STOP button which results in a STOP simulation through haptic controller HTC Vive Pro. In the virtual system of the production system, security buttons are implemented along with a traffic light, fencing with doors etc. They match the HMI that triggers and interrupts the automatic cycle of the production system. On the Fig. 6 are shown the views from VR glasses mirrored into the ABB RobotStudio as the final results of safety door and push buttons testing that are functional in the VR system and the signals are transformed into the robot control system and RAPID program for industrial robot ABB.

Conclusion
The expected outcome of the scientific approach is to verify the method of VR integration into the VC simulation environment through mutual interaction. Extending VR with VC opens up the possibility of validating the manufacturing system environment at a different level in interaction with the virtual production system. The advantage is the safe testing of the robotic system as well as monitoring the production process during operation and from inside the system. In the future, we are considering connecting security zones and emergency Stop to the real PLC module that triggers an immediate stop to all kinematic devices.