Augmented reality application for training in pipe defects ultrasonic investigation

The paper presents the development process of an Augmented Reality (AR) application used for training operators in using ultrasonic equipment for non-destructive testing (NDT) of pipework. The application provides workers useful information regarding the process steps, the main components of ultrasonic equipment and the proper modality of placing, aligning and moving it on pipe and weld. Using tablet or mobile phone device, an operator can see on screen written details and images on standardized working method, thus offering assistance during the training process. Allowing 3D augmented visualization of ultrasonic equipment overlaid on the real-world environment consisting in pipes and welds, the AR application makes the NDT process easier to understand and learn, as the initial evaluation results showed.


Introduction
Although the first Augmented Reality (AR) application for training was developed in 1992 [1] for Boeing Company by Tom Caudell and David Mizell (who also coined the term), this field knew notable development only in the last ten years.This can be explained by the improvements brought to hardware equipment such as head mounted displays (HMD), hand held devices, stereoscopic glasses, mobile devices (tablets, phones), tracking systems, sensors and wearable technology.Moreover, the price decreased for all this equipment due to an increase in the number of producers and developers.This started AR technology consolidation process and made affordable the creation and use of AR applications for education and training in engineering [2][3], medicine [4][5], automotive and aerospace for training and maintenance [6][7], etc.
Literature [8] defines AR as a "human-machine interaction tool", which can supplement or overlay "a real-world environment with computer generated sensory inputs" [9].There are several conditions to be fulfilled by an AR application: combines virtuality and reality, takes place in real time and objects are 3D represented in the virtual environment, which offers important advantages such as: enhanced perception of the real world, real-time display of messages for users, integration with 3D CAD systems and interactive modalities to learn and experience the virtual worlds.
In this general context of interest and due to the usefulness of AR application in engineering training, the current paper focuses on presenting the development process of a new application for training operators in using ultrasonic equipment for non-destructive testing (NDT) of pipework.
A survey of the field showed that not many AR applications are available for ultrasonic NDT of pipeline.TWNKLS company specialized in AR created for Applus+ RTD a mobile application that allows operators to see augmented data on defects as coloured volumes superposed on the image of inspected pipe.3D printed AR markers are placed as a ring around the pipe for generating images of dimensions, position and the nature of the defect.This application is dedicated mainly to maintenance work and not to training and it is based on 3D ultrasonic IWEX (Inverse Wave Field Extrapolation) technology [10].Another application is reported in [11] and uses AR glasses for visualizing digital C ultrasonic scans overlaid on a tested object like they form "a unity with the structure".Thus, it simplifies testing results' evaluations, these being usually showed as 2D graphics which are harder to interpret, especially on curved surfaces such as pipes.Hybrid markers are considered by authors more appropriate for the application as they can provide both visual and sound characteristics.Another AR based NDT application (called HaroUT), developed using LabView and Microsoft Hololens is presented in.Ultrasonic delays are calculated by the software and the 2D and 3D plots generated are mapped on a virtual pipe.
The literature review pointed a niche in the field of training applications based on AR technology for ultrasonic inspection of pipework.Therefore, starting from the working flow for such type of non-destructive testing, a mobile application AR marker-based was developed.It allows operators to see, overlaid on a real pipe and weld, different displays of information regarding the operations to be performed before starting the inspection process.The correct placement and movements of ultrasonic probe (using an augmented 3D model) on the pipe and the identification of several types of defect by presenting scans images from a defects' database can be also learned using the AR application.
The developed application was initially tested by 5 researchers from University POLITEHNICA of Bucharest.Further tests will involve a larger group of participants for assessing the application usefulness, learning curve, interface friendliness based on a on a 5 points Likert scale questionnaire.

Materials and method 2.Ultrasonic non-destructive inspection
A typical ultrasonic testing system is composed of several functional units: transmitter/receiver, transducer and a display device.A transmitter/receiver is an electronic device that can produce high-voltage electrical impulses and turns them into high frequency mechanical vibration.Led by transmitter, the transducer generates high frequency ultrasonic energy.Ultrasonic energy is introduced and propagated through the material form of waves.When there is a discontinuity (such as a crack or air inclusion) in the sound waves, a part of the energy will be reflected back due to the defect.Reflected light signal is converted into an electrical signal by the transducer and displayed on a monitor.
By using different types of probes, ultrasonic NDT allows determining and positioning surface and internal defects of various sizes.The method involves the steps presented in Table 1.

Table 1. Ultrasonic NDT process steps
Step 1: Prepare the surface of the object to be explored: -Remove surface non-uniformities (notches, spatters etc.) and paint, dirt etc.The surface finish should be better that 6.3 μm -Apply couplings such as: oil without additive, glycerine or grease Step 3: Establish the level of inspection Step 4: Defects' detection and localization: -Detect the existence of longitudinal defects -Localize discontinuities along material -Establish the depth at which the discontinuity is placed -Establish the existence of transversal discontinuities Step 2: Preparatory operations for inspection -Choose the working frequency -Choose the angle of incidence for wedge probes -Establish the reference level -Establish the evaluation level -Establish the oscillograms characteristics to different type of defects Step 5: Establish defects' shape, orientation and size Step 6: Evaluate the nature of the defect Phased array is one of the ultrasonic testing methods characterized by the fact that computer-controlled excitation is applied to individual piezocomposite elements included in a probe.A steerable and focused ultrasonic beam is generated in phase by transducers which are pulsed independently.Table 2 presents different types of images created using phased array testing method, such as: B-scans lateral scanning, C-scans top scanningbased all on A-scans (amplitude vs. time).There are different ways to display information and the obtained images are dependent of the position of ultrasonic beam, the inspection direction and object sections that are inspected.These are used as 2D images augmented on the pipe during training, the trainee placing and moving the augmented probe on two directions perpendicular on the weld seam in order to examine the whole volume of welded joint.For longitudinal indications, the probe should be moved as in figure 1a, while for transverse indications, the scanning should be made as in figure 1b (for level ground at sound metal) and 1c (if the weld is ground level).For linear and volumetric indications, the probe should be rotated around axis as depicted in figure 1d.

Table 2. Types of phased array ultrasonic scans
During training using the developed application, if the probe is not moved by operators according to the information displayed on the screen, the 2D image will not be formed as augmented on the pipe.
Furthermore, during training, the correct succession of the ultrasonic NDT steps (table 2) is tested.An AR-marker is assigned to each stage and the operator should order them for obtaining the right sequence.

Development of the AR training application for ultrasonic NDT
The training application was built using Unity 5.3.4,a software development package which was preferred for its large database of scripting and 3D visual resources available open-source and for its extensive documentation.
The AR components of the application were built using Vuforia 6.2.10 SDK for Unity.The AR SDK utilizes the main video camera of a portable device (smartphone or tablet) and Unity prefabs to do image tracking, by identifying and tracking custom image markers.For this project, three image markers were created in graphic design software following the design criteria provided by Vuforia.The markers are 60 x 60 mm in size and have been verified to be suitable for use in image tracking applications using Vuforia's Target Manager developer tool.In order to better visualize the ultrasonic probe when using the AR application, an accurate 3D model was designed in CAD software, converted in a suitable format and then imported in Unity.Images of defect visualizations taken during real-world ultrasonic NDT testing are used to simulate the presence of defects in the application.This is achieved by meshing these images on the surfaces of a 3D model of a cylinder which is then made transparent using alpha compositing.The 2D visualizations of defects were provided by Namicon SRL and included in a database of defects which can be accessed by trainees.
For correct target detection, the application requires normal and stable lighting conditions.The application is used in an office/classroom environment that offers these conditions.However, in improper lighting conditions, the application can deploy the camera flash of the device through the setFlashTorchMode() function from the Vuforia API.Probe position tracking is enhanced by using the continuous autofocus mode (FOCUS_MODE_CONTINUOUS_AUTO).Moreover, the application uses embedded algorithms for extended tracking allowing improved tracking performances and sustained tracking even if the marker is partially out of frame.Vuforia SDK uses a predefined dimension for markers specified by the user.The dimension of the virtual model is set to be proportional to the marker dimension.This enables the autoscaling of the AR model when distance and viewing angle change.After the Unity project was fully developed, it was packaged using Android SDK 24.1 and installed on various Android devices for testing.

Results
The developed AR application is focused on acquiring knowledge and training skills: -Knowledge on NDT process steps; -Knowledge on applied ultrasonic NDT inspection steps by correctly placing and moving the probe relative to the inspected pipe; -Knowledge on identifying different types of defect augmented on the pipe.
An end-user starting the AR application can observe on the first page the application logo and four buttons which give access to Basics, AR Testing and About sections, as well as an option to leave the application.Basics section presents the user with basic information about ultrasonic NDT, while About gives information regarding the authors and application development.
AR Testing section offers the user the possibility to watch a video tutorial showing how to use the application and its capabilities, as well as the possibility to start the main feature of the application -the AR ultrasonic NDT training process.
This feature can be accessed by touching the Start AR Training button in the AR Testing section.Touching this button switches the application into the training mode.Upon entering training mode, the main video camera of the device starts and the image tracking features are enabled.Training mode offers an interface which teaches the user the steps for performing ultrasonic NDT procedures.
The user can control the displayed information about NDT steps using three buttons (Show next step, Show previous step, Reset process).The type of NDT investigation to be pursued (B-scan, C-scan, etc.) can be selected from a dropdown menu.Another feature from the training mode is a simulation of the probe movements for each investigation type.This shows the user how the ultrasonic probe should be positioned and moved in order to correctly check for defects.This feature can be accessed by touching the Show probe path button.In order to assist with training, the application creates a virtual model of a pipe.The diameter of this virtual pipe can be modified using a slider.The training mode can be exit by touching the Back button.
The AR experience is achieved with the help of three embedded markers (fig.2): 1. Namicon marker which is used to determine the position of a pipe and creates a virtual equivalent of it; 2. Scan marker which creates a virtual defect; 3. Probe marker which is placed on the 3D printed probe in order to track its position.
To get the optimal training experience, Namicon and Scan markers should be placed on a physical pipe, but the training process can be carried out without a physical pipe, using the virtual pipe created by the application.The user handles the 3D printed probe while watching the screen of the device with the application in training mode.When the probe reaches the proximity of the Scan marker, the virtual defect becomes increasingly visible (thus simulating that the ultrasonic probe is detecting the defect).An initial test with a group of 5 researchers was performed and the participants rated 4,6 out of 5 for the application usefulness and 4,8 out of 5 for interface friendliness.
Several improvements were suggested and they will be considered in the next version of the application, together with feedback obtained from the industry via Namicon SRL.

Conclusions and further work
The AR application presented in this paper was developed as a necessity to improve the learning curve in NDT ultrasonic testing, in the same time decreasing the costs involved in the traditional training.Vuforia SDK for Unity was used for developing the application.A 3D printed model was used as a cheap mock-up of the scanning equipment and probe.
Initial tests were performed by five researchers from University Politehnica of Bucharest.The application was considered useful.However, they suggested several improvements which will be included in the next version of the application.These improvements referred mainly to enhancing the displayed defects database, improve the display of the scan probe correct moving path and increase tracking capabilities.
Further work implies using the assessment data for improving the AR application.A larger group of participants, including also students, will be asked to test the application and give their feedback.

Fig. 1 .
Fig.1.Examples of correct placement of the phased-array probe on the pipe for different indications