Development of Augmented Reality Application for Interactive Smart Materials

The paper presents research related to thermochromic inks printed on textile materials. These types of materials are smart materials, they act as packaging indicators, and we use them to develop a software application on mobile devices. That application identifies the most significant changes in parameters of the goods contained in the packaging, such as temperature changes during the storage and transport (goods packaged in packaging that is sensitive to heat), or such as pharmaceuticals and frozen food, etc. This paper aims to demonstrate the possibility of connecting thermochromic materials and Augmented Reality technology through the development of an application for identifying thermochromic materials changes. The tested samples were printed on the textile with a screen printing technique with conventional and thermochromic leuco inks on various textile materials. The Augmented Reality platform Vuforia was used for developing a software application for mobile devices. The application was developed for the iOS mobile platform. The result of the research is the functional application that actively monitors and displays changes in thermochromic materials due to temperature changes, which identifies the effect of heat on the goods in textile packaging.


Introduction
Interactive smart materials are already being used in mechanical engineering, healthcare and the electronics industries. The possibilities for textile and packaging are still almost at an experimental stage. Interactive smart materials are those that respond to a change in the environment, such as temperature, pressure, UV radiation, magnetic field, energetic impact or moisture. In each case, and for each example, the material's response can be different: the material could change colour, translucency, hardness, or size [1].
Thermochromism can be defined as the reversible change in the colour of a compound with temperature [2]. This thermochromic colour change is quite noticeable, often dramatic and occurs over a small temperature interval. In the last few years, the research on the application of these systems on textiles has picked up significantly which has the potential to unlock commercially important nascent high-tech applications [3]. * 2 2 In the packaging industry, interactive smart materials are used for intelligent packaging systems. These systems are one of the recent technologies that witness a higher growth rate as it provides the consumer with all the information about the condition of food or its environment. Intelligent systems including sensors, RFID, and indicators monitor the quality of food product packed inside, thereby ensuring long shelf life, safety, and better quality of the product. However, thermochromic ink dominates intelligent packaging system as these inks are temperature sensitive devices that change colours based on temperature. Such inks are printed onto a package or labels to convey the consumer a message whether the product is ready to consume, based on the colour of ink.
According to a report from "Allied Market Research", the global smart packaging market will grow to $37,797 billion in 2022, registering a CAGR of 7.8% during the forecast period. The report lists various "smart materials" that enhance smart packages. These include shape memory alloys to control the opening and closing of packages depending on environmental conditions, piezoelectric materials to provide power for lighting and audio features on packaging, smart adhesives that can be used in conjunction with smart labels, and thermochromic inks to show when optimal or dangerous temperatures have been reached [4].
Augmented Reality (AR) as a real-time direct or indirect view of a real-world physical environment that has been enhanced/augmented by adding virtual computer-generated information to it [5]. AR Packaging and AR have been aligned from the inception of early AR technologies. AR allows packaging to move from a static consumer experience to one that provides information, advice, games and entertainment. Through the use of the vast amounts of product and nutritional information available on the pack, information can be scanned and both gamified as well as utilised for practical purposes around how a product should be cooked, stored, ordered and tracked. AR along with smart devices can even display products on the shelf when they are still sealed in the box [6].
In this research, we have developed smart textile packaging printed with thermochromic and conventional inks and AR application for controlling ink state on the packaging. The application was able to recognise the temperature change trough scanning different textile materials and to give real-time information about product state.

Materials
Four types of textile materials with characteristic dimensions 250 x 350 mm were used in this study. For each of the material samples, the following are determined: fabric weight (ISO 3801:1977) [7], number of threads per unit length (EN 1049-2:1993) [8], linear density of yarn removed from fabric (ISO 7211-5:1984) [9] and material composition (EN ISO 1833-1:2010) [10]. The characteristics of the samples are given in Table 1. For the fabrication of the screen, we used thick fibres (T) dimensions 500 x 700 mm and monofilament weave weights of 54 threads/cm. The fabric was attached to the measuring aluminium frame (580 x 840 mm). In the printing process, we used four "Silitec" reversible thermochromic water-based textile screen inks (magenta, blue, red, yellow), activation temperature 31°C and conventional water-based textile printing "Epta Inks" (magenta, blue, red, yellow).

Printing Process
Samples were printed with screen printing using "Screen Printing Machine S550". In the first printing pass, we have printed thermochromic ink, and after drying process in the second pass, we have printed conventional ink. The printing speed was 150 mm/s, the return speed of the squeege and the ink carrier was 150 mm/s, the distance of the exposed emulsion from the substrate was 4 mm, the distance of the ink application from the substrate 2 mm and the distance of the squeege from the substrate was 2 mm. During the printing process, the following ambient conditions were present: temperature 22 ± 2 °C, pressure 101 ± 1 kPa and relative humidity 40 ± 2%. For these measurements, we used Extech RH520A.

Samples
After the printing process, we had 16 different samples (four different textile materials each in four different colour) which were the base targets for AR application development. The samples were printed in the form of QR code so the code can be changed depending on the temperature change ( Figure 1).  } } This is the simple script connected with 3D object "Fresh.obj" that will rotate the object on the scene after reading target. Variable "speed" is set to 10f, which is, in this case, an integer for rotation speed. The rotation of the object is defined for the y-axis, and the variable speed is connected to Time.deltaTime, which is every frame of rotation. The same code with the different variable name is also used for rotation of the 3D object "DontUse.obj". After application testing, we build it for iOS 10.1, and in Xcode, we finished the application setup and developed it for the iPhone device.

Vuforia
For creating AR targets, we have used the Vuforia developer portal. In target Manager, we have created a database for Unity with two target images that were quite similar, but both of them get a 5-star rating in the readability test ( Figure 2). The database was downloaded for local device use in Unity Editor development platform.

Xcode
To publish the application, we have used Xcode 10.1. After building a project in Unity, we have opened it in Xcode and in "Info.plist" we have added "Privacy -Camera Usage Description" string with a value describing that user needs to give the app permission to access the camera. This is the important step cause without this string, the application will not be able to access the camera, and it will be stuck in black screen. The application was successfully published on iPhone X, iOS version 12.1.4. (Figure 3).

Discussion
The application was able to recognize codes and show information for all 16 samples. Depending on the sample colour, and material type application had different performances in reading codes. Sample 1 (cotton fabric) with the code printed in blue show the best reading performances, while Sample 4 (cotton polyester mixture fabric) show the lowest reading performances. In the reading, process application was not able to read more than one code in the moment of reading, and it was difficult for an application to recognize live temperature change.

Conclusion
The result of this research is a working application that can be used for identifying thermochromic materials changes. After application testing, we concluded that using lighter colours and polyester materials with low fabric weight can be problematic for reading codes in comparison to darker colours and cotton materials with high fabric weight. The design of the code can be the part of the next research cause although codes showed 5-star rating in Vuforia readability test their similarity in design caused difficulties in recognizing live temperature change. Moving the camera away from reading object and bringing it back show improved reading performances, so we conclude that updating the application code with refresh rate can also improve reading in this specific situation.