Usability evaluation of mechatronic system by primary school children

. The paper examines the ease of use of a mechatronic system by children aged 7-12+ years living in the area of Thessaloniki. The research was conducted on students who participated in educational robotics seminars and as the results show, the trainees consider a mechatronic system very easy to use. The study concluded that older boys in particular were more familiar, than girls, in the construction and the programming of mechatronic systems. Children seem to be excited and this is evidenced by their strong desire to use it often. The area criterion does not make difference to the results. It seems clear that a mechatronic system is easy to use by primary school children. Through educational robotics, children can be taught in the different objects that combine this special tool. Learning takes the form of a game and escape from the narrow boundaries of conventional teaching that is now outdated and without interest.


Introduction
The aim of the paper is to investigate the usability of a mechatronic system by primary school children who use it during an extracurricular activity, educational robotics. But what does a mechatronic system mean? Any mechanical system that combines electro-mechanical parts with electronic devices and is controlled by microcontroller sytem, has been established to be called a mechatronic system. A mechatronic system includes the microcontroller, sensors, actuators and software. The sensors collect data from the process environment while the microcontroller together with the software take control of the variables and perform the control actions on the process. The software provides flexibility in changing the characteristics of the microcontroller, is able to easily and quickly perform calculations and shows better behavior in terms of sensitivity, noise. Finally, the actuators close the control loop of the process and the respective mission is implemented.
In order to be useful a mechatronic system, it must be easy to use. The usability of such a system is considered very important because, if the interface is not easy to the use, the user focuses on how to interact with it and not on its content [1]. In the literature, are different definitions of usability. In international standards, usability refers to efficiency and effectiveness in achieving specific goals and user satisfaction.
"Ease of use: the degree to which a product can be used by specific users to achieve specific goals with efficiency, effectiveness and satisfaction within a specific user framework" [2] The main factor's of usability examined in the literature include ease of learning [3,4], ease of use [3,4], usefulness [4], user satisfaction [5]. This study examines the "usability" of a mechatronic system in terms of usefulness, ease of use, ease of learning, and satisfaction.

State of the art
Usability refers not only to software and hardware products, but to any object or product that is designed and manufactured for use by people (users). Usability has been recognized as an important parameter in the design and evaluation of products, systems and services by researchers with different starting points in scientific fields such as Cognitive Psychology, Applied and Cognitive Applied and Cognitive Applied Economics, Computer Science and Industrial Product Design. Usability is described extensively in important relevant scientific books. Child computer interaction (CCI) [10,11] research has expanded greatly over the last decades, partly due to the realization that children's mental models are different from those of adults. Embodied interactions and body posture have also been analyzed to detect children's engagement and social interactions with robots [12,13]. In another case [14] the interaction of children was analyzed with the help of gestures that showed a good percentage of acceptance of the new means of communication. The children were found to have great motivation for the activity and the ease of interaction was assessed as very positive [9]. The work of Nacher, Garcia-Sanjuan and Jaen showed that in young children and around the age of 6, it is difficult to interact with mechatronic systems. To be precise, most of them could not achieve the final goal, while in the parameter that had to do with the completion time, it showed that it is inextricably linked to the age group. The older children, are more successful on the goal of achieving time. In the same survey in response to (whether children can guide the robot on a predetermined path) and according to statistical analysis, the research question is answered in the affirmative to children aged three or older, as they were able to complete the work with success rates of about 100% [7]. In the work of Vasey et al. they compare the ease of use of an application in an android environment with a remote control, to control an Aldebaran Nao robot-type mechatronic system. The results were positive, pointing to the ease of use and organization of the interface [8]. Ranganathan et al. also studied the interaction with a robotic robot arm. Examining the ease with which children aged 9-12 were treated. Their results showed an impressive effect of age on motor learning. The 9-year-olds had great difficulty with the job, with only ~ 40% being able to complete the job, while the 12-year-olds had more difficulty controlling the robot arm as well as a low job completion rate, longer driving times and travel lengths [14].

Methodology
In this study, a sample of students consisting of 100 children aged 7-12 + years were asked to evaluate the usability of a remote control in relation to an autonomous robotic vehicle (Fig.  1). The evaluation was performed using a weighted questionnaire.

Fig. 1 -Remote control -motor vehicle
A Lego Mindstorms EV3 Infrared Beacon was used to build the remote control from the home edition version and some extra beams-bricks to achieve more stable handling. The Lego Mindstorms EV3 Infrared Beacon controller communicates with the Lego Mindstorms EV3 home edition's infrared sensor via infrared radiation. The commands send to a microcontroller on which are the wheels connected and the mechtronic system controlled. A Medium motor is also connected to an additional output, which is responsible for driving the vehicle by turning the front wheels either left or right. The system is controlled through appropriate programming that runs in the memory of the microcontroller and has been created in a Lego Mindstorms EV3 programmable language environment.
Having built the remote control of the robotic vehicle that interacted with it, it was deemed appropriate to define the research questions. In order to evaluate the overall usability, the following research questions were formulated.
• Did the trainees find the platform useful?
• Did the trainees find the platform easy to use? • Did the trainees find the platform easy to learn? • Did the trainees find the platform satisfactory?
• Did the trainees find the platform easy to use?
The research was conducted with the help of a structured questionnaire as a means of collecting information, with statistical analysis of the results and drawing conclusions as it was considered to be the most appropriate way to answer the above research questions. The 5-point Likert scale was used for the answers in the questionnaire.
Given that the object of the research is to evaluate the usability of interaction with an autonomous robotic vehicle (mechatronic system), the tool used after a thorough literature review is the structured USE Lund questionnaire (2001). The term USE is an acronym for the corresponding English words Usefulness (U), Satisfaction (S) and Ease of Use and Learning (E) and is a tool developed to evaluate software usability. The USE Questionnaire [6] measures the subjective utility (usability) of a product or service. This is a 30-item survey that looks at four dimensions of usability: usability, ease of use, ease of learning, and satisfaction.
The questionnaire, created electronically with the Google Forms tool, recorded the views of 100 children aged 7-12 + years living in the Thessaloniki area and using the mechatronic system as part of educational robotics seminars in which they were in groups of 4 -6 people.
Initially, a pilot survey was conducted in which the questionnaire was given to 5 students who were then asked about the ease and timing of completion as well as in case of misunderstanding or difficulty answering a question or a whole unit of questions. Their observations were useful as some minor changes were made, mainly in the vocabulary, and the average time to complete it was calculated. The USE questionnaire requires information such as age, gender and residential area. The structured USE questionnaire (Usefulness, Satisfaction, Ease of Use) has been proposed by Lund (2001)  These levels correspond to the numbers 1 to 5, where "1" corresponds to "I completely disagree" and "5" to "I completely agree", in order to be able to quantify the variable response. These 30 questions are grouped into the following four categories: The usability of the interface is based on the four categories, calculating the average number of user responses to the questions corresponding to each category.

Experimental results
The research was conducted to investigate the usability of the interaction of learners with a mechatronic system. The sample consisted of primary school children aged 7-12 + years living in Thessaloniki and its wider area and taking part in educational robotics seminars. The study involved 100 students, 61 of whom were boys (61%) and 39 were girls (39%). 9% are in the 7-8 age group, 58% are in the 9-10 category and 33% are 11-12 + years old.
The 30 statements were grouped into 4 categories and composed 4 new variables, "Usefulness", "Ease of use", "Ease of learning" and "Satisfaction". Cronbach's reliability index for each of the above variables was estimated. In order to be valid the answers given by the students, and therefore to be reliable the results and conclusions of the research and impartial, the estimated value of each factor a Cronbach should be higher than 0.6. The Cronbach's index for "Usefulness" was found to be 0.922, for "Ease of Use" 0.942, for "Ease of Learning" 0.902, and for "Satisfaction" 0.953, which shows high reliability, in the sense of internal consistency, of the 4 categories.
From the analysis of the statements that compose the "utility", its average was calculated. A total of 3,745 cases were found, suggesting that students' views on "usefulness" were mild. The standard deviation is 0.730 respectively. For ease of use, there were 3,313 cases, suggesting that students' views were neutral towards a soft agreement. The standard deviation is 0.860, respectively. From the analysis of the statements that make up the "ease of learning", its average was calculated and 3,698 facts were found, which suggests that the views of the students have a mild agreement. The standard deviation is 0.838, respectively. Finally, for the variable "Satisfaction" it appears that the average is 2,954, which suggests that the views of students show neutrality. The standard deviation is equal to 1,000.
In terms of overall usability, an average value of 3,427 indicates that learners tend to have a neutral to mild agreement on "Ease of Use" of the system as a whole. (Fig 2)

Conclusion
From the analysis of the results of the above research, important data emerged regarding the ease of use of the interaction of primary school students with a computer system.
Regarding the overall usability of the computer system, the majority of respondents are positive. They declare, on average, a mild agreement on overall usability, which is in line with other research. With the research of Pons (2019), Read (2013), Hourcade (2015), but also with the research of Sanghvi (2011), Henkemans (2017) and Ranganathan (2019) in the publications of which it seems that there is a positive interaction of the respondents with technology and electronics systems. It is also in agreement with the research of Vasey et al (2017), which is similar to the present research, which points out the ease of use and organization of the interface. Nacher's (2016) study found that children over 6 years of age reported ease of use and learning as opposed to younger children. In contrast to the results of these studies, Ranganathan's (2019) research is inconsistent because the findings of their research show difficulty in using and learning for children aged 9-12. In summary, students 'views on "usefulness" and "ease of learning" are mildly consistent, while students' "ease of use" are neutral in terms of mild agreement. Finally, with regard to "Satisfaction", it appears that the views of students show neutrality.