Effective Robotics Education : Surveying Experiences of Master Program Students in Introduction to Robotics Course

Technology-driven world poses new challenges for the modern education system. To prepare skilled specialists for academic and industrial needs it is important to create competitive educational ground. Our team works on developing and implementing world-class master program in Intelligent Robotics. To pave the way for a highquality educational program we invest efforts into studying students’ attitude and motivation for connecting their professional life with robotics. In this paper we describe the curriculum for master program that was designed and implemented at the Higher Institute of Information Technology and Information Systems at Kazan Federal University and present the results of our continuous research of comparative analysis of surveys among students of Introduction


Introduction
The current state of higher education in robotics in Russia is at the stage of a rapid development, thus both the scientific and the industrial communities are experiencing a skilled labor shortage.There is a long growing list of industries that continue to automate their manufacturing process and introduce robots to make their production more effective, but still companies feel an imbalance between qualifications of recent graduates and requirements of industry.This problem raises new challenges for the modern educational system.
In 2017 before opening a new master program track in Intelligent Robotics at Higher Institute of Information Technology and Information Systems (ITIS) at Kazan Federal University (KFU) we started our research on robotics engineering education by conducting surveys among bachelor and master students.The surveys targeted bachelor students of Applied Informatics program, who had selected robotics courses as elective courses, and master students of Software Engineering program, who had a compulsory course in robotics research [1].We collected and analysed data from bachelor and master students of different robotics courses through a set of surveys [2], and the analysis of student responses helped determining methodological directions for implementing new teaching methods throughout all courses of Intelligent Robotics program.
We explored existing robotics and mechatronics educational programs in Russia, and unfortunately the majority of them are currently behind similar programs of developed countries [1,3,4].Realizing all consequences of such marginalization Russian government has been focusing recently on the innovation-driven growth of the country.Russian Federation Government enacted Decree № 2227-r on Strategy for Innovative Development of the Russian Federation until 2020 [5].According to this document besides improving of engineering education quality it is necessary to focus on development of students' personal attitude to life and facilitate elaboration of such qualities as flexibility, mobility, lifelong education capabilities, propensity for entrepreneurship and risk acceptance.
The key priority for our team is to create and implement internationally acceptable top level master degree program, which will fill the gaps of engineering education in the field of intelligent robotics.The proposed program is intended to provide students with both knowledge in robotics research and abilities to perform applied projects independently and in teams with other students.
While creating a new educational program, we were aware of the fact that robotics is a relatively new and rapidly evolving field of science and education, which could give us more freedom and flexibility in designing the curriculum, unlike traditional fields like applied mathematics or physics [6].But on the other hand we had to consider students' expectations from the program and their attitude to robotics field.These required to conduct series of detailed surveys among the students who intend to study robotics in a way that we could outline a concept of motivation and level of interest to relate the students' future career to robotics field.
In our previous work we have published our results of surveys among Spring-2017 bachelor students of Introduction to Robotics course that was conducted before opening the master degree Intelligent Robotics program, MSc-IR [2].Continuing our research, this paper introduces the results of survey analysis of the first year students of master's degree program, which was conducted during the first semester of Fall-2017.
2 Engineering education at ITIS KFU The curriculum covers two years of study and consists of twelve special subjects that are taught in English.Within Fall-2017 (the first semester of the program) we had delivered three robotics core MSc-IR courses: Introduction to Robotics, Robot Operational System and Computer Vision.Teaching courses in English will allow students to be acquainted with international academic terminology of robotics field and to keep pace with their foreign mates.The program is premised on the involvement of mathematicians, physicists and computer science specialists from KFU, as well as invited Russian and foreign professors for teaching.
Features of the proposed program are: -Full-time education; -Robotics courses are taught in English; -Most of the courses include intensive teamwork; -Home assignments require to use not only various simulators, but robotic hardware that ranges from LEGO robotic kits within the first semester to Robotis OP-3 humanoids within the third semester for practical evaluation of gained theoretical knowledge; -Opportunity to make a research with real robots at the hands-on laboratories under a tutor supervision.Key priority of the track is creating and implementing globally acceptable robotics education.It is expected that besides the lectures and practical courses during the program students will carry out their projects within research laboratories.

2.2
Research projects integration into educatonal process The problem of engineering staff shortage in Russia is a significant barrier for broadening a variety of educational programs and particular engineering and computer science courses in Russian academy.One of the reasons for the lack of human resources in robotics is conservatism and inflexibility of the higher educational community, its inability to meet the challenges posed by a rapidly changing world.In 2016 Institute for Statistical Studies and Economics of Knowledge (ISSEK) of the National Research University Higher School of Economics (HSE) carried out a survey among robotic companies.The study had revealed that for all engineering staff categories (researchers, engineers, and technicians) there is an imbalance between the qualifications of recent graduates and requirements of industry.Beside the problem of qualification, the study of HSE ISSEK had emphasized that robotics industry experiences lack of specialists: 61% of the survey participating companies indicated their need for professional human resources and none of companies had an engineering staff excess [7].To some extent such problem is a consequence of deficiencies in higher education.
Nevertheless, the overall picture of the engineering education in Russia, and particularly in robotics, demonstrates some improvements, including: -Last decade activity of Russian schoolchildren in robotic creativity has significantly increased [8]; -Some Russian universities have started to drive growth in robotics field and collaborate with national and foreign colleagues [1,9]; -The government started supporting the interest to technical innovations and entrepreneurship among school, college and university students; -Emerging robotics laboratories provide students with opportunity to carry out their research projects.
Research, being integrated into the educational process, provides learning process with opportunity to engage students in carrying out self-guided study and practice their theoretical knowledge.Considering the fact that robotics is a truly multidisciplinary field, such approach allows students to choose their particular direction of interest within the broad and fascinating robotics fields [10,11].Currently in the Laboratory of Intelligent Robotic Systems students of robotics master program undertake robotics research projects in the fields of aerial swarms, mobile robotics, urban search and rescue robotics, path planning, simultaneous localization and mapping, humanoid robot locomotion, manipulation, robotic surgery and others.Matlab and Gazebo environments are used for algorithm prototyping and simulation, while experimental work is performed with crawler and wheeled mobile robots (e.g., Servosila Engineer [12], Unior [13], PAL Robotics PMB-2 [14]), DJI Phantom drones, humanoids (e.g., AR-601 [15], Robotis OP-2 and OP-3 [16]), KUKA manipulators and other robots.In addition to experimental work, we encourage our students to present their results through scientific publications and provide them with supervision in preparing their research papers.
We continue to implement interactive teaching and learning strategies in the classes and integrate robotics research into the educational process, because such approach would prepare students for carrying out serious applied robotics projects and give them a chance to be well-prepared both for academic or industrial career as researchers, developers and team leads.

Curriculum of Introduction to Robotics course
Introduction to Robotics is a core course of the MSc-IR program, which is taught in the first semester of the program.This course is intended to provide students with an understanding of the basic robotics concepts and principals, to introduce the recent applications and prospect of this field, and the main idea behind this course curriculum is to sparkle student interest in robotics and to help them selecting a research topic for their master diploma.The course includes the following topics: -Robotics in Russia and abroad introduces the history of robotics, explains the role of robotics in the modern society and demonstrates various attractive examples of practical applications of robotic advanced technologies around the world.Special attention is paid for robotics R&D in Russia so that for each foreign application example a local equivalent is demonstrated, whether it succeeds to over perform the foreign application or not.
-Introduction to industrial robotics familiarizes students with types and particular industrial applications of robotic manipulators.
-Linear algebra and coordinate systems is a brief review of linear algebra topics that concentrate mainly of matrix operations, while coordinate systems include different representations of translation and rotation operations, homogeneous coordinates, Euler angles, quaternions and gimbal lock problem.
-Forward and inverse kinematics of manipulators covers kinematic principles of robotic arms and teaches students to solve forward and inverse kinematics problems with analytic and numeric approaches.Students familiarize with degrees of freedom, workspace, properties of robots, Jacobians, velocities and static forces, manipulator singularities, etc.
-Trajectory generation introduces path description and generation, joint-space and Cartesianspace schemes, geometric problems with Cartesian paths and path generation at run time.
-Manipulator-mechanism design discusses manipulator design dependence on task requirements, kinematic configuration and actuation schemes, position and force sensing.In addition, a set of introductory lectures cover a number of selected topics such as military and defense robotics, roboethics [17], search and rescue robotics [18], space robotics and path planning [19].The main course book for theoretical approaches and problem solution is a classical book of J.J.Craig [20], while for simulations in Matlab with Robotics Toolbox we use Corke's book [21].In addition to pen-and-paper home assignments and coding in Matlab, students have a number of presentations, including interactive demonstrations of their Robotics Toolbox manipulators [21], and a final project.LEGO Mindstorms EV3 robotic kits and LeJos programming language are utilized in order to practice theoretical knowledge with a hardware, and for majority of participating students the first experience in their life to face engineering problems and gain insights into a drastic difference between pure software coding and dealing with hardware issues of a robot.In order to encourage active learning, students are asked to discuss particular questions in small groups in the class in real time and then present their joint solutions for all course participants.

Research method
This research data section presents analysis of Introduction to Robotics course.Students studied the course for 3 hours per week during 18 weeks in the 1 st semester.The two surveys were conducted: an initial survey took place in the beginning of the classes, just after the first lecture, and a final survey was run in the end of the course, just before the final test.The target group consisted of 11 students that were successfully enrolled to the master program and participated in the course, however, unfortunately, not all students volunteered to respond each of the surveys.9 students responded to the initial survey and 10 students responded to the final survey.To consistently observe dynamics of the students' progress we selected responds of 9 students that participated in both surveys.However, as we target to evaluate students with only technical background, we excluded a student with a BA in Public Policy, thus decreasing the respondents number to 8.Among the selected respondents 5 had BSc in Applied Informatics, others had BSc in Physics, Information Security and Gas-Turbine Engineering (Fig. 1).
We applied the same research methodology that we had utilized previously [1,2] and provided students with questions related to English language comprehension, self-efficiency, active learning strategies, significance of studying robotics, and stimulation of learning environment.This time two questionnaires were developed for the two surveys and each one consisted of two parts.Both surveys were conducted in Russian language to guarantee that all respondents absolutely understand each and every question and their subtle differences.The initial survey contained 48 questions where the first part contained questions basically related to their background, knowledge received before starting the class etc.The second part contained questions that were related to the students' expectations from the course.
In the final survey we suggested 42 questions where the first part consisted of the same questions presented in the initial survey (i.e., the identical questions) and the second part was related to the experience that was gained by the end of the course.The survey targeted at students' background, English language, and motivation to study robotics.The questionnaires were provided on-line via Google forms in the following way: each question appeared on a separate page, a new question became available only after submission of a previous question's reply, and, moreover, there was no opportunity to return to previously answered questions.
The questions were divided into statements, openended questions and multiply choice questions.Each statement presented a 5-point scale with optional answers -(1) SD, Strongly Disagree; (2) D, Disagree; (3) NO, No opinion; (4) A, Agree; (5) SA, Strongly Agree -which appear along X-axis in Fig. 2-6.Y-axis of Fig. 2-6 indicate percentage of the respondents that selected the corresponding options.

Analysis
In this section we compare the students' expectations in the beginning of Introduction to Robotics course against their experience after they participated in classes for one semester (and just before taking a final test of the course).We are analyzed English language comprehension, selfefficiency, active learning strategies, motivation to study robotics, stimulating learning environment, and opinions about the course.

English language comprehension
Overall English language comprehension dynamics was positive: by the end of the course 62.5% (SA-25%, A-37,5%) of the respondents confirmed their confidence of speaking English in the class (Fig. 2), while 12,5% of students did not feel confident (decreased by 12,5%) and 25% had no opinion (increased by 12, 5%).Such results demonstrated that the majority of respondents by the end of the semester felt confident while speaking English during the course, which shows that teaching methods and class activities did not discourage students to use foreign language in the class.Additionally, there was a decrease of students' worries of speaking English with Russian native speakers during inclass discussions (Fig. 3).However, even though 62,5% of students had taken classes in English before starting the course, there was significant negative dynamics on the item "I am not scared when I do not understand what the teacher says in English, because I can ask to explain it in Russian".This might be caused by robotics material complexity or language barrier in English listening and comprehension in.The fact that they had taken classes in English before does not necessarily mean that those were science related classes.We assume that the tendency could be positive in the next semester after the students adapt to hearing in English (Fig. 4).Even though the students worried about English language comprehension of the material, their motivation to study English increased by 12.5% (Fig. 5) by the end of the course and 87,5% in total were motivated to continue studying English.

Self efficiency
In general, the students' expectations towards the robotics field difficulty level were met.Moreover, by the end of the course a small number of students realized that robotics is even a more difficult subject than they had expected at the start (Fig. 6).This might provide additional explanations of the students' tendency to worry about their understanding of the material in English in the class (demonstrated by Fig. 4).Nonetheless, by the end of the course 75% of the respondent group were sure that they could understand difficult robotics terms from the lectures (Fig. 7).These responses hint that the complexity of robotics material was not the main reason of negative effect on material comprehension during the class (Fig. 4).Thus, this strengthens the assumption about the language barrier (i.e., listening the course in the foreign language) as a reason of this negative effect.While 87.5% of respondents were sure that they could successfully pass a final test (Fig. 8), by the end of the course 100% of the students responded that they could learn robotics if they put enough efforts (Fig. 9) even though 87,5% of them had never taken any robotics related classes before.A strong self-efficiency was observed in the final survey: 87.5% of the respondents were ready to apply efforts in order to learn the course material even if the content was difficult (Fig. 10).

Active learning strategies
This subsection presents the observations of active learning strategies, which were used by the students.By the end of the course 100% of the students made efforts in understanding material and trying to connect new material with the knowledge they had possessed before taking the course (Fig. 11) thus demonstrating their motivation towards robotics study.Further we noticed a positive tendency among students in looking for additional sources in order to improve material comprehension as by the end of the course only 12.5% of the respondents had no opinion on these item (Fig. 12) and 12.5% had no opinion about discussing unclear material with the classmates or asking the teacher for additional explanations, while 87,5% (SA-62,5%, A-25%) desired to use these additional resources; unfortunately, the number of positive responses for this item decreased, while it had been 100% at the course beginning.The passive no-opinion position on the both items could be also related to students' self-confidence of having enough knowledge and/or abilities to process the material independently, which may be implicitly connected to the students' self-confidence of learning complicated material (Fig. 9).

Motivation to study robotics
The survey demonstrated that 87.5% of the students had never participated in robotics related classes prior to MSc-IR program enrolment.The question if a student believes he/she will use the gained in the course knowledge for their future job scored 100% of positive responses both before and after the course, while by the end of the course there was a 12,5% increase in SA responses.Moreover, the same tendency was obtained for the question if studying robotics is important because it helps to stimulate student's thinking (with slight difference in SA and A responses, Fig. 13).In comparison with the initial survey, the final survey showed that more students realized the importance of solving problems while implementing robotics projects (Fig. 14).

Stimulating learning environment
Almost all students explained their participation in the course by their interest to the course content.By the end of the course 62.5% of the respondents strongly agreed that mental effort in studying and preparing for classes served as an additional motivation to participate in the classes, while 12.5% changed opinion to a disagree option (Fig. 15).To double check if teaching in English succeeds to stimulate learning environment, the survey included a number of similar questions.One of such questions verified if a student wanted to participate in the class because the course is conducted in English, and the respondents' opinions were positively shifted (SA and A in total summed up to 75%, Fig. 16); this well-correlates with the previously demonstrated tendency of English language studying motivation (in Fig. 5).    the respondents' evaluation of the robotics classes interesting content and the level of complexity (Fig. 18, 87.5% of SA and A responses).Moreover, the question if the students would still select the course in a case it was an elective course (the course is a core obligatory one within MSc-IR program) scored 100% strongly agree responses, which was indirectly confirmed by a high rate of the students' appearance in the lectures while the presence in the lectures was optional.
Figure 18.I believe that the robotics course (content) is interesting and has an appropriate level of complexity.

Open-ended and multiple choice questions
In addition to the likert-scale statements the students were asked to response open-ended and multiply choice questions.One of the multiply choice questions demonstrated that, surprisingly, the students preferred theoretical content and practical tasks to interactive tasks and own ideas presenting (Fig. 19).As teachers, we are interested to facilitate the students to produce, present and discuss their own novel ideas as well as to actively interact with their team members.Therefore, we plan to emphasize the acquirement of these skills through multiple practical tasks within this course as well as in other courses of MSc-IR program in the future.With a help of open-ended questions, we learned that the students liked such methods as individual approach, practical tasks with Lego EV3 robots, opportunity to listen the course material both in English and Russian languages, practicing presentation skills in front of the class and receiving feedback from the teacher.

Conclusion and Future work
To prepare skilled specialists for academy and industry it is important to create an educational ground, which will be competitive with other leading universities on the world-scale.Our team is developing a world-class master program in Intelligent Robotics at Kazan Federal University in English language.To guarantee a highquality education we invest essential efforts into monitoring of students' attitude and motivation to relate their professional life with robotics field and collecting feedback about the program courses from students in order to continuously improve the program.
In this paper we presented the results of our continuous comparative analysis of surveys among master students of Intelligent Robotics program.The survey of Introduction to Robotics course was run twice: the initial survey took place after the first lecture and the final survey took place before the final test.The survey demonstrated that teaching in English encouraged the students to improve their language skills and, while this definitely demanded significant efforts in order to overcome the language barrier and to master complicated robotics material, the students enjoyed the course and could understand difficult robotics topics in the class.By the end of the course the students' self-efficiency and self-confidence improved, while they also realized the complexity of the robotics field and the master program curriculum.Furthermore, the students expressed motivation increase for robotics mastering and applied active learning strategies in studying the course.Yet, surprisingly, the students preferred theoretical content and practical tasks of the course to interactive tasks and own ideas presenting.
As a part of our on-going work we analyse data that was collected through surveys in other courses.It will be interesting to further explore the changes in students' responses and motivation after all of them will gain enough experience of working with sensors and complicated robots within their personal research topics for graduation thesis.

Figure 2 .
Figure 2. I do not worry to make mistakes while speaking English.

Figure 3 .
Figure 3.I do not feel nervous when speaking English with Russian native speakers.

Figure 4 .
Figure 4.I am not scared if I do not understand English because I can ask to explain in Russian.

Figure 5 .
Figure 5.After participating in the course I have more motivation to study English.

Figure 6 .
Figure 6.I am sure I can understand the content of the class despite the complexity of material.

Figure 7 .
Figure 7.I am not sure if I can understand difficult robotics terms during the class.

Figure 8 .
Figure 8.I am sure I can successfully pass the final test.

Figure 9 .
Figure 9.I cannot study robotics no matter how much effort I put.

Figure 10 .
Figure 10.If a lecture is too difficult, I prefer not to learn this material.

Figure 11 .
Figure 11.When I study new (robotics) material, I try to connect it with my previous experience.

Figure 12 .
Figure 12.When I do not understand new (robotics) material, I try to find additional information in order to understand it.

Figure 13 .
Figure 13.I think that studying robotics is important because it stimulates my thinking.

Figure 14 .
Figure 14.I think that (in robotics) the most important is to solve the problems, which I face during the projects.

Figure 15 .
Figure 15.I want to participate in the classes because studying and preparing for classes require mental effort.

Figure 16 .
Figure 16.I want to participate in the class because lectures are conducted in English.

Figure 17 .
Figure 17.Teaching methods are new for meThere were minor positive changes in the students' evaluation of the applied teaching methods novelty with regard to their prior expectations (Fig.17), as well as in

6 ,
), as well as in

Figure 19 .
Figure 19.What students found interesting during classes.

2.1 Robotics curriculum at ITIS KFU Current
robotics curriculum for master students at ITIS is a pilot educational track within the framework of the existing Software Engineering master program, which is supposed to be extended toward a full master program according to Federal State Education Standard with assignment of qualification of the Master in Robotics and Mechatronics.During summer admission campaign we focused on accepting primarily prospective students with bachelor's or specialist's degree holders in engineering, physics, computer science and IT.Students were supposed to have a background in programming or they were expected to learn programming basics independently within the first month of their classes.