Ontology-based indirect interaction of mobile robots for joint task solving: a scenario for obstacle overcoming

This paper describes an ontology-based approach to interaction of users and mobile robots for joint task solving. The use of ontologies allows supporting semantic interoperability between robots. The ontologies store knowledge about the tasks to be performed, knowledge about the functionality of robots and the current situation factors like a robot location or busyness. Ontologies are published in a smart space which allows indirect interaction between participants. On the basis of the knowledge, a robot can define a task that is to be performed and get the current status of other robots. The paper presents a reference model of the approach to indirect interaction between mobile robots for joint task solving, an ontology model for the knowledge organization, and application of the presented approach for the scenario for obstacle overcoming.

The utilization of robots is useful in case of the impossibility of employment of human resources, in the presence of threats to health and human life, as well as to perform consuming tasks in adverse conditions. Performing a task a robot tends to face obstacles on its path because of both heterogeneity and complexity of a landscape. Depending on their design and the task to perform as well as the nature of these obstacles robots can avoid or, if it is impossible, overcome them. The implementation of both of these options requires an appropriate approach to problem solving and interaction with users, which takes into account various factors such as an obstacle size and shape.
This paper proposes ontology-oriented approach to indirect interaction of mobile triaxial robots between themselves and users. These robots have to be able to work and move on the ground having uneven terrain, and perform various tasks remotely given by one or more users or other robots. Each robot consists of several mobile units (blocks) with one or two pairs of wheels and is equipped with ultrasonic sensors. Robots are able to detect obstacles, and overcome not only low ones (the height is less than the radius of the wheel), which can be run over without making additional operations, but also obstacle height 1-4 times bigger than the radius of the wheel.
Robots are supposed to distribute tasks among themselves effectively, to take into account promptly events occurring during the execution of tasks. For example, a robot should try to overcome encountered obstacles automatically.
The interaction and information exchange between robots and users is conducted using smart space technology. This technology allows organizing the information and knowledge sharing for various actors due to an ontology-based publication subscription mechanism. Smart space, in fact, is an information model that describes the interaction of mobile robots and the tasks they perform. This technology allows the use of robots and user devices of different vendors within a single scenario.
The rest of the paper is organized as follows. Section II describes related work in the area of mobile robots interaction. Section III presents the reference model of the approach to indirect interaction between users and mobile robots. Section IV describes the implementation. Finally, Conclusion summarizes the paper.

Related works
Paper [4] deals with an intelligent system of robot and human interaction. This system uses adaptive learning mechanisms to account for the behavior and preferences of a human. It includes algorithms for speech and face recognition, natural language understanding, and other ones so that the robot could recognize human actions and commands. A robot used in this paper (Pioneer 3-AT) is equipped with multiple sensors, scanners and a camera. The paper focuses on the process of teaching the robot and its mechanisms of interaction with a human.
Paper [5] describes a coordinated process for the agents (in particular, mobile robots) jointly performing a task, using the knowledge about the environment, their abilities and possibilities for communication with other agents. The effective use of knowledge about the agents capabilities is implemented by using suitability rates which allow agent to choose the most appropriate action. This approach was tested during a football match between mobile robots. The interaction in this article is considered in terms of the choice of an agent to perform an action.
Paper [6] proposes a methodology for coordination of autonomous mobile robots in jams in congested systems. The methodology is based on reducing a robot's speed in jams and congested areas, as well as in front of them. Thus, the robots slow down detecting other robots in front of themselves. The effectiveness of the methodology was tested in the simulation of congested system with a jam.
Paper [7] discusses combining wireless sensor networks and mobile robots. Authors suggest layered swarm framework for the interaction of decentralized self-organizing complex adaptive systems with mobile robots as their members. This framework consists of two layers (robots and wireless LAN) and communication channels within and between layers. During the experimental verification of framework mobile robots independently reached the destination.
Paper [8] deals with a system of robots that allows garbage collection in the territories without the involvement of human. The system consists of a group of the robots responsible for the recognition of objects and bypassing the area. While collecting garbage robots can interact with each other via communication channels.
Paper [9] is devoted to the problem of robots group control in non-deterministic, dynamically changing situations and, above all, in conditions of organized resistance. The paper contains a formal statement of the robots group management task for these conditions, as well as a model of collective interaction of robots in groups. Simulation of the robots group control task in a military counteraction showed the effectiveness of the approach.
Paper [10] shows a system of robots interaction based on BML -Battle Management Language. This language is intended for expressing concise, unambiguous orders read by both humans and robots. The orders are transmitted from the system of multiple robots and processed for further distribution specific commands between them.
The paper [11] describes designing human-robot interaction, improving human situational awareness on the basis of an agent-oriented approach. This approach is intended to apply at movement of mobile agents (autonomous robots and cosmonauts in a protective gear) on the Moon's surface in the controlled territorial area.
Paper [12] discusses the use of multi-agent systems for the organization of industrial robots interaction. On the basis of them a system architecture of robotic systems interaction was built. During the interaction of agents within designed architecture the programs are creating which govern behavior of the agents. These programs are stored in knowledge base.
Paper [13] considers the problem of program agents' access to the smart space based on Smart-M3 platform. This paper suggests a model of concurrent sessions to work with multiple smart spaces. The model allows an agent to use multiple connections and control objects in a single session. Moreover, it provides local storage, divided between session connections. The storage is synchronized using subscriptions.
The approaches used in papers [4] and [6] require a well-defined functionality of robots, so the set of undertaken tasks is restricted. In paper [5] the knowledge of the capabilities of robots is generated dynamically during the execution of tasks what allows managing the robots effectively. Thus, no limitation to robots functionality and dynamic determination of the options available can provide the widest range of tasks to execute. Paper [7] shows the effectiveness of the indirect interaction of robots, that is, the interaction through an intermediary that is Smart-M3 platform in the paper. Paper [8] demonstrates an additional module necessity for the efficient interaction of robots and users. Paper [9] provides a formal model that can be used for interaction. Paper [10] shows the effectiveness of using specialized language for robots control. Papers [11] and [12] demonstrate the applicability of the multi-agent systems approach for robots and humans interaction in various fields. Finally, paper [13] proposes a model of interaction between agents and a smart space.

Interaction of mobile robots
The reference model of the approach to indirect interaction mobile robots is presented in Figure 1. It is conducted on the basis of ontology-based publish / subscribe mechanism implemented in Smart-M3 platform [14,15]. Smart-M3 platform includes a semantic information Broker (SIB) and knowledge processors (KP).
The SIB broker contains the information provided in the form of a knowledge base; KP processors provide an access to the information and its changes and ensure notifications of new information. The interaction between agents and broker operates according to Smart Space Access Protocol (SSAP) [8]. All communications between the smart space and users, robots, additional services are conducted via a LAN or the Internet.
Functions of the robot can be classified into two types: basic and additional (Fig. 2). The basic functions are necessary to move the robot. These ones include moving in two directions and overcoming obstacles by lifting, lowering and pulling units.
Additional features are necessary to expand the number of tasks performed by robots. An obstacle recognition allows storing a sequence of actions to overcome obstacles. The movement to a given point involves automatic overcoming encountered obstacles or manual control request if automatic overcoming is impossible. The "Explore a point" combines moving to a given point, and filming and sending a video.  Smart-M3 platform involves storing information in the form of RDF-ontologies, in which information is presented as a set of triples of "subject-predicate-object". A triples subject describes an entity such as an object, a device or a person. This entity has a property or can perform an action described in the triple. A triples object describes an entity, which the subject is associated with, that is its function or property. The object of one triple may be the subject of another one.
A triples predicate describes the type of relationship between subject and object. It can determine the type of functions performed by the subject, or the correlation between subject and object.
A simple example of a triple stored in the ontology is <"robot", "task", "goToLocation">. The subject "robot" describes a robot, which information in the triple relates to, the predicate "task" defines what this information is, and the object "goToLocation" is a particular "task" for the subject "robot". At the same time "goToLocation" is also a subject in the triple <"goToLocation", "coordinates", "40; 200">.
Devices connected to the smart space based on Smart-M3 platform are able to "subscribe" to a certain pattern triples to respond timely to new information published in the smart space. In the case of a new triple satisfying pattern all devices, which are subscribed to triples of this pattern, are notified.
A typical example of the ontology provided in the smart space at the time of publication of a task for the robot is shown in Figure 3. Triples contained in the ontology are presented in the figure in the form of a set of ovals and arrows connecting them. An oval, which is a source for the arrow, denotes the subject; the arrow denotes the predicate, and an oval, which is a target for the arrow, denotes the object. This example shows information about a robot consisting of three blocks (<"roboti", "blockAmount", "3">). Each block has its main engine ("moveEngine"). The front block is also equipped a lift engine (<"frontBlock", "hasPart", "liftEngine">) that is used to raise this block. The robot can move backward (<"roboti", "implements", "moveBack">) or forward (<"roboti", "implements", "moveForward">), overcome an obstacle (<"roboti", "implements", "acrossObstacle">), and snapshot (<"roboti", "implements", "takePhoto">). Besides, the robot can go to a given point (<"roboti", "implements", "goToLocation">) and explore a location (<"roboti", "implements", "exploreLocation">). Fig. 3 shows the moment the robot is given the task to move back (a triple with "task" predicate). In addition, triples with the "event" predicate can be published in the ontology describing the events that occurred during the execution of robotic tasks such as stopping due to detecting obstacles (<"roboti", "event", "stopped">).
When starting the robot "subscribed" to the triples with the "task" predicate. Therefore, when triple <"roboti", "task", "goToLocation"> appeared in the ontology, the robot analyzed the subject of the triple. It determined whether to perform the task (on the basis of the coordinates of the point and the location of its own and other robots), and then went to the point.

Implementation
To assess the approach efficiency the scenario of a point exploring and encountered obstacles overcoming is chosen, as it uses most of the robot functions. During this scenario, if a robot finds an obstacle using ultrasonic sensors, it makes a request to the cloud service to get a sequence of actions to overcome such an obstacle. If the information about obstacles of this type is not found in the cloud service, the control over the robot is transmitted to a user. A prototype of a mobile robot has been developed for this scenario. The robot prototype is able to move forward, to detect obstacles and overcome them. It is based on a set of robotic Lego Mindstorm EV3. This set allows one to easily design a robot with required functionality for training purposes. At the same time there is the possibility to use electronic units, motors and sensors and program them in Java. Fig. 4 shows the design of the robot constructed for the scenario of forward moving and encountered obstacles overcoming. In order to overcome obstacles in accordance with the present scenario, the robot was constructed from several blocks. Due to the design the robot could climb an obstacle gradually. First, it lifted and fixed the front block, then basing on the front and back blocks it raised the middle one. All blocks were equipped with a pair of drive wheels; in addition, the middle block had a pair of wheels without a drive for balance. An ultrasonic sensor for measuring the distance to objects was also mounted on the central block.
The sensor detected obstacles in the path of the robot, automatic stop occurred. At this stage, only basic functions of the robot were necessary, so it was not equipped with a camera or other devices to receive any information about the nature of the terrain. Fig. 5 shows the scenario of forward moving and encountered obstacles overcoming. The diagram shows six main components of the system.
• Smartphone application publishes tasks to perform and shows the status of performance.
• Image processing service is designed to retrieve information about an object or area from a photo.
• Smart space is a repository of information on the basis of Smart-M3 platform.
• Service of the robot's interaction with the smart space is a program which receives, processes and transmits information to the robot from the smart space. At the same time the service receives information about the events and results of task performance from the robot and publishes this information in the smart space .
• The robot executes commands received from the interaction service.
• Cloud service is a knowledge base that stores information about various obstacles in the form of a sequence of actions to overcome these obstacles.
When starting each robot publishes information about its functions and limitations from its own ontology in the smart space. Moreover, while executing tasks a robot publishes information about its location and the current task.
For example, a user publishes task "Explore a point" in the smart space. Since the robot is free, that is, it does not perform any other tasks, its interaction service determines whether the robot is to perform published task. For this purpose, the service follows the information about the task and about other robots. If a robot is the closest one to the point, it proceeds to the task.
When a robot detects an obstacle in the way, it automatically stops, informing the interaction service about it. For further performance of the initial problem the robot needs to overcome this obstacle. Therefore, it sends a request to the cloud service on how to do it. If there is no information about such an obstacle, the interaction service publishes information about the impossibility of overcoming the obstacle automatically in the smart space.
After receiving a notification a user accepts the control over the robot. As a result, the commands that are to be done in order to overcome this obstacle are consistently being published in the smart space. The interaction service inserts the sequence of instructions and information about obstacles in the template database for further use. After overcoming the obstacle the robot continues to perform the task and finds the second obstacle that is similar to the first one. Since the information about it is already in the obstacles templates database, the robot overcomes it without the user's help.
When the robot reaches the destination, and receives command "Take a photo", it takes a snapshot of terrain and sends it to the interaction service, which in its turn publishes an image in the smart space. Then, the image processing service receives information about the new image and processes it. After receiving information about the point, the service publishes it in the smart space, and then information is sent to the user.

Conclusion
The approach to the indirect interaction between mobile robots for the joint problem solving, described in the paper, provides an effective solution to a wide range of tasks due to the indirect interaction of not only the robots and users, but also the robots with each other. The use of ontologies minimizes the dependence of the approach on the manufacturers as well as software of robots and user devices: only possibility of working with ontologies support is necessary. To evaluate the working capacity and effectiveness of the approach a robot prototype has been designed.
Further development involves adding computing services, and equipping the robot with new types of sensors to expand the set of tasks to perform. Moreover, the implementation of additional robot functions is planned. Such modifications include, for example, equipping the robot with cameras and developing the image processing service that can publish the nature of the terrain, the type of encountered obstacles, the presence or absence of any given object in sight. This service is likely to make the robot control easier and more convenient (to make the robot itself be able to identify and overcome obstacles), and make the use of a robot more efficient (due to new tasks performed). In addition, it is necessary to evaluate the effectiveness of the joint problem solving. For this task it is planned to design the second robot prototype.