Map representation using hidden markov models for mobile robot localization

. This paper describes a map representation and localization system for a mobile robot based on Hidden Markov Models. These models are used not only to ﬁnd a region where a mobile robot is, but also they ﬁnd the orientation that it has. It is shown that an estimation of the region where the robot is located can be found using the Viterbi algorithm with quantized laser readings, i.e. symbol observations, of a Hidden Markov Model.


Introduction
It is the general case that Robots are conceived to perform a task, and the chosen world representation is highly correlated with the performance of the robot in such task [1]. The mapping process is restricted to the robot features, where a map is generated using a collection of sensor readings from a traversed trajectory in the scene given by the robot shape and its degrees of freedom, and this information might be useful to define the map [2] [3].
In the many years of research of robot navigation [4] and SLAM [5] [6] [7], several map representations have been proposed. From metric and topological to symbolic and probabilistic representations [8] [9].
In this work, we propose a probabilistic representation given by a Hidden Markov Model from a number of sensor readings and robot displacements; this approach has been proven useful for robot localization and navigation [10], [11]. We extend those works by incorporating the trajectory information in the Markov Model, allowing us to obtain the position and orientation related to a number of nodes on the trajectory.
The remaining of the paper is divided as follows: in Section 2 we introduce the VIRBOT system while in Section 3 we detail the cartographer module and develop the map representation using Hidden Markov Models based on the system characteristics. Then, in Section 4 we present a localization method using the proposed map representation. In Section 5 we detail the experiments and results and in Section 6 we conclude this work.

The VIRBOT system
There are several architectures to control mobile robots, in our approach, the VIRBOT system [12], the mobile robot's operation it is divided into four general layers: Inputs, Planning, Knowledge Management and Execution, having * e-mail: robotssavage@gmail.com each of them several subsystems, see Figure 1. Each subsystem has a specific function that contributes to the final operation of the robot.
This system has similar features presented in the IN-TERRAP agent architecture [13]

Inputs layer
In this layer are the robot's internal and external sensors, as well as, the simulator that simulates these sensors when a simulated Robot is used.

Planning layer
Action planner: The objective of action planning is to find a sequence of physical operations to achieve the desired goals.
Motion planner: This module receives, from the action planner, a set of locations that the robot needs to visit and it finds paths to reach them using a topological map and the A* Algorithm.

Knowledge management layer
Cartographer: This module creates and contains different types of maps for the representation of the environment. This section is explained in more detail in section 3 Knowledge representation: A rule-based system, CLIPS, developed by NASA, is used to represent the robot's knowledge, in which each rule contains the encoded knowledge of an expert.

Execution layer
This module executes the actions and movements plans.

Cartographer
The cartographer module is responsible for estimating the region where the robot is located. One of the problems of using only dead reckoning to estimate the robots' position x, y, θ is that, due to errors in the movement of the robot and errors in the sensors that measure them, after several movements, the real position of the robot can not be estimated accurately. In some robotics architectures, the ones based only on reactive behaviors [14], it is not necessary to know the exact pose of the robot but only an estimation of the region where is located. There have been several approaches based on Hidden Markov Models and Bayesian techniques to solve this problem [15]. In our system this task is done by a localization module, using laser readings, Hidden Markov Models and the Viterbi algorithm.

Hidden Markov models
A Hidden Markov Model (HMM) is a two random variable process, in which X one of the random variables is hidden, and the other random variable O is observable [16]. The hidden random variable X represents the states, in a model that is a collection of states connected by transitions. In our system, the states represent the centroids of the regions in the free space where the robot navigates, see Figure 2.
Each state has two sets of probabilities: an output probability, that provides the conditional probability that given that the system is in a particular state, a symbol is generated from a finite set of symbols, and a transition probability from going from one state to another.

Generation of the HMM symbols
The symbols are obtained using clustering techniques like K Means or Vector Quantization (VQ). VQ techniques are used for data compression, given a set of vectors, S j = [s 1 , s 2 , ..., s m ] that represent lasers readings, see Figure 3, a set of centroids are found which represent them. The collection of centroids is called a codebook. This codebook is designed from a long training sequence that is representative of all data, laser readings, to be encoded by the system. To generate the codebook, the Lynde-Buzo-Gray algorithm is used [17].
After the codebook is created, each vector S t of the laser readings to be encoded is compared with each of the stored vectors C i , and the vector S t is coded by identifying the vector C k that best represents S t according to some distance measurement d.
where L is the size of the codebook. The distance measurement used was the Euclidean distance.

HMM model
The symbols are the indexes of a vector quantizer, for each laser reading we have L possible observations vectors that correspond to the L vectors of a quantizer. Then for a vector S t the symbol O t corresponds to the index of the centroid that best fits the laser readings, O t = k.
And the probability of emitting symbol k being in state where X t = i means the Markov chain was in state i at time t, and O t = k means the output symbol at time t was k.
The transition probability defines the probability of taking the transition from a particular state to another state or itself. Where a i j is the probability of taking a transition from state i to state j: Also included is the initial state distribution probability: In the first order HMM, it is assumed that whether the Markov model will be in a particular state X at time t + 1 only depends on the present state at time t, and not on the past states.
Another assumption is that if a symbol will be emitted it only depends on the present state at time t, and not in the past states or emitted observations.

Discrete hidden Markov models for laser readings
Each region R j of the environment where the robot navigates is represented by a Markov model λ j = (A, B, π), where the matrices A and B are: A = a i j and B = b i (k) for all i, j and k, with N states and L observation symbols.
The probabilities of each Markov model λ j are found during training. For each region j, 0 < j < N regions , the robot is set in region j with orientation of 0 degrees according to an global origin and then it follows the steps described in Algorithm 1. HMM states from 1 to 8, corresponds to the rotation process and from 9 to 16 to the navigation one, see Figure 4. For each path that the robot takes in a closed region, see Figure 5, we get the observations   For each laser reading, we have L possible observations that correspond to the L vectors of a quantizer. This quantizer was created using laser data during several paths that the robot followed. With the set of observations an estimation of the parameters of the HMM λ j , that is A, B and π for each region j, was found using the BaumWelch algorithm. In the Figure 6 we can see the complete HMM for the environment in Figure 2. This HMM represents the concatenation of each region in the environment, where each node is the particular HMM λ j for each region.

Robot localization using discrete hidden Markov models
To recognize the region where the robot is and its orientation, first, the robot start rotating every 45 degrees, until it rotates 360 degrees, for each rotation it makes a laser reading, generating a vector of laser readings that is encoded through a vector quantizer of L vectors. This quantizer gives a set of observations that correspond to the VQ vectors that best fit the laser vector at time t during its rotation. Then using the observation vector with all the HMM j for each region, the probability of each model λ j is found using the Forward algorithm [16], see Figure 7. Then the robot is in Region j : (2) Fig. 7. Probability evaluation for each region.
After it is found in which region is the robot as well its orientation the robot starts navigating to a defined destination, collecting again another observation vector O, and using this vector, the hidden variables, the states, are found using the Viterbi algorithm. This algorithm finds the best sequence of states in the network, and thus the best sequence of regions that the robot follows during its path. Figure 8 shows the best path followed by the robot going from one region to another.
In order to retrieve the state sequence, it is necessary for each t and j to keep track of the argument which maximize the previous equation. Using the array ψ t ( j), the procedure is as follows: 2.1) Initialization:

2.3) Termination:
2.4) State sequence backtracking: .., 1. Figure 9 shows the overall system to find the best sequence of regions that the robot visited.

Experiments and results
We tested the algorithm using the VIRBOT simulation module, in which laser readings can be simulated, see  one have a 180-degree laser, with a separation of 12 degrees each, making 15 laser readings in each sensing. The real robot uses a Hokuyo laser.
The algorithm was tested in an environment with 4 regions, shown in Figure 2. The robot was put in any of the regions of each environment, and from this original position, it tried to reach a set of destinations using reactive behaviors. In Figure 8, the robot starts in a particular region and to reach the goal destination, that is in another region, it needs to pass through the nodes that link the regions, when it reaches each of them it rotates to localize it shelf. With the symbols that were found using the vector quantizer, the start orientation of the robot, as well as, the best states were found on using the Viterbi algorithm. Table 1 shows the results with the robot's simulator, column T r.S ize represents the number of vectors used to create the laser VQ; column N.S ymbols indicates the number L of centroids of the VQ, that is, the number of symbols of the HMM; column N.Obs. indicates the number of observations T used in the Viterbi and Forward algorithms, and finally the column Error shows the percentage of error for detecting in which region the robot is. As we can see the best performance was obtained with a training set of 8000 vectors, 512 observation symbols and using 64 observations each time the Forward and Viterbi algorithm was performed. From the estimated states we found 89% of the times the region where the simulated robot was, using only the laser readings without dead reckoning. Table 2 shows the results with the TurtleBot robot. As we can see the best performance was obtained, as with the simulator, with a training set of 8000 vectors, 512 observation symbols and using 64 observations each time the Forward and Viterbi algorithm was performed. From the estimated states we found 85% of the times the region where the real robot was, using only the laser readings without dead reckoning.