Design of an Autonomous Forklift Using Kinect

Material handling is a necessary, but expensive activity in factories. Autonomous robot technology can help reduce the cost and relax humans from the exhaustive job of driving forklifts. In this paper, we describe the mechatronics design and implementation of an autonomous forklift. The robot can perceive the 3D dynamic world and can plan its motion autonomously to lift materials from a source to target locations. Dynamic map of the world is built using data from a Microsoft Kinect head and readings from wheel encoders, thus enabling the robot to avoid obstacles and reach target locations safely. Experiments showed success of the robot to move and load the cargo to


Introduction
The process of loading, unloading and transport of materials is one of the key issues for every production site and has a great impact on product cost.The material flow within the product manufacturing site is expensive; therefore, the efforts to find favorable and flexible systems continue to be of great importance.The demand toward a higher level of autonomy is strong and will change the operation of warehouses significantly [1,2].
Forklifts are general purpose material handling machine and are generally operated manually with a driver.Operators are trained on forklifts so that they understand the limitations of the device, and the rules of safe and efficient operation.The job of driving a forklift requires long training for the drivers to ensure safe operation.Besides, the job is exhaustive and not suitable for humans and hence it is an ideal candidate for automation.
With recent decrease in prices of sensors and computing devices, several researchers realized that the mobile robot technology is matured enough to run a forklift in warehouses.Taking only high level orders from a human supervisor, an autonomous forklift can safely execute tasks that require interaction with people and objects in an unknown environment.These include the ability to detect and manipulate loaded pallets, both on the ground and on a truck bed, as well as navigating among obstacles [3,4].
Although some product exists in the market, yet the cost is high and there is still a room to develop simple systems that are user friendly and with affordable prices [5].
In this paper, our objective is to design and develop a forklift able to move autonomously, lift the cargo and move it to target location.It should be able to map the working environment and to move autonomously or allow and ask for a human intervention when needed.The use of Kinect, though had been used in other applications [6,7], is new to this forklift application and will be used to map the robot environment.
This paper is arranged as follows: the next section outlines the related work about building autonomous forklift technology.Then, the mechatronics design of the system is introduced in section 3. The control system logic and experiments with the system are presented in section 4 and finally conclusions are summarized in section 5.

Background
The Autonomous Guided Vehicle, AGV, is probably the first industrial application for mobile robots in factories.Generally, the path is predefined and can be either visible, marked by a surface with high contrast or it can be invisible like a magnetic tape.The AGV works as a line follower robot and usually has sensors for obstacle avoidance and transfer material among fixed spots in the factory.This system have limited autonomy and inconvenient when the path is changed especially in flexible manufacturing lines.AGV concepts were combined with forklifts to build systems that enjoy both powerful material handling and simple autonomy by AGV.This was achieved in several products with even increased autonomy by replacing the line following principle by 3D scanners [8,9].3D scanners enable the autonomous forklift to perceive the environment around it, build a map and plan its missions and paths in an autonomous self-controlled manner.Several manufacturers produce autonomous forklifts, and we will discuss three major products, shown in Fig. 1, namely: Hitachi, Clark and CSAIL autonomous forklifts.Clarks developed an autonomous forklift whose body is mounted on four wheels includes two castor wheels at the front side and one small castor wheel at the right rear side.To achieve autonomy, the forklift is modified so that all the necessary sensors and a PC for steering and driving motors are attached.The information about the forklift environment is obtained by using vision, laser range finder and sonar sensors.
CSAIL is an autonomous robotic forklift developed for the military.It can move pallet loads from place to place under voice command, and eventually to pack up an entire outdoor warehouse of pallets and transport it to a new location.The forklift is designed to operate in unstructured environments such as outdoor packed earth.It enables dual operation whether autonomous or manual [10][11][12].Work in design of autonomous forklift interface had been made in [13,14] using voice or other graphical interfaces.Coordination of several forklifts had been described in [15].The system architecture is shown schematically in Fig. 2. The robot communicate its data through an arduino board which is connected to the host computer.Kinect sensor communicates directly with the host computer to send its visual measurements.

Mechanical design
The system consists of the mobile base of the forklift and the vertical moving mechanism that contains metal forks as shown in Fig. 3.
The base has three wheels, two front wheels, and one rear wheel.The front wheels are driven by DC motors with reduction gear box.The differential drive is controlled to drive and steer the base.The concept of the forklift is to have a lifting mechanism attached with two metal forks to leave the cargo.The dimensions of the forklift were made to work in indoor environment and hence the dimensions were selected to pass through normal doors.The basic dimensions are shown in Fig. 3.The height is made to be 120 cm.

Sensors and actuators
For the purpose of 3-D map building, a Microsoft X-box Kinect is attached to the forklift to detect obstacles in the near sides of the forklift.The choice of the Kinect was made in comparison to popular laser range finders used in other works having full scale forklift such as hokuyo urg-04lx, sick lms200.
The laser range finders are relatively expensive and its processing and map maintenance requires complex computations.This will also require heavy and expensive computing device.Several researchers pointed out for the use of Kinect for 3D scanning of the world at an affordable cost and satisfactory resolution [6,7].
While to the Kinect as giving accurate maps and easy to use and require simple computing power.The Kinect give good resolution for near objects which is just enough for this application (6mm) as shown in Table 1.Therefore we decided to use the Kinect.
Optical encoders are attached to DC motors to measure the robot position and orientation.
As for the lifting mechanism, we thought of using either electric of hydraulic actuation.Since our prototype is designed to lift 10 kg maximum, therefore we exclude the hydraulic option due to light payload and the high cost and complexity of the hydraulic circuit.The electrical actuator selected is a DC motor coupled to a shaft connected to rack and pinion.Finally, we decided to choose the electrical actuation system.Because the lifting mechanism we are using depends on the rack and pinion technique, it was required to have a rotary actuator.So, we decided to use a DC motor to be attached to the mechanism to lift the cargo.
The weight of the forklift is 70 kg taking in consideration the weight to be lifted and the counter balance.So, each motor should drag 35 kg assuming that the motor size is selected to be 25kgcm.
The selected motors have a gear box with gear ratio 1:200 to increase torque and decrease the rotational speed and hence robot velocity.

Computing and interface
There are several types of controllers that can be used to compute the control logic.In selecting the robot controller, we had to select a platform compatible with the Kinect and its software.So, we used a laptop with Intel core i5 processor and windows operating system in order to connect to the Kinect using a USB interface and to the robot's microcontroller using a USB serial interface.The interface window for making commands to the forklift and monitoring its status was made through a GUI program coded in C++ language.We also used an arduino microcontroller board to control and interface with limit switches, ultrasonic sensors, encoders and DC motors.

Experiments
The autonomous forklift is shown in Fig. 4 with its final implementation.The Kinect is attached on to and the laptop is fitted on the back side.
The motion command for the autonomous forklift is generated based on the information of its position provided by the Kinect and motor encoders.The flow chart for robot control algorithm is shown in Fig. 5.The motion control command is generated by the path planner to the goal position ensuring safety of its motion.At first, we can prepare a scenario-based behavior to accomplish a predetermined task and the other task was to go to its target and avoid obstacles using the map captured from the Kinect and create its own path using the path planner algorithm.The controller will read the data from the encoder and use these data to calculate the orientation of the robot and the distance traveled by the robot.
The robot is aware of the working environment in order to avoid obstacles while reaching its destination.The lifting mechanism has sensors to feed the controller back with the forks position.
Depth images from kinect are converted to obtain the top view image of robot front.The obtained image shows clearly the kind of obstacles and free area confronting the robot.Fig. 6 shows the depth image from Kinect and the converted to view image showing this case where a box is inserted in front of robot.Fig. 7 shows the robot test path for obstacle avoidance.The Kinect images were used as input to a visual SLAM algorithms running on Extended Kalman Filter.The program can learn novel environment and recover the self-motion of the forklift.Results are not shown for space limitation.

Conclusions
We have designed and developed an autonomous forklift with a simple design, and high degree of autonomy, rigidity and stability to lift payloads and move autonomously in stores and warehouses.We exploited the Microsoft Kinect to obtain depth maps of the environment and convert these data into a top view in order to navigate the working environment for the robot.The robot self-localization was determined based on data from encoders and Kinect to mark robot position in the map.The robot self-localization was made using wheel encoders with differential drive kinematics.The robot accomplished a specific task and lifts the cargo from its origin place to target destination while avoiding static and dynamic obstacles.The actual robot path was monitored during its motion together with the required given path.

1 MATECFigure 1 .
Figure 1.a) Example for AGV( By Muller), Autonomous forklifts, b) Hitachi and c) Clark HITACHI has successfully delivered an AGV system to South Korean industrial machinery manufacturer, Hanwha.The system can handle special purpose trolleys which are placed at two different heights.Besides, it can travel without following any travel guides.The traveling route of AVGs needs to be easily modified in case of changes to the factory layout.Clarks developed an autonomous forklift whose body is mounted on four wheels includes two castor wheels at the front side and one small castor wheel at the right rear side.To achieve autonomy, the forklift is modified so that all the necessary sensors and a PC for steering and driving motors are attached.The information about the forklift environment is obtained by using vision, laser range finder and sonar sensors.CSAIL is an autonomous robotic forklift developed for the military.It can move pallet loads from place to place under voice command, and eventually to pack up an entire outdoor warehouse of pallets and transport it to a new location.The forklift is designed to operate in unstructured environments such as outdoor packed earth.It enables dual operation whether autonomous or manual[10][11][12].Work in design of autonomous forklift interface had been made in[13,14] using voice or other graphical interfaces.Coordination of several forklifts had been described in[15].

Figure 2 .
Figure 2. System Architecture for autonomous forklift equipped with Microsoft Kinect.

Figure 3 .
Figure 3.The mechanical design of new forklift .

Figure 6 .
Figure 6.a.The depth map from Kinect, b.The computed top view.

Figure 7 .
Figure 7.The robot path for testing obstacle avoidance.

ICMME 2017 Table 1 .
Comparison between three ranging systems