Automatic Grasping of a Pole Climbing Robot using a Visual Camera with Laser Line Beams

A pole climbing robot was developed to inspect or repair pipe structures. The robot has a 5 degree-offreedom manipulator and two grippers, and moves along cylindrical pipes, overcoming obstacles such as flanges and valves like an inch-worm. The robot works in places very high from the ground, and thus the robot should be able to grasp the next pipes semi-autonomously as the operator cannot see the next pipe for the robot to grasp. This paper proposes a very practical vision-based scheme with line lasers for grasping a cylindrical pipe autonomously. The scheme can be used practically for a pole climbing robot in real time.


Introduction
We developed a pole climbing robot to inspect the integrity of the pipe structures such as the containment spray piping system in a nuclear power plant or the roof frames of a stadium consisting of several pipes.The robot has 5 links and two grippers at both ends of the robot, and moves along cylindrical pipes, overcoming obstacles such as flanges and valves like an inch-worm or gibbon.
Such kinds of pole climbing robots have been studied by different researches.The 3D-Climber was developed by the University of Coimbra [1].This robot has a 4-DOF manipulator and 2 grippers.It has a very useful mechanism to climb arbitrarily bended poles; however, it weighs too much and has a bulky gripping assembly.Climbot [2] was also developed by South China University of Technology for pole climbing using a 5-DOF mechanism.It can climb cylindrical pipes as well as rectangular truss structures, but is also too heavy and bulky to be used commercially.
The most essential technology for a pole climbing robot is how to grasp the pipes firmly, quickly, and exactly.Several researches have been performed to solve this problem [3]- [4].This is very difficult as the robot climbs up far away from the ground where the human operates the robot.The human operator cannot see the robot's hand and object pipe from the ground.Unfortunately, however, the robot may not be equipped with several stereo cameras owing to their weight and volume, and their remote visual operation still requires highly trained human skills.This paper proposes a practical scheme for a pole climbing robot to grasp pipes autonomously.The scheme can be used for a pole climbing robot in real time.

Configuration of the pole climbing robot
The pole climbing robot was originally designed to climb up a spray piping system located at the containment of a nuclear power plant.The piping system has obstacles such as flanges and valves along the pipe, and the ceiling parts are located at up to 40 meters high.The pipe has branches to feed spray water into horizontally circumferential pipes.
The robot has five links and two grippers at both ends of the robot, and moves along the cylindrical pipes, overcoming pipe obstacles such as flanges and valves like an inch-worm or gibbon.It has a symmetric shape in view of the determination of the base link.The two gripers work as a base by turn.Joints 1 and 5 implement roll movements, allowing the robot to rotate about its own axis.Joints 2, 3, and 4 achieve rotational movement like a human elbow or knee, with its revolute axis perpendicular to the link axes.The robot is grasping a horizontal pipe with gripper 1, which works as a base of the robot.The pipes are bent along the center line with a unit vector, u.To grasp the bent pipe, the robot should mate the coordinate vector, x 5 , with the pipe orientation vector, u.The axis vector, z 5 , is directed along the center line of the pipe.

Vision-based grasping algorithm
For the robot to grasp a pipe, we have to know the position and orientation of the pipe in the form of a homogeneous transformation.This can be calculated through a transformation of the robot-to-pipe coordinate system. where denotes the homogeneous coordinate transformation matrix from coordinate system {A} to coordinate system {B} as a notational convention, and Here, we have three assumptions: y A three-dimensional cylinder is assumed as a planar strip on a marked plane, as shown in Fig. 3.The 3D cylinder is shown a little bit larger than the planar strip in a real situation.
y The pipe is cylindrical to be symmetric about the zaxis.Thus, we set the pipe coordinate system with 0 z I .y The robot may grasp the pipe at an arbitrary point along the z-axis of the pipe.It does not matter where along zpipe the robot grasps.Thus, we set the origin of the pipe coordinate system as D z =0.As shown in Fig. 4, the planar pattern of the pipe is projected onto the image plane as a trapezoidal shape.The straight vertical line on the pipe-mark plane will be projected onto an image plane as an oblique line with slope A 0 and x-intercept C 0 .Thus, the projected line is expressed as We can obtain two slopes, A 0 1 and A 0 2 , and two xintercepts, C 0 1 and C 0 2 , through image processing.In other words, we have four unknown values, ).We propose a practical scheme to solve this problem.We turned our attention to the point that the four unknowns I Thus, we can calculate the inverse kinematics by supplying four variables for all visual feedback where k is the sampling time and K Dx is the gain, which is determined experimentally.T

T T T T T
Thus, we can obtain command of the joint angles

Pipe Edge Detection with Line Laser Beam
The camera image is used to calculate the transformation variables of the gripper from the pipe position.Through the processing of the captured image, we can obtain two slopes, A 0 1 and A 0 2 , and two x-intercepts, C 0 1 and C 0 2 , as defined in Fig. 4. Fig. 5 shows the image processing deriving two slopes and two intercepts.Using algorithm (7), we can calculate the variables  , and two x-intercepts, C 0 1 and C 0 2 , through image processing.Fig. 5 shows a general image processing procedure to obtain the line equation of the objective pipe edge, which are lenz distortion compensation of the original image from the hands-on camera of the robot, edge detection, selection of a straight line, and so on.Detection of the edge line of the pipe is not an easy job especially in poor illumination conditions.We also have to intelligently choose the right line among several straight lines behind the pipe.Thus, we use a line beam of the laser to detect the straight lines and determine the right line besides many background lines.The laser beam is emitted several times varying the incident angle to the pipe.We obtain the differential image between the image with a line beam and an image without line beam.Thus, we obtain only the laser beam line on the pipe surface, and the edge variables.

Summary
A vision-based scheme was proposed to grasp a cylindrical pipe for a pole climbing robot autonomously.Our climbing robot has a 5 degree-of-freedom manipulator and two grippers, and moves along the cylindrical pipes, overcoming obstacles such as flanges and valves like an inch-worm.The robot works at very high places, grasping the next pipe semi-autonomously with the aid of a visual camera and laser line beam.The scheme is much more robust than the visual camera only, and can be used practically in a pole climbing robot in real time.We will study further the pipe identification against the worst illumination conditions and an error analysis based on our three assumptions along with a series of experiments.

Original captured Image
Edge Detection Line Extraction x, y Coordinates

Figure 1 .
Figure 1.Vision-based pole climbing robot of the camera frame to pipe frame as shown in Fig.2.We can calculate the first five transformations easily, the current joint angle of the manipulator.The problem is how to determine the transformation of camera to pipe: T C pipe

From
relationship with the four known variables

Figure 2 .
Figure 2. Kinematic modeling of the pole climing robot by solving the above inverse kinematics (6).The inverse kinematics was derived through our serious effort.When the position and orientation of the object are given, we succeed in obtaining the final solution of the inverse kinematics: planar pattern of the pipe (b) projected image of the pipe pattern

Figure 4 .Figure 3 .
Figure 4. Planar pattern of the pipe and its projected image

Figure 5 .
Figure 5.The captured image of the pipe and its processing procedures