Research on manipulator grasping method based on vision

Aiming at the problem that the manipulator cannot grasp the object accurately when the robot changes the position and pose of the object in the process of static grasping, a set of manipulator grasping method research based on vision is proposed. Firstly, a set of camera detection and robot grasping system model is built. Secondly, each coordinate system is created for the grasping system, and the transformation relation between each coordinate system, matrix model and the quantity that needs to be calibrated are introduced in detail. Thirdly, the trajectory function from the image coordinate system to the coordinates of the manipulator is obtained through the direct linear method calibration experiment. Finally, the experimental platform for camera detection and manipulator grasping objects based on xavis was built, and the grasping success rate of the experimental platform was tested. The experimental results show that the grasping error rate of the experimental platform is within the control range. Therefore, the manipulator grasping method based on vision is of reference significance for engineering applications.


Introduction
Under the background of increasing labor costs, difficulty in recruitment, increasing demand for diversified production and stable development of automation technology, more and more industrial robots are involved in industrial production activities, among which solving the problem of product positioning and grasping is an important problem that industrial robots will inevitably encounter in practical application [1][2].In the past, the most common solution was to fix the position and posture of the product, and then control the industrial robot to grab the product through manual teaching in advance and action reproduction in the later stage [3].When the actual position of the product deviates from the teaching position, the traditional way is difficult to meet the requirements. The emergence and development of machine vision technology provide new solutions for the positioning of industrial robots [4].
Liu Zheng-qiong introduced how to realize the recognition of parts larger than the camera field of view based on local feature matching [5]. Ni He-peng. introduced image de-repetition algorithm and acceleration and deceleration control algorithm in the process of robot tracking. Zheng hua. mainly studied CSARA robot operational model and methods of extracting and processing visual image features [6].Chen Zhi-wei. introduced the image processing, control algorithm and control program in the system of visual guidance parallel robot [7].
The above literatures focus on the research of visual image extraction and processing methods, or the dynamics control of the robot itself, but do not involve the coordinate system pose data transformation and calibration methods among vision, conveyor belt and industrial robot. In this paper,the conveyor belt, intelligent camera and industrial robot as research objects, proposes a complete set of object pose data calculation methods based on camera detection [8]. Through the transformation formula of coordinate matrix created, only the visual detection data need to be read, and the pose data of the product coordinate system in the robot world coordinate system can be calculated automatically [9].

Construction of camera detection and robot grasping system platform
Firstly, the visual robot is fixed on the robot operating platform through the chassis, and then a single visual robot workbench is assembled through the connector. The two visual robots are distributed on both sides of the conveyor belt. The physical position layout of mechanical arm, camera and assembly base in the robot grasping system platform is shown in figure 1. Select one of the arms to complete the grab task. The specific placement size is shown in table 1, and the robot static tracking and grasping system is shown in figure 2.   3 Kinematics analysis of manipulator

Analysis of the relationship between the creation and transformation of coordinate system
The objects controlled in this paper are all eye-in-hand systems [10]. The following is an eye-in-hand robot as an example to illustrate the relationship between the coordinate systems of the system. Let {W} represent the base coordinate system of the robot, {E} represent the coordinate system of the robot end-effector, {C} represent the camera coordinate system, {I} represent the image plane coordinate system, and {O} represent the target object coordinate system [11]. The position relationship between the coordinate systems is shown in figure 3.

D-H parametric method
In this paper, D-H parameter method is used to model the mechanical arm [12]. Firstly, the coordinate system of each link is established separately, the pose relation of adjacent link is obtained by using homogeneous transformation matrix, and the pose relation of the coordinate system of end-effector relative to the base coordinate system is obtained by transformation in turn, and the kinematics model of the manipulator is established. The parameters of the D-H of figure is shown in figure 4.The position and attitude of one coordinate system relative to another can be obtained by coordinate transformation. This relationship can be described by four homogeneous transformation ( ) T i matrices.
Equation (1) is the homogeneous matrix of the transformation of robot connecting rod. The transformation between the coordinate systems of connecting rod with any degree of complexity can be described by taking the product of the appropriate number of the matrix. In the control of the manipulator, it is usually necessary to obtain the spatial pose relationship between the tool coordinate system {T} and the universal coordinate system {S}.The transformation matrix of tool coordinate system and universal coordinate system can be finally obtained by measuring the position and pose relationship

Object identification and location
Object recognition is realized on computer, object features are recognized by machine learning, and objects are located by image processing.

Support vector machine
Machine learning algorithms, it can be trained with a small number of samples. SVM sample space is established, information of different samples is extracted and mapped to the sample space, and classification is conducted by establishing segmentation boundary (hyperplane) for different samples [13].
Take 50 image samples for each object again to identify the object. The images were converted into binary images after processing, as the judgment basis of the support vector machine, the recognition test results were shown in table 2. The overall recognition success rate was 98%.

Position of object
After the classification of the object is determined, the appropriate grasping method is selected according to the shape of the object for grasping. The position of the object with circle characteristics is determined by detecting the position of the center of the circle. The direction of the object is arbitrary.Hough circle transform (HFT) is an accumulator based voting mechanism used to detect circular features. For a circle with a center of (a, b) and a radius of r, the functional expression of the circle is shown in formula (3).

Direct linear calibration
The method to realize the direct linear calibration method is: place the workpiece on the conveyor belt, use the moving workpiece detection algorithm to detect the position of the workpiece, and read the coordinate information of the workpiece in the image space [14]. The industrial robot trainer is used to move the robot gripper on the manipulator to the position above the workpiece centroid and read the position coordinate information of the manipulator at this time. Read the position coordinates of the workpiece at different positions on the conveyor belt, establish the mapping relationship between the position coordinates of the workpiece in the image coordinate system (U,V) and the position coordinates of the mechanical arm coordinate system ( , , ) w w w X Y Z , and complete the calibration of the camera.  relation curve between the image coordinate system and the workpiece position in the robot coordinate system is shown in figure 8.
In order to obtain the functional relation expression of the curve in the figure 8, the curve in figure 8 needs to be fitted and The curve after fitting is shown in figure 9. Because of the first-order linear function, there will be a large error and generated the robot arm cannot be guided to track and grasp accurately. In this paper, the second-order least square method is used to complete the fitting. The mathematical expression is formula (4), where a, b and c are the parameters obtained by curve fitting. When the image processing algorithm substitutes the centroid coordinates of the extracted moving workpiece into the function expression, the position coordinates of the workpiece can be calculated.

Conclusion
After the experimental platform is set up and each coordinate is calibrated, static workpiece is placed for grasping. The image coordinate, the coordinate calculated by the fitting function and the actual captured coordinate are shown in table 4. The actual captured artifact is shown in figure 10 and the grasping success rate of workpiece with different colors is shown in table 5. From the grasping results, the color characteristics of red and blue workpiece are obvious and the grasping success rate is high, while the color characteristics of black workpiece and spring workpiece are not obvious and the grasping success rate is slightly lower.The maximum detection error of uniaxial coordinate data of each workpiece is within 0.5mm, which meets the grasping requirements of mechanical arm.It is of reference significance for engineering applications. In the next step, on the basis of the research results, the robot arm will be further studied to dynamically grasp objects, so that the industrial robot can accurately grasp moving objects in space.