An eye and neck coordination strategy based on Jacobi matrix

In order to research the effect of eye and neck rotational velocity on tracking accuracy in the process of target tracking of humanoid vision system, an optimal method of angle decomposition was be proposed based on the Jacobi matrix. By establishing mathematical model, the relationship between rotational angular velocity and decomposition angle is proposed. Through simulation and physical experiments, the relationship between angle of decomposition and rotational angular velocity is approximately linear. Compared with the equalization method, with the increase of angle and angular velocity, the time efficiency of method proposed in this paper increases. The work in this article provides basis for control scheme of target tracking for humanoid vision system.


INTRODUCTION
Human vision system has the features of high speed, precision and stability in target tracking.To ensure the clarity of the image in the tracking process, it is necessary to control the eye and neck coordination movement in real time, constantly change the direction of vision and switch the point of fixation [1][2][3].
Dan Zhang et al, York University in Toronto, Canada, used the technique of Monte Carlo to model and optimize the rules and conditions of the eyes and neck of the 4-degree-of-freedom hybrid robot [4].X.R Li, University of Alborg, Denmark, has proposed a method, which integrated electromagnetic drive spherical robot with three degrees of freedom rotating motion, and developed kinematics and dynamics models [5].Behzad Danaei, the professor of Tehran University, employed the spiral theory to analyze the movement patterns of the bionic visual mechanical structure, and to obtain the position, velocity and acceleration relation between different parts of the mechanical structure [6].Sidney Fels, Columbia University, by simulating human eye movements, established a set of four degrees of freedom camera positioning system, which used the miniature CMOS color video camera to connect a pair of hinges, to simulate the human eye vision eye movement [7].Milad Eyvazi Hesar, Amy Kabir university of technology in Tehran, developed the image of the control architecture for 2 degrees of freedom spherical parallel robot bionic vision through adopting the PID and sliding mode controller [8].Villgrattner, University of Munich, designed 2 degrees of freedom and 3 degrees of freedom parallel eye-movement mechanism, which used the aeromode motor to drive the camera, have a better tracking effect [9,10].Bang, Seoul national university in South Korea, by adopting the bending bar, designed a 3-dof eye-movement parallel mechanism, which can be fixed in the head of the robot and has a good range of motion [11].
Although the current studies on bionic visual motion mechanism of mechanical design and motion control have made a lot of fruitful works, but these studies general applied in specific domain, the coordinated movement of the eye and neck Angle distribution model hasn't yet do deeply research.The research contents in those papers were be used for humanoid service robot, the target tracking is strict with accuracy, and has a certain tolerance to tracking speed.This paper proposed a decomposition method for the motion angle, by introducing the rotation speed of the eye and neck joint, to optimize the condition number of Jacobi matrix to build the mathematical model of eye and neck Angle decomposition in the process of coordinate motion, and ultimately complete mathematical analysis of the human eye and neck coordinated movement.

Humanoid eye and neck system
The paper designed a two levels motion platform including four degrees of freedom to imitate human eye and neck coordinated movement, shown in Fig. 1.
Considering the visual system view angle and the scene application, this paper only allocates to the eye and neck movement angle according to the target movement speed, and rotation angle of vertical direction does not take into account.By the analysis, the motion control factors need to consider are visual axis' rotation angle θ G , neck movement threshold φ, eye movement threshold Φ.According to the above principles and tracking requirements, three different eye and neck control strategies developed to achieve accurate tracking targets.When 0<θ G <φ, the target tracking task completes by the neck movement, driven by the high-precision stepping motor, without the need of eye movement, the rotation angle θ N =θ G .Then φ<θ G <Φ, the target tracking task completes by the eye and neck simultaneously, the neck rotation angle θ N , the eye rotation angle θ E , θ G =θ N +θ E .Else, Φ<θ G , the target tracking task also completes by the eye and neck simultaneously, the eye rotation angle θ E =Φ, the neck rotation angle θ N =θ G -Φ.According to the eye and neck coordination control strategies, this paper proposed an optimized angle decomposition method based on Jacobi matrix when φ<θ G <Φ.The values of φ and Φ determined by experiment.

Rotational angular decomposition of eye and neck
The platform mainly consisted by q N1 -q N2 neck motion structure, q E1 -q E2 eye motion structure, and E l , E r representing bionic eyeball.The model shown in Fig. 2. The O-XYZ is base coordinate system.The O 1 -X 1 Y 1 Z 1 is eye motion coordinate system.A 1 A 2 represents the horizontal position of the binocular.B 1 B 2 is the projection of the initial position of A 1 A 2 in the base coordinate system.θ N is neck rotation angle.h c Ec, h 1 E1, and h 2 E2 respectively represent the position vector of O 1 , E l , and E r after a neck rotation.

Kinematical model
During the process of eye and neck coordination movement, the position vector r=(x,y,z) T of O 1 in the base coordinate system could be represented as follow: OBi is the projection position vector of bionic eyeball A i in the base coordinate system, b 0 is the projected length between A i and O 1 in the base coordinate system.h i is the projected length between A i and B i , and Ei is the unit vector between A i and B i .R is the rotation matrix of bionic eyeball in the base coordinate system.
The position vector of O 1 in the base coordinate system represented as follow: h c is the length between O and O 1 .Ec=(cosαsinβ, −sinα, cosαcosβ) T .

The calculation of Jacobi
The rotational speed of the whole system related to the rotational speed of each degree of freedom.

'
( ) v is the speed of O 1 in the base coordinate system.In order to calculate the Jacobi matrix, equation ( 3) represents the part of the eye movement vector which can be equivalent to: Equation ( 2) derivative of both sides with respect to time, By equation ( 3) and ( 5): Assumed J D = J 1 +J 2 ， in order to introduce the angular velocity of the eyes and neck, the following processing made for the matrix J D : Matrix J is the bionic neck structure Jacobi matrix.Set minimum average condition number of Jacobi matrix as the target for multi-objective optimization, the function f is as follows: f is the average condition number of Jacobi matrix, σ max and σ min are maximum and minimum singular values.By equation ( 8), the f is a function about eye and neck rotation angular velocity, the optimization process with f is the optimization process with the parameters of ω N and ω E .Further, the optimization process with the rotating angular velocity is the process to decompose the horizontal rotation angle into neck' rotation angle θ N and eye level rotation angle θ E in the process of target tracking.Therefore, ω N and ω E determine the decomposition result of the horizontal rotation Angle directly, which is important for the eye and neck coordination movement in the process of target tracking.

The calculation of Threshold
The threshold parameters of eye and neck coordinating motion system related to mechanical and electrical parts actually.In this paper, the neck structure is consisted by the rotating organization and the pitch organization, its precision is 0.01 °.The precision affected by switch current and gear space in a little rotation.Therefore, this paper designed the experiment to measure its stable rotation angle range.
The measurement experiment implemented 10 groups, each group included 40 times.The mean value of error and standard deviation shown in Fig. 3.

Fig. 3 the mean value of error and standard deviation
The Fig. 3 shown that when the steering engine rotation angle between 10° to 24°, the mean error at about 0.08°, the mean error is 0.1° at 8°, after exceeding 24°, the mean error quickly rose to 0.24°.Between 10° to 24°, the error of the standard deviation is 0.04; error standard deviation is 0.08 at 8°, after more than 24°, error standard deviation changes rapidly to 0.15 with poor stability.
The steering engine run more smoothly while rotation angle in [10°, 24°], the neck movement threshold could be set at 10°(φ=10°), the eye saturation threshold could be set at 24° (Φ=24°).When rotation angle is less than 10°, the rotation task completed by the neck structure.When the rotation angle in [10°，24°], the rotation task completed by the neck and eye structure.When the rotation angle is greater than 24°, the eye structure rotated 24°, the neck structure completed the rest angle.

Experiments and analysis
According to the movement speed of the human beings, the rotation velocity will be limited to 30°/s -120°/s.According to the mentioned above, the average condition number f can be described as a function about ω E , ω N , θ E , θ N .To solve the Jacobi matrix average condition number minimum value with different ω E , ω N , get different proportions regarding θ E and θ N .This paper selected the results of 80°, for example, as shown in fig. 4.

Fig.4 The relationship between rotation angular velocity of eyes and neck and angle decomposition
The simulation results show that the ratio of the tracking angle of the eye and neck structure related to their rotation velocity.When ω E fixed, the ratio linear change with the increasing of ω N .
Further, with a fixed rotation velocity ω N (40°/s, 60°/s, 80°/s), ω E (60°/s, 90°/s, 120°/s), for example, the tracking angle of the eye and neck structure shown in Fig. 5.The decomposition angle value is different with different eyes and neck rotation velocity.When the velocity of the neck is greater than that the eye, θ N is also greater than θ E .When the velocity of the neck is less than the eye, θ N is also less than θ E .When the angular velocity is equal, the decomposition angle is also equal and linear.The linear fitting ratio of Fig. 5 shown in table 1.The results show that, within a given range, the rotation angle of eye is proportional to the rotation angle of the neck, and the ratio is the same as that θ E =θ N (ω E /ω N ).The paper takes two Angle decomposition methods to verify the simulation results, according to the above, ω N takes 40°/s, 60°/s, 80 ° / s respectively, ω E takes 60°/s, 90°/s, 120°/s respectively.The first method is the Jacobi decomposition method, and the other is the average decomposition method, the results shown in the Fig. 6.The rotation velocity of eye and neck determined by lots of factors.If the rotation velocity of eye and neck is higher than the processing speed of target tracking algorithm, the loss of data will be generated and the tracking accuracy will be affected.When the speed is too low, the image produces a double image, resulting in the failure of image processing.Moreover, the speed of motor rotation will also restricted by current, power and other factors.Therefore, reasonable selection of the rotation velocity of the eye and neck directly related to the accuracy of target tracking.Based on the above experimental results, this paper selects ω N =40°/s, ω E =120°/s.
It can be seen from Fig. 6 that the time efficiency of the method described in this paper is better than the mean method.And this advantage will be more and more obvious with the increasing of the tracking angle.As is shown in Fig. 6 (a), when the rotation velocity of neck fixed on 40°/s, as the rotation velocity of eye increasing, the time cost for tracking is decreasing relatively.As is depict in Fig. 6 (b), when the velocity of the eye fixed on 120°/s, with the increasing of rotation velocity of neck, the time efficiency gradually reduced.

CONCLUSION
The vision bionics eye and neck angular decomposition mathematical model related to the rotation velocity of eye and neck in the process of target tracking.From the experimental results, within a range of rotation velocity of eye and neck, the percentage of decomposition angle of the eye and neck is the same as the ratio of their velocity.In addition, compared with the mean method, the method based on the Jacobi matrix has advantages in tracking time efficiency, and this advantage will be more and more obvious with the increasing of the tracking angle.When the neck velocity fixed, as the eye velocity increasing, the time cost for tracking is decreasing relatively.When the eye velocity fixed, as the neck velocity increasing, the time efficiency gradually reduced.The systematical mean error is less than 0.1, and the systematical error of the standard deviation is less than 0.04 when the model operates.The research of this paper provided a good reference for the eye and neck decomposition of humanoid robot target tracking, and played a certain role in promoting the development of humanoid robot research.

Fig. 1 .
Fig. 1.Bionic mechanical structure of neck and eyes

Fig. 2 .
Fig. 2. Coordinate of bionic neck and eyes structure

Fig. 5
Fig. 5 Angle decomposition result with giving angular velocity

Fig. 6
Comparison result of time deference between 2 ways of angle decomposition

Table 1 .
Slope of the linear fitting curve