Development of Registration methodology to 3-D Point Clouds in Robot Scanning

The problem of multi-view 3-D point clouds registration is investigated and effectively resolved by the developed methodology. A registration method is proposed to register two series of scans into an object model by using the proposed oriented-bounding-box (OBB) regional area-based descriptor. Robot 3-D scanning is often employed to generate set of point clouds of physical objects. The automated operation has to successively digitize view-dependent area-scanned point clouds from complex shaped objects by multi-view point clouds registration. To achieve this, the OBB regional area-based descriptor is employed to determine an initial transformation matrix and is then refined employing iterative closest point (ICP) algorithm. The developed method can be used to resolve the commonly encountered difficulty in accurately merging two neighbouring area-scanned images when no coordinate reference exists. The developed method has been verified through some experimental tests for its registration accuracy. Experimental results have preliminarily demonstrated the feasibility of the developed method.


Introduction
A 3D model is a digital representation of a physical object, which can be used for many purposes.In manufacturing, producers apply 3D model to make new products, perform dimensional and comparative analysis of parts, or inspection robotics.For medical industry, 3D models play an important role as inputs in organ fabrication, creation of customized prosthetics, implants, and anatomical models.They are also used in architecture industry to demonstrate proposed buildings, in the movie industry as characters and objects for animated and real-life motion pictures.
Recently, automatic CAD model generating technology, known as many different names including 3-D scanning, 3D digitizers, laser scanning, industrial CT, LIDAR, and others, has an emerging interest.The common factor of all these devices is that they capture the geometry of physical objects and describe them through thousands or millions measured points called point clouds.This kind of technique is the most efficient way to generate a CAD model from a physical object that has any kind of complex or freeform shape.3D scanning technology can be applied at any point in a typical manufacturing cycle, saving time, money and material.Because of the dominant advantages, 3D scanning systems have become more and more popular with numerous applications such as 3-D printing, reverse engineering, quality inspection, prosthetics, etc. Specially, reverse engineering (RE) is an efficient approach to significantly model the product and reduces the product development cycle [1][2][3][4].
According to the sensing principle being employed, the current solutions can be generally classified into two main categories, namely hand-guided and automated scanning techniques.Hand-guided scanning allows for acquiring arbitrary shapes.However, the effectiveness of this scanning method highly depends on the skills of the user and the scanning process is generally time consuming.To enhance the efficiency, the 6-axis robot arm integrated with 3-D imaging scanners has recently emerged as a technical developing trend for 3-D surface scanning for objects having arbitrary or complex geometry [5][6][7].Both Callieri [5] and Larsson [6] presented a system for automated 3-D modeling consisting of a 3-D laser scanner, an industrial six axis robot and a turntable.The advantage of this system is that it automatically achieves the shape of the object from initial scanning and thus, if possible will scan the object using an orientation of the scanner which gives the most accurate result.Meanwhile, an autonomous 3-D modeling system, consisting of an industrial robot and a laser striper, for efficient surface reconstruction of unknown objects was proposed by Simon Kriegel [7].The system iteratively searches for possible scan paths in the local surface model and selects a next-best-scan (NBS) in its volumetric model.
The working principle of the above robotic 3-D scanning system is that the optical probe is attached on the robot end effector, in which the probe is controlled and scanned continuously by the robot in the working area and simultaneously posed with different gestures in the process of scanning.Shown in Figure 1, initially, the robot moves to a pre-defined position for the first probe scan.By doing so, the system uses the acquired image to determine the initial object orientation.According to the processed information, the position and gesture of robot for next view are further estimated and employed to generate the robot path for next movement.This process is repeated until the whole object is scanned.In the scanning process, the system controls the robot position and gesture to continuously scan and register point clouds of consecutive scenes of the scanned object in real time.However, due to the lack of a global registration between successively scanned data, the existing scanning techniques still fail to robustly satisfy accurate object digitization with 100% surface coverage.Figure 2 show two cases with missing some part of object surfaces after performing automated object digitization using robot scanning.This issue is mainly caused by the fact that 3-D sensor cannot measure point data from the scanned object surface that is in contact with its supporting ground.
Thus, modeling scanning completion with 100% surface coverage still remains one of the most challenging problems in the current 3-D object digitization.Accurately registering point clouds of two series of scanned point clouds into the reconstructed object model without an established coordinate reference has been proven to be a difficult process [8][9][10][11][12].To overcome the above difficulty, an automated robot scanning system integrated with a NTUdeveloped optical 3-D measuring probe is developed on a global registration method to accurately register two series of scans into an object model underlying 3-D imaging digitization by using the proposed orientedbounding-box (OBB) regional area-based descriptor.The hardware and basic system configuration as well as data processing algorithms such as preprocessing, overlapping detection and robot path planning have been described in [12].
2 Global registration method based on the OBB regional area-based descriptor

Principle concept and flow chart diagram of the global registration method
To find the transformation between two series scans that are acquired from multi single scans, the OBB regional area-based descriptors of source point clouds are matched with the descriptor of the target point clouds.Each the proposed descriptor includes two components.The first component contains the information about the OBB of the point clouds.Each OBB is represented by a corner C(x C , y C , z C ) and three vectors CC 1 (x max , y max , z max ), CC 2 (x mid , y mid , z mid ), CC 3 (x min , y min , z min ) corresponding with the maximum, middle, and minimum dimensions of the OBB, respectively.The second component represents the distribution of the surface area of the object in the OBB.The similarity between the regional area based descriptor that represents the source point clouds and the regional area based descriptor that represents the target one is determined by using the normalized cross correlation.
To make the proposed method clear in its operation procedure, the following Algorithm 1 is used to describe the proposed method in its main 6 steps.3. Define and perform rough rotation Q based on applying initial three-axis alignment between P and Q, which is an orientation alignment between two OBBs along the three principle axes.
4. Generate descriptors of P and Q.
a. Calculate the OBB regional area-based descriptor of P.
b. Rotate Q around the three axes of the coordinate system defined by the origin is center of gravity of Q and three unit vectors considered as the three directions of the OBB of point clouds P with every increment 'T on each axis and generating the OBB regional area-based descriptors of Q.

Definition of translation matrix
Assuming that two series of scans to be registered are represented as two sets of unorganized points P = {p 1 , p 2 ,…, p k } and Q = {q 1 , q 2 ,…, q k }, the initial transformation, which represents the coarse alignment between two series, can be estimated as follows.
If P and Q represent the same free-form shape, the least squares method can be used to calculate the initial rotation and translation by minimizing the sum of squares of alignment errors: Where R is the rotation matrix, t is the translation vector, p i and q i are the corresponding points pair between the P and Q point clouds.
Let p be the centroid of the set of corresponding points in P {p 1 , p 2 , …, p k } and q be the centroid of the set of corresponding points in Q {q 1 , q 2 , …, q k }. p and q can be defined as: The new coordinates of points are denoted by: The registration error term is then expressed and rewritten as: ' ' (8) where The sum of squares of errors becomes: The first term in Equation ( 9) does not depend on tc .Therefore, the total error is minimized if 0 t c , and then: q p R t (10) Therefore, the initial translation matrix can be computed based on the difference between the centroid of P and the centroid of Q.
In addition, the weight loss at missing parts of two scan series may result in a significant bias between the translation matrix being estimated (by the difference between the centroid of P and the centroid of Q) and the real translation one.The shifting of two gravity centers (the red and green points shown in Fig. 3) being created using a virtual camera from a model having the centroid (white point) in Figure 3 (a) illustrates the influence of weight loss from the missing points to the centroid point position.Assume that the two point sets are separated as shown in Figure 3 (b).The accurate translation matrix to merge set Q (green point clouds) with set P (red point clouds) is defined through the difference between the current centroid of P and the original centroid of green point clouds (green points).Therefore, it is necessary to estimate the original centroid of P on the object for translation.The boundary points located nearby missing part can be detected through the geometric relationship with the support plane and using a set of connected edges that satisfy two requirements, in which all edges have a similar orientation and each edge lacks an assigned triangle, either on the left or on the right side [7] (as shown in Figure 3 (c)).Then, the RANSAC algorithm [16] is used to fit a mathematical model to the boundary points in each subdivided box of the OBB (The total number of subdivided boxes in the OBB is m = k 1 x 1 x 1).After that, applying the rough rotation matrix to the point set of Q, the original centroid of green point clouds O G can be estimated as follows: where O W is the centroid of the scanned object computed in Equation (12); O GM is the centroid of missing points of Q, F v is the surface area of object in each subdivided box from the missing points of Q being computed based on the fitted mathematical model; and, f v is the surface area of the object in each subdivided box from green point clouds Q.
As a result, the initial translation matrix can be computed based on the difference between the current centroid of Q and the original centroid estimated O G .

Alignment of two series scans based ontheir OBBs' orientation
The oriented bounding box is a rectangular bounding box that covers entire object point clouds.Each OBB is represented by a corner C(x c , y c , zc) and three vectors CC 1 (x max , y max , z max ), CC 2 (x mid , y mid , z mid ), CC 3 (x min , y min , z min ) corresponding with the maximum, middle, and minimum dimensions of the OBB.The orientations of the OBB can be determined by using the covariancebased method, which is proposed by Gottschalk, 1996 [14].).Alignment of Q to P based on their OBBs' orientation leads to change of the coordinates of the point q i (x i , y i , z i ) as follows:

The OBB regional area-based descriptor
We assume that the three directions of the OBB are divided into k 1 , k 2 , and k 3 segments (as shown in Figure 5 (a)), the total number of subdivided boxes in the OBB is n = k 1 x k 2 x k 3 .The surface area of the object in each subdivided box V ijk (i = 0, ..., k 1 -1; j = 0, ..., k 2 -1; k = 0, ..., k 3 -1) is denoted as f v , where v = k(k 1 k 2 ) + j(k 1 ) + i. f v can be determined by subdividing the considered surface into the triangle mesh [15] (as shown in Figure 5 (c)).The distribution of S v is defined as the regional area-based descriptor FV = {f v , v = 0, …, n}, shown in Figure 6 (d).

Generating database and calculating descriptors
To define initial rotation matrix, we can consider the coordinate system O G x p y p z p with the origin O G defined in Equation ( 20) and three unit vectors Q 1 , Q 2 , and Q 3 , which are considered as the three directions of the OBB of point clouds P. The regional area-based descriptor of point clouds P is FV P = {f Pv , v = 0, …, n} in the OBB of P. Rotating Q around the three axes of the coordinate system O G x p y p z p is performed with every increment 'T on each axis.For each rotation, the regional area-based descriptor Q in the OBB of P is FV Q can be then determined as described as Algorithm 2. To enhance the accuracy, the descriptor FV P and FV Q can ignore the subdivided boxes in which the missing part of point clouds P are located.

Algorithm 2. Generation of database Input:
Series scans Q = {q 1 , q 2 ,…, q n }, the OBB of P Output: A set of descriptors of Q at different poses Parameters: (-θ x ,θ x ), (-θ y ,θ y ), and (-θ z ,θ z ) are ranges for rotation about the O G x p axis, O G y p axis, and O G z p axis respectively.

Define rotation by searching the best matching pose between P and Q using the descriptors matching.
The similarity between the regional area-based descriptors that represent the object point clouds P and Q is determined by using the normalized cross correlation.The normalized cross correlation between where and The rotation of Q is the initial rotation, in which the normalized cross correlation Although the parameters of transformation matrix can be obtained in the initial registration, the accuracy may not be satisfactory due to the error caused by the limited number of the rotating Q iterations.Fortunately, a refined registration like the iterative closest point (ICP) algorithm can be performed to achieve precise registration between multi-view point clouds.

Case study on synthetic data
In this section we provide a discussion on influence of parameters to performance of the proposed method.To estimate the computation cost of the registration process, the experiments are processed on a computer with a Core i7 processor having 3.40 GHz and 8 GB RAM.In real application, it is very difficult to distinguish different error sources such as shape measurement error (noise, surface sampling, etc.), correspondence error (occlusion, outliers, etc. ), and registration error.In order to evaluate the coarse registration errors, we have generated synthetic data, which is provided by the Industrial Technology Research Institute (ITRI) (as shown in figure 6).The dimensions of socket model and connector model are 45 x 25 x 25 mm 3 and 41 x 33 x 25 mm 3 respectively.Then, the point clouds corresponding with different views of the CAD model are extracted by a virtual camera.The motion between green and red point clouds is a displacement of 50 mm in each axis and a rotation of π/4 around a fixed axis.Therefore, we have precise knowledge of motion parameters rotation matrix R and translation vector t to serve as reference for error estimation and validation of the coarse registration methods.The measures used to determine the accuracy include rotation error and translation error.In order to evaluate the accuracy, the estimated transformation is compared to an expected one.
Results obtained shown that sampling does not affect considerably the accuracy of registration results.Thus, low resolution surfaces can be used in proposed approach to reduce the computation time.Meanwhile, as is shown in table 1, errors in proposed registration algorithms extremely depends on the rotation increment 'T (rad).To enhance the accuracy, the rotation increment should become smaller.But the increment has a significant impact on the runtime of the proposed coarse registration process.As an example, the experiment with 1000 points in table 1, computation time in the case 'T =0.01 (rad), at 126.439 (s), was much higher than that in the case 'T =0.05 (rad), at only 1.063 (s).
From table 2, it can be seen that if the OBB contains many subdivided boxes, it is more expensive to find the best rotation.For a fast registration procedure, less subdivided boxes are preferable.However, it is also required that the number of subdivided boxes is great enough to achieve expected rotation.For instance, in experiment using connector model of 1000 points, the rotation error rose to 0.057 (rad) for the OBB n = k 1 x k 2 x k 3 = 4 x 5 x 6, while it reached just 0.017 (rad) if the OBB n = k 1 x k 2 x k 3 = 7 x 8 x 9.
Besides, the ratio of the overlapped area is especially crucial.As shown in table 3, translation and rotation errors in proposed approach grow directly proportional to the percentage of non-overlapping region.This change is especially significant with over 40% of outliers.This is because it is difficult to estimate translation exactly without high level of overlapping.The excellent results are obtained for over 80% shape overlapping.In this case, the translation and rotation for tested models were at under 1.5 (mm) and 0.02 (rad), respectively.As described in the introduction section, the proposed registration method is developed to register two series of surface scans which exist certain degree of optical occlusion from each individual viewpoint of the probe scanning.To accurately register two series of scans under such the circumstance, it is important to have them overlapped as much as possible.However, realistically, the overlapped ratio really depends on the complexity of scanned object's geometry and how the object is placed on the rotation table.In general, the higher the overlapped ratio between two series of scans, the more accurate the registration can be achieved.

Socket model
Connector model

Case study on measured data
To verify the feasibility of the developed methodology, some experiments were performed and evaluated.In the experiments, the point clouds of a hammerhead were acquired from a 3-D optical scanner being developed by using random speckle pattern projection and triangulation measurement principle.The dimensions of the hammerhead are approximately 110 x 35.5 x 21 mm 3 .Firstly, the hammerhead was placed on a fixed table, allowing for viewing the object from several positions around the object.However, not all the object surface could be detected due to optical occlusion.Therefore, after the 1st series of automated scans to obtain point clouds P, the object was reoriented manually and the 2nd series of automated scans was continuously carried out to acquire point clouds Q.
Rotating Q around the three axes of the coordinate system O q x q y q z q with an increment of 'T on each axis can be used to calculate the normalized cross correlation curves, shown in The performance of the registration can be estimated by using the distance from each overlapping point q i in the point clouds Q to the fitting control point p i that is the projected point of q i onto the triangle mesh of P. If d i denotes the distance between a point p i in P and its closest neighbor point q i in Q, the mean distance P and standard deviation V, which are used to evaluate the performance of the object registration, are computed as follows:

Conclusions
In this study, a global registration method is developed to overcome one of the most challenging problems remaining in current 3-D scanning systems for accurate image registration even for no reference coordinate existing between neighboring scanned surface patches.The key innovation in this work is the strategy in determining a robust registration transformation between two neighboring scans.The experimental results demonstrate the validity and applicability of the proposed approach.The registration error can be controlled within a few tens micrometers in one standard deviation for a size of object reaching 100 millimeters.

Figure 1 .
Figure 1.Overview of automated 3-D scanning process using a 6-axis robot.

Figure 2 .
Figure 2. Illustration of two series of scans with missing part of object surfaces.(a) Point clouds of hammer; (b) Point clouds of connector

DOI: 10 2 .
Two series scans P and Q Output: Transformation matrix 1. Define and perform translation Q to P based on center of gravity of point clouds and estimation and enclosing the missing regions.Determinate individual OBBs of two series scans.

Figure 3 .
Figure 3. Improvement of translation: (a) The influence of weight loss from the missing points to the centroid point position; (b) Two point sets are separated; (c) Detection of the boundary points located nearby missing part; (d) The process of estimating the original centroid from two point sets being separated.

Figure 3 (
d) indicates the process of estimating the original centroid from two point sets being separated.The center of gravity of the scanned object O W can be estimated as follows: O R is the centroid of red point clouds P; O RM is the centroid of missing points of P; F v is the surface area of object in each subdivided box from the missing points of P being computed based on the fitted mathematical model; and, f v is the surface area of object in each subdivided box from red point clouds P.
Each OBB is represented by three dimensions and three directions.The OBB matching process finds the view of the model that has the same dimensions with the object point clouds.The OBB of the point cloud of 1 st series scans P is represented by three vectors C 1 C 11 (x 1max , y 1max , z 1max ), C 1 C 12 (x 1mid , y 1mid , z 1mid ), and C 1 C 13 (x 1min , y 1min , z 1min ).The OBB of the point cloud of 2 nd series scans Q is represented by C 2 C 21 (x 2max , y 2max , z 2max ), C 2 C 22 (x 2mid , y 2mid , z 2mid ), and C 2 C 23 (x 2min , y 2min , z 2min

Figure 4 Figure 4 .
Figure4shows a case with point clouds being derived from two series of scans before and after employing the rough rotation by alignment of two series scans.

Figure 5 .
Figure 5. Illustration of the OBB regional area-based descriptor.(a) The OBB of the object is subdivided into smaller boxes; (b) The object point clouds and OBB; (c) Triangle mesh is generated over the point cloud data; (d) the regional area-based descriptor of the object.
this experiment, the mean deviation value is 0.0347 mm and the standard deviation value is 0.0235 mm.

Table 1 .
Experimental results using connector model obtained by proposed coarse registration method with different samplings and increments 'T.