The Depth Map Construction from a 3D Point Cloud

. A depth map transforms 3D points into a 2D image and gives a different view of an observed scene. This paper deals with a depth map construction. It describes the whole process, how to transform any 3D point cloud into a 2D depth map. The described method uses 3D rotation matrixes and the line equation. This process allows to create the desired view from arbitrary point and rotation in an exploration space. Using of a depth map allows to apply image processing methods on depth data to get additional information about an ambient space.


Introduction
Depth maps (RGB-D) in mapping systems and robot navigation [1]- [4] are nowadays often used, but also in medical area for the respiratory motion detection [5], and in other areas for the motion and object detection.
For the 3D range scanning we are using a mobile platform with a laser and a camera, described in [6].A laser spot is dispersed to a vertical line which allows measurement in the height range.In our previous work we introduced the new modelling of a colored laser line by using the laser Gaussian Mixture Model (GMM) [7].The last paper describes an elimination of the barrel distortion correction [8].These works contribute to the increase of the measurement points amount and measurement precision.The described 3D range scanning system is a part of the bigger project called ARES (Autonomous Research Exploration System) [9][10][11].Our system is able to create a 3D point cloud, but up to now was not possible to create the depth map, which is useful for mapping systems mentioned above.
This paper describes the new algorithm for the depth map construction form a point cloud.This will give more information about depth data and it will allow to apply image processing methods 2D depth data.

Optical range finder
Our 3D range scanning system for the autonomous mapping, see Figure 1, is based on the triangulation method.The principle of the distance determination and the 3D point cloud constructions is introduced in [6].
The whole measuring device is pivotally carried by tripod.For the image capturing a high-quality Basler color camera with a resolution of 2590x1942 pixels is used.The used laser diode has an output power 200 mW, especially for good recognition after vertical swapping.The laser spot is dispersed by an optical filter with an angle of 90 degrees.The 360 degrees rotation of the measuring head provides a powerful and fast stepping motor.The maximal measured distance is 15 m. Figure 1.3D range scanning system.

3D point cloud
The Figure 2 gives measurement results of one frame, which includes 80 measurement points.The green laser line is segmented by the GMM, described in our previous work [7].
After one 360 degrees scan a 3D point cloud is created.An example of the 3D point cloud is in Figure 3.This measured point cloud is used for the description of the depth map construction process in the following Section 3.

Depth map construction
For a depth map construction there are several initial parameters.The first is the camera position ( , , ) The vector P C (1) is a position from the depth map will be created.The second parameter is the camera field of view: horizontal H FOV ; vertical V FOV , , , , 2 2 2 2 The expression (2) is the field of view in the top, bottom, left and right area.It gives the used camera view range, which is able to cover.The last important parameter is the camera orientation defined by the following vector ( , , )[ ] This allows "to look" in the space in any desired orientation.The main advantage of the described algorithm is the possibility to set these parameters according our requirements, to create the desired resultant depth map.
For next described steps the two mathematical tolls are needed.For points rotations and transformations, the rotation matrixes in the 3D space (4), ( 5) and ( 6) are needed, described in paper [9].To express the view range, the line equation ( 7) is used.) tan(D (7) The line parameter offset b is zero, tan(α) is the line angle and x express the point where y is calculated.

3D point cloud initial preparations
In the initial stage the 3D point cloud is rotated according the camera orientation C O (3).Although the input 3D point cloud is rotated, this rotation simplifies data selection process which describes the next Section 3.3.For the algorithm illustration C O = (0, 0, 90) is used.In

Range view construction
The first algorithm step is the view range construction by using the line equation (7) and rotation matrixes in Y ( 5) and Z (6) axis.The line point step q L is 1 cm.Firstly, the "V" (consist from two lines) with the V FOV angle is created at the level Z = 0, subsequently the "V" is rotated in the angle of 90 degrees around Y axis (5).This rotated "V" is further rotated in angle of H FOV /2 around the Z axis (6) in both sides (left and right).The whole range view is shifted to the camera position P C = (1.79,3.3, 1.2).The result is shown in Figure 5.There are also orientation angles C O (3) marked.
By using of the constructed view range, see Figure 6, the points for the depth map creation can be selected.To get the point of data selection describes Figure 7.There are marked four grey rectangles, which shows stages of the segmentation process.

Door
where n is the view range line point from total N L points (grey PC is the point cloud.Values y and z means points in the appropriate axis Y and Z. Parameter R S is segmentation rectangle and indexes min and max means the min position value in the appropriate axis and vice versa.The final segmentation equation ( 9) by using equation ( 8) is the following The last unknown value is x which are points in the axis X.The whole segmentation process is logical sum of all points which comply the condition (9) (all points inside grey rectangles).The next Figure 8. shows the segmentation result with the marked view range.

Points depth image transformation
The final step is to transform points into the depth plane.The following Figure 9. illustrates points transformation.There are three red transformation lines for the demonstration.The depth plane size is the last grey rectangle in Figure 7.The depth map resolution is possible to select by parameter q DM , which express the depth map quantization.For the testing example parameter q DM is equal to 1 cm, and then the final depth map horizontal resolution W is 724 px and the vertical resolution H is 662 px for the used point cloud.
The direct distance to the depth plane D DH is known from the data range in X axis.Parameters d Pz and d Pz are the subtracted vectors P t and P C .The distance D P to the measured point P t is the point norm of vector in R 3 ¦ c t P P P D (10) Angles φ y and φ z are calculated by using of goniometric functions.From known parameters is estimated the red line angle and calculated the distance D PDH .This allows to express the red line point D(x,y) and get exact position x and y in the depth plane.This process is done for all points in point cloud.If two points are projected on the same position the closer point is used.
The depth map pixel intensity is equal to point distance D P quantized by 16 bit.The maximal intensity 2 16 is equal to the maximal rangefinder range 15 m.
It is possible to assign all points directly in the depth plane from their position in space, but the main goal is to create the depth map from the camera view point P C .For this purpose, it is necessary to use the line transformation, showed in Figure 10.The point cloud typically contains points which can be covered by other objects like tables and etc. and these points is necessary to suppress in the resulting depth map.In this depth map construction process the Z-buffer method it is used [12].The Z-buffer is known from computer graphics.When an object is rendered the Z-buffer ensure that the hidden points are not shown in the output frame.Practically the Z-buffer is the depth map.The point cloud does not give information about surfaces or points connection.The resulting depth map is therefore checked for the farther points in all points neighbourhood.If in the neighbourhood area of 10 px (q DM = 1 ~ 10 cm) lies a pixel with the distance farther than 0.5 m, these pixels are erased.These are estimated like covered by the closer pixel.

Resulting depth map
If the parameter q DM is small like in this example 1 cm, it is possible to use the morphological operation "dilate" to increase the depth map point size.In the resulting depth map Figure 11. the dilatation by the element size 10 is used.

Results evaluation
The resulting depth map it is possible display also in surf graph, see Figure 12, which allows to "look" on the depth map in different view.The table, door and cupboard is obvious in this representation.The Figure 15.shows the increasing distance to the edge of the room.The fineness effect is more obvious here.Points discontinuities in the closer distance are caused by the lower density of input point cloud.This is possible to solve by increasing the parameter q DM , which has influence on depth map details.

Conclusion
In this paper we presented the algorithm for the depth map construction.We described the whole process with our optical rangefinder parameters.The main advantage of this algorithm lies in possibility to create a depth map from any point cloud with any camera position and any camera orientation.
There can be some black parts and holes in the resulting depth map.This disadvantage mainly depends on the used point cloud and the selected detail parameter q DM , which is necessary to choose with the deliberation.
Symbols α, β and γ are rotation angles in corresponding axes X, Y and Z. b x y

Figure 4 .
are points rotated in C O angles.

Figure 4 .
Figure 4. Rotated point cloud in angle of 90 degrees (Z axis)

Figure 5 .
Figure 5. Constructed range view with rotation angles.

Figure 7 .
Figure 7. Points segmentation illustration.The equation (8) describes the points which lies in grey rectangle R SYZ in Y and Z axis

Figure 8 .
Figure 8. Segmented points from input point cloud.

Figure 9 .
Figure 9. Illustration of point transformation.Each point form the segmented point cloud has to be transformed into the final depth plane.The detailed transformation line description is in Figure10, where the important transformation parameters are marked.The distance parameters in Figure10marked by the uppercase letter means the direct distance to the depth plane and the parameters with the lower case letter are differences between two points.The parameters are the following: D(x, y) is the transformation point P t position in the depth plane; D PD is the direct distance to depth camera position P C ; d y and d z is the distance offset from the depth plane centre; angles φ y and φ z expresses the transformation line orientation; D P , d Pz , d PH are parameters of the point triangle; D PD , d z , D PDH constitute the second triangle (to the depth plane) and W and H is the depth map resolution.

Figure 11 .
Figure 11.Resulting depth map.The black pixels in top and bottom part of the resulting depth map are caused by the used input point cloud.There are missing points to display them in the created depth map.The darker pixels are closer to the camera position PC.The fineness and compact image is in farther distances where is the higher density of measurement points.

Figure 12 .
Figure 12.Surf depth map interpretation.The next figures (Figure 13. 14. and 15.) show the depth map with the different camera position and

Figure 13 .
Figure 13.Depth Map in different orientation.

Figure 14 .
Figure 14.Segmented points for second depth map.

Figure 15 .
Figure 15.Surf interpretation for second depth map.