D road surface reconstruction based on point clouds data assimilation algorithm

Urban areas 3D model reconstruction is one of the major fields of application of 3D scanning technologies. In the future, vehicle-based laser scanning, here called mobile laser scanning system, should see considerable use for 3D road environment modelling in urban areas. In this context, one of the main limitations perceived by the mobile laser scanning system is the incompleteness of the sampling. Whenever we scan urban area road environment, the produced sampling usually presents a large number of missing regions. Many algorithmic solutions exist to close those gaps from specific hole filling algorithms to the drastic solution of using water-tight reconstruction methods. In this paper, a method for filling holes of road surface point clouds and generating 3D model of road surface from mobile laser scanning data is developed. The data is classified into road surface, on-road and off-road surface point clouds. Many large holes in the road surface point clouds are filled by using data assimilation algorithm. Then, the road surface is 3D modeled as a triangulated irregular network. It is shown that the whole road surface 3D model is integrated after data processing. The above mentioned steps are applied to a large set of mobile laser scanning data of urban area road environment, in order to obtain the whole urban road surface 3D model.


INTRODUCTION
As scientists, engineers and planners require ever more detailed information about our urban areas the requirement for accurate 3D mapping of road and city structures is likely to increase considerably in the years to come.The technology available for providing 3D mapping may be applied from the ground using static instrumentation or applied from an air or space based platform.Advanced digital mapping tools and technologies are so-called 3D laser scanner are enablers for effective e-planning, consultation and communication of users' views during the planning, design, construction and lifecycle process of urban areas.The regeneration and transformation of cities from the industrial age to the knowledge age is essentially a 'whole life cycle' process consisting of: planning, development, operation, reuse and renewal.In order to enhance the implementation of urban areas solutions during the regeneration and transformation of cities, advanced digital applications can have a significant impact.However, the rapid development of laser scanning has shifted, since the middle of the first decade of the new millennium, from airborne laser scanning (ALS) systems and terrestrial laser scanning (TLS) systems towards the mobile laser scanning (MLS) systems.A MLS system integrate GPS/INS system and data acquisition sensors on a rigid moving platform for collecting laser scanning point clouds from the surroundings of the mapping system have been developed since the early 1990s.A good overview is presented in Barber's research article [1].At present, the integrated GPS/INS/LS system is the key component in both MLS systems and ALS systems.MLS systems are different from ALS systems which assembled on an airplane or a helicopter, but fixed on platforms such as a car or a van.Compared to ALS systems, which typically collect point clouds with a resolution of 0.5-40 points/m 2 from an altitude of 100-3000 m, MLS systems provide point clouds with a resolution of hundreds or even thousands of points/m 2 from a distance of some dozens of meters.ALS and MLS data-sets complement each other in several ways as they have different viewing geometries.For example, ALS data-sets have become an important source for object extraction and reconstruction for various applications, such as urban analysis (the roofs of buildings) [2][3][4]; vegetation analysis [5]; landform mapping [6]; DTM generation [7,8] and forest inventory [9][10][11].However MLS data-sets are not only including the application of vegetation analysis [12][13][14], but best for detecting objects of urban areas, e.g., walls of building and collecting even more information from road surface [15], In the case of urban areas the detection and quantification of road surface is important for the implementation of urban areas solutions during the regeneration and transformation of cities.On the other hand urban road surface models are needed for accurate three-dimensional mapping of urban areas.

MATEC Web of Conferences 139, 00005 (2017)
The objective of this paper is to develop a method for automatically road surface point clouds detecting and road surface 3D modelling, and present a data assimilation algorithm for filling holes in road surface point clouds from mobile laser scanning data.The article is organized as follow: we will explain the proposed method which includes road surface extraction, holes filling in road surface point clouds and reconstruct 3D model of road surface in section 2. The experimental results are presented in section 3, and section 4 concludes the article.

METHOD
In this section, a developed method for automatic urban area road surface detection and 3D modelling is applied from the MLS point clouds.In our research, the aim is to detecting the road surface point clouds from the raw MLS data, and then modelling 3D road based on the road point clouds.The method is selected to deal with measurement noise (trees, building, car etc.), reorganizing point clouds (road, on-road and off-road) and missing data (holes in road point clouds).Our approach consisted of four processing steps: (1) The raw MLS data are partitioned along road directions as sub regions called road parts, for which the road surface point clouds extraction is applied.The laser points are roughly classified into road, on-road and offroad.We eliminate the on-road point clouds and the offroad point clouds (e.g.buildings, cars, kerbstones, and parterre).
(2) The road segments are further examined if there is any missing point clouds data (holes).Especially the holes are caused by on-road point clouds.The holes filling method is based on a data assimilation algorithm.
(3) From the extracted road point clouds, we propose an algorithm to fill the scanning holes in the road surface point clouds.(4) Finally we create a TIN and iteratively shaped the TIN into the smooth and intact road surface point clouds by filling the scanning holes.

Road surface extraction
The range scanner samples line profiles more than 40m on either side of the motion direction.Hence each profile has points pertaining to road surfaces and points pertaining to objects beyond the road such as trees, parking cars, or curbs within the scanning range.We segment the profiles to ignore points belonging to surfaces from those objects that are not of interest for the mapping.The on-road point clouds mean such objects as moving cars, people and trees which are on the road surface.The on-road point clouds differ from road surface point clouds in terms of their heights.The offroad point clouds mean outside both sides of road belong to such objects as buildings, curbs, parking cars, kerbstones, and parterre.There is a 'distance' between off-road point clouds (both sides) and road point clouds, and the off-road point clouds differ from road surface point clouds in terms of heights.The height information and distance information is then processed into segments that could be considered off-road point clouds.After that, we execute the following two-step algorithm for raw MLS point clouds to extract road point clouds: (1) Make a distinction between road points and beyond road points.If there is a distance between some point clouds and road surface point clouds, and the height level of those point clouds has changed dramatically.We can make sure that those point clouds are off-surface points which should be assigned as point clouds of no interest; (2) After step (1), we find out the top point of road surface point clouds, and the height (z) value of the top point is applied as a height threshold.We eliminate points which height value is higher than the threshold value, and the left point clouds are assigned as of interest.This procedure essentially filters out unnecessary data obtained from the laser scanners where the data are not of interest.This step is an important pre-processing step before we can execute surface reconstruction algorithms from 3D scattered point cloud data.

Algorithm for holes filling
Scan holes occur when parts of the scanned object are obscured or visually inaccessible.Well-planned scanning positions can minimize scan holes but complete avoidance is often, if not in most cases, unachievable.Urban area road environment components which are typically not visible to the scanner are detail surfaces of the decorative attachments, small cavities, ruined road surfaces, road surfaces behind trees and surfaces that are at a level higher or lower than the scanner.Incidentally, the same problem arises when acquiring urban area 3D model and photography for texturing the model.Filling scan holes, somewhat euphemistically referred to as surface augmentation fabricates information where there is none.This is highly questionable for urban road model construction and other applications where full surface detail is required.All these applications require reliable and faithful surface representations and do not tolerate fabricated data.Some hole filling occurs during the meshing process depending on the algorithm used, the choice of holes to be filled is not under the control of the software operator and therefore potentially detrimental.Holes filling can either be done automatically or manually.We present a new holes filling procedure to repairing missing part in the urban area road surface point clouds based on the MLS projection procedure.We use a statistically-based algorithm that aims at refining the positional and radiometric accuracies of missing point clouds (holes) in the road surface point clouds.This algorithm utilizes the theory of data assimilation to enhance the 3D geo-referencing accuracy as well as fine-tuning the radiometric intensity by means of exploiting the correlation between two oppositelycollected datasets over the same study area.Data assimilation is an optimization algorithm that is based on the principle of least squares analysis.The purpose of data assimilation is to combine two different datasets or models of the same phenomenon in order to achieve the best estimate of the true state.This method was proved to be viable by Reichle who used it to integrate satellite remotely sensed snow data with ground field hydrology measurements [16].Having found this method to be successful for the case of snow data which were collected by two different systems over different scales, the urban area MLS case will, presumably, has much higher probability of success as it was collected by the same laser sensor over the same area and scale under the same ambient conditions.Even more, the two MLS point clouds were collected over relatively same period of time unlike the case of snow data.Apparently, therefore, data assimilation will have better chances of recognizing the underlying correlation between the two MLS point clouds.Those factors, altogether, are the rationale behind choosing data assimilation to enhance the accuracy of road surface 3D modelling.Suppose that we have two different point clouds Vi and Vj of a certain phenomenon.Let σi 2 and σj 2 be their respective variances.Now if V is the true state, then our goal to minimize the weighted sum of squared residuals: where J is known as the cost function and wi, wj are the weights The best estimate V' can be obtained by differentiating equation ( 1) and equating the derivative to zero, which yields: It might be of interest to demonstrate the robustness of this model by computing the variance (C) of V': which implies that the new model is closer to the true state than either point cloud.In the event of having multivariable dataset, equation ( 2) can be generalized as follows (S is symbol of covariance):

V'=inv(S(Vi)+S(Vj))*(S(Vj)*Vi+S(Vi)*Vj) (4)
As mentioned previously, the assimilation process uses the existing correlation between different observations of a particular phenomenon in order to obtain the best estimate of the true state.The assimilation method is meant to integrate the attributes of two point clouds that are corresponding to the same geospatial positions where the two point clouds occupy the same 3D coordinates.However, in the case of MLS data, the geospatial positions themselves, in addition to the intensity, need to be assimilated.Since the 3D coordinates of the two point clouds do not coincide (they will coincide only if they are both true state), a new definition should be introduced to decide on which points are to be assimilated.
The MLS point cloud is irregularly spaced by nature, which will be inherited when the two point clouds are blended.The criterion of which point is to be assimilated should be based on the spatial neighborhood.When the two point clouds are merged, points from the first point cloud (point cloud M) might come closer to the points of the second point cloud (point cloud N), and then only the points that are sufficiently close will be assimilated.The status of sufficiently close will be declared for the points that are closer than the minimum spacing of each point cloud individually.If point A and point A' are two points from the forward and reversed data cloud then they will be assimilated if: AA' < minimum distance of point cloud M AND minimum distance of point cloud N The minimum distance of point cloud M (N) means the distance between one scan point to another scan point in point cloud M (N).
When that status occurs, the laser sensor is, most likely, targeting the same object (labeled as white), yet missing the exact position because of error measurements.The new point that results from assimilating A and A' will be higher accurate and closer to the true state as indicated by equation ( 3).However, there is a subtle downside for this approach as it suggests replacing two points with only one point which reduces the resolution of the MLS point cloud data when it is least expected to improve it.This situation can be avoided by changing the AND in the criterion to OR: AA' < minimum distance of point cloud M OR minimum distance of point cloud N.The following cases will be encountered during processing when A and A' are sufficiently close: Case (I): The distance between A and A' is less than the minimum distance of M point cloud and lager than that of the N point cloud.In this case A will be replaced with the assimilated point while A' is retained.No point is lost.Case (II): This is just the opposite of Case (I).Again, no point is lost.Case (III): The distance between A and A' is less than the minimum distance of both the M point cloud and the N point cloud.In this case A' will be replaced with the assimilated point.To avoid reducing the resolution we can only change the height level of point A'.
Having done that, the total number of points will be exactly equal to the sum of the individual point cloud.Provided that the average distance of the MLS point clouds of this research is 5 cm and the minimum distance is about 1 cm, the positioning accuracy is expected to be within cm level.

Road surfaces reconstruction
A road surface model will be created by extracting all the road surface points in the point cloud.Then we create a triangulated irregular network (TIN) and iteratively shaped the TIN into a smooth surface by filling holes.The TIN model represents a surface as a set of contiguous, non-overlapping triangles.With each triangle the surface is represented by a plane.The TIN data structure is based on two basic elements: points with x, y and z values and a series of edges joining these points to form triangles.In this paper, the triangles are made from point clouds.TIN's triangulation method satisfies the Delaunay criterion.Delaunay triangulation is a proximal method that satisfies the requirement that a circle drawn through the three vertices of a triangle will contain no other point.In preparation for triangulating the road surface, we find the outer bounds of the road based on the previously extracted road surface.A Delaunay triangulation is formed from the edge of road to the other side.From the initial road points, a Delaunay triangulation is created to represent the surface.For representing surface, the hole areas cannot create Delaunay triangulation should be found, and the hole should be filled based on the previously proposed algorithm.After filling all the holes failing to meet the requirement, we form a new TIN.This is done repeatedly for as long as the presenting road surface without holes.

Mobile Laser Scanner Datasets
The point cloud was collected with the FGI Roamer mobile laser scanning system developed at the Finnish Geodetic Institute.The Roamer consists of a Faro LS 880 laser scanner with a measurement frequency of 120 kHz and a NovAtel HG 1700 SPAN58 INS system.With slightly modified hardware for the standard FARO LS, it provides so-called tunnel mode, or profile measurements, synchronized with external positioning and data logging systems.This information is needed to derive the position and attitude information for each 3D point produced by the laser scanner.The mirror rotation frequency, or scan rate of the scanner on the Roamer can be set to 3-30 Hz, thus giving a vertical angular resolution of 0.0096-0.096degrees (0.17-1.7 mrad), respectively.Corresponding point spacing at a typical scanning range of 15 meters in road mapping is thus 2.5-25mm in the scanning plane.The dataset consists of mobile laser scanning point clouds over a 1700 m closed loop at Espoonlahti neighbourhood in Espoo, about 15 km west of Helsinki.The data extent covers five road segments of Espoonlahdenkatu, Espoonlahdentie, Merenkulkijankatu, Kipparinkatu and Espoonlahdenranta streets in addition to Lippulaiva shopping mall (Fig. 1).
The number of points in the raw MLS data is 22,559,451.

CONCLUSIONS
Accurate road surface information is needed in numerous applications, such as road maintenance, 3D city and noise modelling, location-based and driver assistance systems.MLS supplements conventional laser-based mapping techniques; it produces more accurate and denser data than ALS and is much more efficient than TLS when large areas are mapped.In this paper, a method for filling holes of road surface point clouds and generating model of road surface from MLS data are proposed.This is vital since such MLS system allows fast processing of long corridors, e.g.thousands of kilometers of road.Such information will be needed in future 3D road environment mapping, e.g. in automatic updating of the 2D and 3D navigation data for consumer and business applications.Visual experiments confirmed that the methods developed hole filling in road surface point clouds.Future studies will demonstrate the correctness of such methods with larger experimental sample.The methods will be further developed to be applicable for road crossings, road marking, building and other kinds of urban road environment.Also median islands and pedestrian refugees cause problems to the current methods.

Fig. 1 . 3 . 1
Fig. 1.Bird eye's view map of the raw MLS data extent covers five urban area road point clouds.

Fig. 2 .Fig. 3 .
Fig. 2. (a) The model is badly reconstructed based on the point cloud with a missing part.(b) The good model reconstructed after holes filling.The road surface point clouds can be extracted after the on-road point clouds and off-road point clouds are eliminated.We check the road surface point clouds if there are holes areas or not.We fill the hole areas using data assimilation algorithm in order to obtain the road point cloud without incomplete part.The hole areas is a huge problem for creating TIN.After making sure there are no hole areas in the road surface point clouds, we create the complete road surface TIN.The final road surface 3D model bird's eye view map is shown in Fig.3from the raw MLS data Fig.1.