Quantified point clouds and enriched BIM-Models for digitalised maintenance planning

. Digitisation in the construction industry continues to advance and, together with the increasing dissemination and further development of hardware and software, is steadily opening up further opportunities for innovative ways of working. Building Information Modelling (BIM) is currently becoming the standard for new construction but has not yet been optimised for use in existing buildings. Therefore, the Institute of Building Materials Research (ibac) is researching new methods and possibilities for BIM-based building preservation. In this paper, the automated creation and analysis of point clouds as well as the implementation of further information from in situ diagnosis and monitoring systems in BIM-Models are presented. On a practical example, the different steps of a subsequent digitisation of an existing building are demonstrated considering new possibilities as autonomous robots and the intelligent utilisation of sensors and diagnostics tools. The goal is a decision support tool, which is independent from proprietary software, adaptive to different types of buildings and open for various interfaces. Current results show that quantifying point clouds and making BIM-models usable beyond the planning and execution phase for new buildings are essential steps for the digitisation of building maintenance. The proposed digital workflow holds great potential for effective building diagnoses and efficient service life management.


Digitisation in construction
In 2015, the Federal Ministry of Transport and Digital Infrastructure (BMVI) officially proclaimed the digitisation of the german construction industry through the step-by-step plan for digital planning and construction [1].Digitisation is not only accompanied by technological progress, but also by terms such as Internet of Things (IoT), Industry 4.0, Smart Buildings and BIM (Building Information Modelling).In the first progress report of the BMVI's implementation plan, the focus is clearly on the use of BIM as an instrument for planning, progress control and information provision [2].BIM is increasingly arousing interest around the globe and even becoming mandatory in different countries such as United Kingdom and Singapore [3].However, the construction sector is still the least digitalised and digital models and twins are most prevalent in industry and asset management, significantly less in infrastructure and smart cities [4].In Germany, the public sector is playing a pioneering role in the realisation of digitisation in the construction sector.The Federal Waterways Engineering and Research Institute (BAW) and the Federal Waterways and Shipping Administration (WSV), for example, show clear ambitions and describe great potential in digitisation, for whose effective implementation, however, a corresponding collaboration infrastructure still needs to be created [5,6].

Building Information Modelling (BIM)
Building Information Modelling is a computer-aided method for the construction, planning and operation of buildings.In corresponding BIM software, all components can be graphically displayed and provided with specific information.By selecting the respective element, information about the building material, the geometry and the design can be called up.When using this building model, a distinction is made between Closed-BIM and Open-BIM processes.In the case of Closed-BIM, a specific software must be used for collaboration, which has been optimised for the respective purpose and is usually subject to licensing.In the case of Open-BIM, an open file format is chosen that allows work with different programs so that everyone involved in the construction process can have access to the model.In most cases, the IFC format (Industry Foundation Classes) is used for Open BIM processes.
IFC is the open standard in the construction industry and defined by the buildingSMART competence network.The IFC format is designed for unification and standardisation, but this is at the expense of complexity.
Each BIM element carries certain information, socalled features, which define the component or describe the specifications.However, when transferring the data into the open IFC format, information can be lost if the characteristics do not fit into the IFC pattern.For this reason, the buildingSMART Data Dictionary (bsDD) provides a kind of dictionary for the common language in BIM-supported collaboration to ensure uniform data communication.Since the bsDD does not cover all necessary cases, the use of a (national) feature server for the uniform transfer of information is proposed [7].Despite these structural challenges, the application range of BIM is continuously increasing.Besides planning of new buildings, BIM is used for fire safety [8], life cycle assessment [9] and even monitoring systems with IoTsensors [10,11] or for the maturity assessment of fresh concrete [12].Nonetheless, a current review still identifies a lack of BIM-applications in maintenance and transportation infrastructure [13].

BIM-based building maintenance
With thorough planning, BIM-models sometimes describe the target state extremely precisely.However, the actual state after construction has not yet found its way back into the corresponding model.Over the last decade, an increasing number of articles related to BIM in monitoring and maintenance have been published and the optimisation and management of (sensor) data was deduced as need for further research as well as the consideration of environmental effects [14].In [15] a BIM-based bridge maintenance system (BMS) is proposed, considering influences from chloride ingress, carbonation and other agents.Although the BMS is using BIM, it is not completely compatible with IFC and dependent on further software / data formats.To change this and avoid the technical loss of value of the BIMmodel, the Institute of Building Materials Research (ibac) at RWTH Aachen University (Germany) is researching measures for BIM-based building maintenance.This should include the following areas after the realisation phase: The first steps towards BIM-based building maintenance have been researched in the ZIM-project "DigiPark" funded by the Federal Ministry for Economic Affairs and Energy (BMWi).Based on this project, the Innovation Network (www.bim-xd.de),which is also funded by the BMWi, was founded to pursue the vision of a complete digitisation of existing buildings and their maintenance.The network combines competences in building diagnosis, maintenance planning and construction as well as in the areas of software and hardware development, so that all necessary developments can be worked out within the network.
In building maintenance, the complexity and individuality of repairs pose a particular challenge compared to new construction.There is no standard solution, just as there is no standard damage.Accordingly, the digitised methods must be particularly adaptable and applicable to the most diverse objects of investigation.An essential necessity for BIM-based maintenance is the existence of a BIM-model.Since this is usually not available for buildings in need of maintenance, it must be created subsequently.In many cases, point clouds are used for this purpose.

Digitalisation of existing structures 2.1 Point clouds for retroactive BIM-modelling
Using modern laser scanners, the creation of precise laser scans and the subsequent compilation of the individual scans into a point cloud can be implemented in a very user-friendly way.These point clouds consist of several million distance measurements and thus contain the relative spatial coordinates of the scanned areas or objects.With the appropriate hardware, colour information is added to the individual points in addition to their location, so that a precise image of reality can be created.In the point cloud, distances between the individual points can be measured so that distances, angles, areas, volumes and also curvatures can subsequently be determined on the PC.The accuracy essentially depends on the resolution of the laser scanner used, the number of scans and the distance between scanner and surface.
To engage with BIM-based building maintenance, a BIM-model is required.For most existing structures, this model is non-existent, which is why the BIM-model probably must be created retroactive.This can be done with floor plans and other analogue data or with point clouds.An example of a BIM-model which was derived from a point cloud is given in Fig. 1.This procedure is also known as "Scan to BIM".Whereas most geometric information is quite accurate and complete, each column and obstacle (e. g. cars) will cause shadows in the point cloud, if it is not compensated through another scan with a different point of view.

Fig. 1. Point cloud (top) and derived BIM-model (bottom) of a parking structure
To achieve point clouds of a proper quality and completeness for retroactive BIM-modelling and further operations, such as measurement of damaged areas (e. g. concrete spalling), there are several influences to be considered.For the point cloud to be used effectively for both the detection of damage and subsequent BIM modelling, the following questions must be answered before the point cloud is created: • What resolution is needed to capture damages with sufficient precision?• What degree of shading is acceptable?
• What performance characteristics must the laser scanner fulfil or what settings are necessary?• How many scans are to be carried out?
• Where should the scanner be positioned?
The answers to these questions are building-, projectand hardware-specific, so that they cannot be answered universally.If the laser scan is done by professionals, they can act according to their experiences and ensure a satisfying result.But an individual assessment of the building structure and point cloud generation by an expert would lead to additional costs and further increase the obstacles to subsequent digitisation.In the meantime, laser scanning has become so user-friendly that it can also be carried out by untrained personnel and even autonomously acting robots, provided that appropriate planning is available.Therefore, a procedure was developed at ibac that automates and quantifies the planning of a point cloud creation.

Planning of quantified point clouds
During the DigiPark project mentioned above, a Python script was written and expanded in the context of a master's thesis and a cooperation with Leica Geosystems (Switzerland), which enables automated scan planning on the basis of 2D floor plans transferred into bitmaps, see Fig. 2, and the specification of various boundary conditions or requirements (hardware specifications of the laser scanner used, desired point density, ...).

Fig. 2. 2D floor plan of a parking structure shown as a bitmap
An input parameter of the script is, for example, the minimum overlap so that at least a certain number of already registered points are seen from each position.In this way, the scans can be positioned in such a way that the registration (assembly) of the individual scans into a point cloud functions automatically without manual work steps.This procedure is particularly relevant for point cloud generation with autonomous robots, which are to operate without special alignment / levelling of the scanner or corrections by human operation.Different combinations of parameters were tested to elicit proper settings.Afterwards, most of the plannings were carried out of untrained humans but the procedure was also tested and validated in cooperation with STRABAG AG using a robot, see Fig. 3, which was fed an automatically generated path planning and carried it out autonomously.According to the input parameters, the ibac script checks whether the required resolution has been met and gives quantile values for the minimum resolution (e. g. "90 % of the scanned area has a resolution of ≥10 points per cm²").A selection of input and output parameters is given in Table 1.If a BIM-model already exists or if there are further paths to be planned for following tasks that should also be carried out by robots, the path planning can be done using the BIM-model [16][17][18].The aim of this research is to determine the parameter settings necessary for different requirements and then to be able to create point clouds according to needs via automated planning (and autonomous robots).Through the quantifying analysis of scan planning, it would also be conceivable to define quality levels for point clouds and to standardise them for certain uses.High-quality point clouds could open new possibilities for subsequent digitalisation of existing buildings.

Functionalisation of point clouds
One of the new possibilities, achieved through highquality point clouds, could be the automated measurements of damages.Fig. 5 shows a damaged area due to corrosion-induced concrete spalling in a parking lot.This damage was scanned, and the resulting point cloud is shown in Fig. 6.There are several ways to measure the geometry of the damage in the point cloud, most of them including manual handling.As buildings in need of repair often show many damages of different sizes and locations, the measurement should be automated for an efficient workflow.To automate software-based processes, Application Programming Interfaces (API) or Wrappers can be very useful but are not available for every software.The following examples were carried out using CloudComPy, the Python-Wrapper for the potent open-source point cloud software CloudCompare.
Before a damage can be measured, it first must be located and identified.This can be a very difficult task as not every surface deviation has to be a damage.The more non-flat elements the point cloud contains, the more complex this process gets.In the following example, special objects like cars and air ducts were erased from the point cloud before further analysis.The demonstrated steps were all automated via CloudComPy and the analysis is solely done by executing a Pythonscript.For further information about geometric features as surface variation refer to [19].

Fig. 6. Damaged area in point cloud
Firstly, the surface variation is calculated for the entire point cloud, as seen in Fig. 7.The red border indicates the transition zone from flat to lumpy (according to a given tolerance value as input parameter).This can be understood as contour of the damage.The green areas indicate a relatively low inclination, whereas the red areas show spots with a high surface variation.

Fig. 7. Point-cloud-based analysis of surface variation
Secondly, the damage has to be measured.For the overall area, a plane is fitted through the borderline (red line in Fig. 7) and cropped according to it.For the volume, the depths are calculated as distances between the measurement points and the plane in direction of its normal vector.Thus, different values as mean, maximum and standard deviation can be easily calculated for each damage.The damage coloured according to its depths is shown in Fig. 8. addition, the minimal bounding box is calculated, all calculated values are stored in a csv-file and the cropped damage is stored as polygon-mesh.

Fig. 8. Visualising depths of damages in point cloud
Finally, the obtained values can be implemented in BIM.If the BIM-model origins from the same point cloud as the damage analysis, probably no coordinate transformation is needed.Otherwise, it suffices to select the same three surfaces in the BIM-model and the point cloud to transform the coordinates.In any case, the damage can be implemented as element in the BIMmodel, containing all relevant information such as maximal depth and volume, see Fig. 9.If two point clouds are given, one before and one after the damage occurs, the same results can be obtained even easier via the M3C2-Plugin for CloudCompare (featured in [20]).This will probably gain relevance when autonomous robots are used for recurring laser scanning.Further research to detection of geometric primitives in point clouds can be accessed in [21].

BIM as built
An as-built model is a model that corresponds to the actual built condition.For buildings that have not yet been digitised, every subsequent model corresponds to an as-built model, see for example Fig. 1.However, "as built" is usually pertaining to geometric attributes and not to material properties.The created model may look mostly like in reality (usually surfaces are idealised), but it does not yet contain any information relevant to building preservation, it only serves as a 3D plan base for the following steps.The addition of information is only provided within a certain framework, which is currently not suitable for building preservation [22].Accordingly, all results of the building inspection must be added to each element (component) individually.When mapping dozens of inspection points, hundreds of cracks or thousands of values of an area scan grid, a manual implementation would be neither economical nor practical.Thus, different tools for implementation of data in BIM are being developed at ibac.The objectives of the structural investigation are the evaluation of the component and the estimation of the expected remaining service life or the measures to be carried out.In the structural assessment of existing structures, non-destructive testing and diagnostic information are already used as the basis for fully probabilistic models [23].For structured and clear information management, the concept of model-based inspection in BIM can be utilised [24].Information on the concrete used or even test results on reference specimens may be available.However, comparative studies have shown that in situ compressive strengths are on average 20 % lower and carbonation and chloride migration coefficients are 40 to 50 % higher than those of separately produced reference specimens [25].As a consequence, a comprehensive diagnosis should be carried out for a reliable assessment of the component condition.In addition to invasive methods to determine the compressive strength, there are also non-destructive tests to check, for example, concrete cover or corrosion activity.Ideally, the building diagnosis provides information on the following component properties:

spots and areas
For effective BIM-based condition detection and assessment, the various diagnostic results must be transferred to the model in a fully machine-readable and spatially resolved manner.

BIM-implemented building diagnosis data
At ibac, different scripts for Dynamo and Python are developed to be executed in Revit (Autodesk) to implement building diagnosis data in BIM-Models.Besides simple elements such as drill cores, containing information about diameter, depth, volume, carbonation depth, compressive and tensile strength, more complex elements can be implemented (semi-)automatically as well.In addition to the implementation of individual measuring points, further elements can also be derived from them, e. g. the orientation of rebars.Fig. 10 shows the derivated reinforcement and the depth profile of chloride content in a floor slab.
The corrosion activity can also be displayed and visualised in the BIM model as the result of a twodimensional potential field analysis, see Fig. 11.In the examples shown, the free BIM viewer BIMvision (Datacomp) was used.The "Advanced Reports" plug-in can be used to colour the components according to their attribute values.Based on the BIM-implemented diagnosis data, the components can now be efficiently evaluated with regard to their need for repair.The diagnostic results can be variably displayed and hidden.Individual properties or combinations of properties can be considered, allowing contextualisation of the different information.For example, Fig. 11 shows cavities (grey rhombuses) in context with the corrosion activity and each cavity correlates with a local peak of the potential.During implementation, care was taken to ensure that the information is retained when exporting to the IFC data format and is fully machine readable.This is essential to not only enrich the BIM-models with further information, but to enrich them with further functionality.

BIM-embedded decision support tools
The IFC-format may not be omnipotent, but it is open to interfaces, allowing the inclusion in individual application programming interfaces (API) and the manipulation of the file / model.Thus, the BIM-model can be capable to serve as Single Source of Truth (SSoT) and be enhanced by individual functions.For example, if 6 MATEC Web of Conferences 364, 05001 (2022) https://doi.org/10.1051/matecconf/202236405001ICCRRR 2022 a monitoring system is implemented in the BIM-model, the corresponding element in the IFC-file can be updated continuously through an application, that pulls data from a database and overwrites the pertaining numbers in the IFC-file.Thus, the functionality of BIM can be enhanced at will.At ibac, BIM is fused with tools for statistical analysis and Bayesian inference to calibrate prediction models with implemented diagnosis data and make prognoses about the durability and corrosion progress.
Acting as SSoT, the model not only knows what diagnosis data is available, but also when and where it was measured and what kind of building component it is related to.Thus, timely and locally discretised analysis can be done automatically.For example, the concrete cover can be calculated for all columns together, for each column separately or for each side of each column -at will as mean, median or 5-%-quantile.
Furthermore, the model can, for example, check for chloride ingress data, calibrate the ingress model accordingly and calculate the chloride content in 10 mm distance to the rebar surface of the front Using this information, the model can assess different and methods for repair and prevention of concrete structures according to the applicable regulations -for example the ISO 16311-3 [26], the EN 1504-9 [27] or the technical rule "maintenance of concrete buildings" by the German Institute for Construction Technology [28].The latter being best suited, as it provides explicit constraints for the usage of most methods for reinforcement corrosion.
In the midterm, the whole workflow should be realised using only open-source software like BlenderBIM and tools like IfcOpenShell.In the meantime, ibac is adopting a two-pronged approach, using proprietary software as well, to leave no potential untapped.More research about BIM-embedded decision support tools and open-source implementation is to be published separately and would exceed this paper.

In-situ-Application
BIM-implemented diagnosis data can not only serve as SSoT, but also as enrichment of on-site visits.Using common handheld devices and state-of-the-art technology for augmented reality (AR), enriched BIMmodels can be applied in situ as seen in Fig. 12.For execution, the IFC-file was uploaded to a cloud and imported in the AR-software VisualLife.This proprietary software enables different settings to display, hide or blend different layers allowing new possibilities for on-site investigation as shown in Fig. 13 and Fig. 14.
Whereas the visualisation is only as good as its localisation in situ -model and reality must be scaled and aligned properly -the possibilities for augmentation are only bound to imagination and obtained data.The shown examples can just be understood as the beginning.It is conceivable that elements are colour-coded depending on the amount of concrete cover to be removed or restored or show appropriate rehabilitation methods such as coatings or the adding of concrete, visualising the remaining clear height.VisualLife, for example, further allows commenting elements and keeping track of the change management.The site foreman can see what has to be done where and update the status afterwards, enabling feedback to headquarters via cloud service.This feedback may contain texts, audio recordings, photographs, timestamps or virtually any information, depending on hard-and software.The presented concept for the digitisation of building maintenance builds on the technical advances of recent years and introduces a model-centred way of working.Data is to be collected, processed, and networked in a structured manner.Point clouds can be automatically planned, their quality quantified and optimally evaluated with regard to building maintenance.BIM-based evaluations suggest possible courses of action and support evaluations.With a reasonable amount of effort, a BIM model grows over the service life that reliably reflects not only the target condition, but also the actual condition.Implementation of diagnostic information in BIM allows sustainable data management, as all available information is clearly bundled and completely preserved machine-readable in a SSoT.The following conclusions can be drawn from the current state of work: • BIM-visualised diagnoses enable efficient analyses, locally and timely discretised.• Examination results remain machine-readable and can be used for further analyses.• Data analysis and recommendations for action via BIM models lead to condition assessments with temporal and local resolution.• Digitised building diagnoses enable efficient hightech planning and execution of repair measures.• Automation and functionalisation increase the profitability of retrospective digitisation.
Currently, ibac is researching the expansion of the methods presented and the combination of recommendations for action with the real-time analysis of measurement data from monitoring systems as well as more precise recommendations for action.In the outlook, autonomous robots and drones shall be used for the generation of inspection data.Furthermore, new possibilities for the BIM-based execution of repairs shall be researched.As an example, the AR-supported concrete removal or even the fully automated control of an excavator can be considered.

•
condition survey • maintenance planning • repair execution • sensor-based monitoring • durability prognosis • recommendations for action

Fig. 3 .
Fig. 3. Autonomous robot scanning a construction site according to an automatic path planning (© STRABAG AG) Fig. 4 shows a result of automated scan planning in the Cyclone REGISTER 360 software (Leica Geosystems).Path planning and registration took place without any human intervention.The result is a stable, low-shadow point cloud with quantified resolution.According to the input parameters, the ibac script checks whether the required resolution has been met and gives quantile values for the minimum resolution (e. g. "90 % of the scanned area has a resolution of ≥10 points per cm²").A selection of input and output parameters is given in Table1.If a BIM-model already exists or if there are further paths to be planned for following tasks that should also be carried out by robots, the path planning can be done using the BIM-model[16][17][18].

Fig. 4 .
Fig. 4. Resulting point cloud (top-view) from an automated path planning

Fig. 9 .
Fig. 9. Implementation of a damage as element in BIM

Table 1 .
Several input and output parameters from the automated path planning process