An Approach for Integrating Mixed Reality into BIM for Early Stage Design Coordination

Building Information Modelling (BIM) and its associated technologies have proved to be one of the most promising developments in the Architectural, Engineering and Construction (AEC) industry. Over the past few decades, the AEC sector has been restricted in its communication of design as a result of single interface methods based on 2D and 3D visualization of information. Thus, most issues with respect to construction are identified fairly late, resulting in costly changes. With the introduction of BIM, many other approaches to data visualization can be leveraged including Mixed Reality (MR) applications for the virtual representation of spaces and objects beyond 3D. MR offers a revolution in the virtual representation of objects and space through context awareness as well as the incorporation of information beyond 3D offering countless opportunities for more effective design visualization and coordination. Despite the capability of MR, however, few examples exist of its application to design coordination in the AEC. In addressing this gap this study proposes a novel methodology for the application of MR in design coordination as well as investigates the impact of introducing MR into BIM workflow with a focus on the identification and avoidance of clashes. A prototypical model of the MR design coordination is presented and discussed. Findings indicate that MR improves design productivity and quality but also highlights potential infrastructure issues inhibiting the mainstreaming of MR for design practice.


Introduction
Construction industries productivity index has always struggled to catch up with other industries. There is, therefore, an attendant need for the improvement of construction. According to Teicholz [1] and Gerbert et al. [2], construction industry productivity has stagnated for decades across the globe due to among other reasons, poor innovation and adaptation. The report "Reinventing Construction: A Route to Higher Productivity", released in February 2017 reiterates the argument by suggesting that the productivity issues in the construction industry are due to lack of research development and innovation [3]. Kymmell [4] argues that an increase in the complexity of the construction as the main reason for this productivity lag. With such increases in complexity, the design has also become increasingly complex with multidisciplinary input that requires coordination. According to Maing and Vargas [5], despite the increasing complexity of design, productivity gains can be made through the adoption and application of digital innovations such as Building Information Modelling (BIM).
BIM has proved to be one of the most promising developments in the AEC industry [6]. NBS [7] defines BIM as the "process for creating and managing information on a construction project across the project life cycle where the key output of this process is the digital description of every aspect of the built asset". Bernstein et al. [8] have reiterated that the diffusion of BIM in the AEC industry has substantially increased in recent years, especially among contractors who are increasingly becoming responsible for coordinating design due to evolving procurement. BIM has the capability to leverage intelligent 3D models to design, construction, delivery, operations and facility management [9].
Indeed, having an accurate 3D modelling solution, upheld by well-defined data requirements are capable of delivering benefits at every stage of the design and construction process. The transition from a 2D documentation to 3D BIM modelling was a natural evolution which aided in improving team communication and coordination. Viewing the 3D BIM model through a 2D screen is very common, however, interacting with volumes of data behind the 2D is extremely beneficial [10]. Mixed Reality (MR) offers a revolution in the virtual representation of objects and space through context awareness as well as the incorporation of information beyond 3D offering countless opportunities for more effective design visualization and coordination. Despite the capability of MR, however, few examples exist of its application to design coordination in the AEC. In addressing this gap this study explores the following research questions: (1) How can MR be leveraged to support design coordination? (2) How effective is MR use in design coordination? The research thereof proposes a novel methodology for the application of MR in design coordination as well as investigates the impact of introducing MR into BIM workflow with a focus on the identification and avoidance of clashes. A prototypical model of the MR design coordination is presented and discussed.

Information Delivery and Design Coordination
The linear flow of information exists in the design communication between the designers, specialist consultants and on the other hand constructors. Shannon [11] explains this mode of communication through the theory of linear standard communication where a message is generated, encoded into a single transmittable in a desired medium and further decoded upon arrival at the receiver's end Barnes [12] points out that traditional means of communication in the construction was solely based on 2D paper plans which potentially created a risk of noises ineffective transmission of the information. In construction, the design is embraced as a collaborative activity by necessity due to the division of labour and expertise between organizations [13]. A consequence of the structure of the sector is that design activity involves the coordination of complex information exchanges in multi-disciplinary design teams. Coordination and communication challenges underlie difficulties in the integration of work activities of design teams. Communication is central to design collaboration and the coordination of design inputs [13].
Decision made during the design coordination could potentially affect many stakeholders including architects, engineering consultants, construction managers and facility maintenance organization. Coordination among stakeholders is critical to ensure that facility design meets the intended functionality, aesthetic and economic requirements of the owner [14]. Riley and Horman [15] point out that efficient Design coordination is key to reduce uncertainty in the production process in the construction project by decreasing disruption and reducing waste in the construction process. The relationship found on a case study project between the effort spent on design coordination and the resulting elimination in field conflicts revealed that projects A and B which carried out design coordination with 100% diligence and treated as a prerequisite for construction has zero or negligible field generated change orders whereas projects C and D have a considerable number of change orders [15].
The early stage of a design process is of particular importance for the quality of the final results as most of the building lifecycle characteristics and costs are already committed at this stage and the opportunity to influence the final design decreases rapidly as the cost of making the changes or correcting the design errors increases dramatically [16].
Mehrbod et al. [17] identified in their study that despite several advantages of BIM in design coordination, utilization of the 3D BIM models on a 2D screen during design coordination were ineffective as the navigation with the digital artefacts were often inefficient. The major challenge Mehrbod et al. [17] identified in information delivery 3D BIM models during design coordination was navigation in BIM environment on 2D screens which forced the participants to often revert to the 2D paper-based drawings for better understanding. CAD and virtual visualization could be regarded as complementary technologies in design visualization where such Interactive visualization tools are capable of playing a critical role in the effective information delivery during the design coordination phase of a project [14].
Woksepp [16] argues that a 3D system has an inherent limitation when it comes to conveying an understanding of the complex virtual environment. 3D CAD system is developed for a design specialist to create precise three-dimensional representations of the real object, while virtual immersive technology allows the user to display and interact with objects in the virtual world thereby enhancing the design coordination [16]. Recently, studies executed to identify the impact of 3D physical and virtual mock-ups in supporting the design coordination which revealed that physical mock-ups potentially can offer values for design coordination but they can be impractical to communicate design at every stage of the project and physical mock-up could incur cost, time and physical space requirements to set up [18]. On the other hand, virtual mockups are capable of delivering in-depth high-level detail of any system and their interfaces without the high-cost burden [19].

Immersive Technology Applications for Visualisation
Immersive technology is an integration of virtual content with the physical environment in such a way that enables the user to interact naturally with the amalgamated reality [20]. Virtual reality, augmented reality and mixed reality is the most commonly available immersive technology. Virtual reality is the utilization of computer technology to create a simulated environment where the user is completely immersed in the virtual world by simulating as many senses as possible [21]. Thus, the user becomes a part of the virtual or immersed world within its environment and whilst they are able to manipulate or interact with the object. On the other hand, augmented reality is capable of superimposing spatially contextual information over the user's view of the real world providing additional data while still permitting interaction with the real environment [22]. The inherent limitation of augmented reality as per Foundary [23] is that while augmented reality is capable of overlaying virtual object in a physical environment, the virtual content cannot be anchored to the real world. Thus the physical environment and the overlayed virtual object fail to interact which limits its application in the AEC sector especially where coordination relating to different trades are impossible. Mixed reality is a hybrid of augmented reality which is capable of overlaying synthetic content on the real world which can be anchored to and interacts with the physical world [23].

The Role of Mixed Reality
Mixed reality is capable of bridging the gap between virtual and physical world thus eliminating the inefficiency in the current workflow. As discussed earlier, interacting with the information-rich 3D model behind 2D screens are extremely limited where mixed reality tools are capable of bringing those 3D models out of the screen even on to real construction site, allowing the user to engage and interact with the design data more intuitively [10]. Mixed reality tools can potentially play a great role in the design coordination by allowing the stakeholders to visualise and explore the design in virtual 3D without the need for an expert to guide them and dictate the point of view [10]. Feiner et al. [24] investigated the possibility of using 3D Head-mounted displays (HMDs) in conjunction with mobile computing technology, creating a digital prototype which overlaid campus information on top of an unobstructed view of a university campus which was a first of its kind innovation in the construction industry. During the design phase, MR was used for information delivery by presenting relevant data points to the user without interpreting the normal workflow [25].
MR's potential as an onsite visualisation tool has also been well tested and studied where 3D building models were visualised on physical location [26,27]. Further exploration of possibilities of using MR has been carried to augment BIM content enabling on-site viewing [28] documentation and monitoring of construction process [29] and to detect onsite miscoordination issues [30]. The possibilities of deploying MR for training purposes in construction are also well studied. Wang, Dunston and Skiniewski MR training system is a great example for the utilisation of MR for training construction workforce where they utilized it for training heavy construction equipment operation including cranes [31].
While the use of MR for onsite model visualisation and construction workforces training are well documented any efforts to utilize the enormous possibilities of MR in early design coordination has not been studied specifically. This paper examines the feasibility of integrating MR into early design coordination workflow for the purpose of better design communication between various stakeholders, thus reducing reworks, cost overruns, material wastage, thereby promoting lean construction.

Mixed Reality Application for Design Coordination
Mixed reality uses head-mounted devices to project virtual 3D objects into the user occupied physical environment [32]. For the purpose of this study, Microsoft HoloLens is used as the hardware to identify the possibilities of using Mixed reality in for design coordination. Microsoft HoloLens is a wearable, self-contained a holographic computer with advanced sensors that are capable of mapping the physical environment which allows the device to detect physical objects which is very important for coordination in any industry [10]. Emerging devices such as the Microsoft HoloLens is capable of encouraging two key approaches when it comes to collaborative design planning. Firstly, it allows multiple users to examine and coordinate same projected 3D design simultaneously from remote locations [32]. Secondly, the accurate convenience of the visibility of the physical space is an important factor that makes HoloLens projected mixed reality far better than virtual reality for collaboration and coordination of designs [32]. Various software such as Visualive3D, Trimble Connect, BIM Holoview, enables the interoperability between BIM modelling tool and the Hololens. For this study, Trimble Connect was used as the software platform for integrating a digital model of design (Autodesk Revit) into the MR environment (HoloLens). This work uses a mechanical plant room as a prototypical model to develop an understanding of the performance impacts observed using MR for design coordination. Plant room as chosen to implement the proposed system given its complexity in the design coordination process thus often leading to clashes and redesign in typical project scenarios. A qualitative approach was used for evaluating the effectiveness of the proposed system with six participants given the opportunity to visualize the model assembly using two different visualization approaches viz MR and traditional paper-based method. Out of the six participants, three had more than eight years of MEP design experience; two were facility managers with 11 years experience, responsible for mega facilities and one of the participants was a client representative who is responsible for ensuring client requirements are met as required. The following section presents a detailed discussion on the procedure adopted in developing, pushing the model for MR experience and evaluation of the system based on the participant's experience.

Description of Approach and System Architecture
To develop an approach for integrating MR into the early design phase, the 3D model developed using Autodesk Revit needed to be imported into an MR environment. Even though a number of devices could theoretically be used for this purpose, the researcher chooses to use a Microsoft Hololens, which is head-mounted display (HMD) device with seethrough a screen which is capable of augmenting 3D virtual object on a physical environment. The MR capabilities of a Microsoft HoloLens include the ability to augment virtual objects by relying on spatial mapping to identify the physical surface which enables the consistent visualization of the model. The chosen HMD has an inbuilt Windows operating system which provides the virtue of hands-free operation without plugging into an external computer, which further enabled the participants to manoeuvre freely in space without jeopardizing the user's safety. Below, Fig. 1 demonstrates the process flow from model preparation to visualization using various software and hardware.

Development of MR Visualization Environment
To develop the chosen MR environment, the prototypical plant room model is developed using parametric software (Autodesk Revit). Due to the interoperability issues between the Revit native file format (.rvt) and the MR environment cloud-based software, Trimble Connect (requires .skp file format) is used to enhance data transfer of the plant room model developed using Revit and subsequently imported to Trimble Sketchup. Texture property loss during importing is restored using Trimble Sketchup. Further, the model is then uploaded to the Trimble Connect cloud-based platform to generate an MR environment on Microsoft HoloLens. Once the model is successfully imported into Trimble Connect, the model is opened by executing Trimble Connect on Microsoft HoloLens as shown in Fig. 2. The true scale (1:1) MR model is anchored to a physical space for the participants to visualize and walk freely to explore the model further in detail.

Evaluation of System
All the participants were initially given the opportunity to visualize the paper-based drawing Fig. 4, followed by model visualization through the MR environment. Participants were provided with an opportunity to walk through the model to visualize every placement and installation details of the MEP equipment. Fig. 3 illustrates the full scale (1:1) first-person view of the plant room and equipment viewed through the Microsoft Hololens. Participants explored different options such as scaling down to view as a tabletop model, teleporting enabling walkthrough at smaller physical space, anchoring model at different orientation etc. through Trimble connect MR environment. All the participants were able to use the device with ease even without having previous experience using such devices.  Few of the participants reported a lack of model brightness as a limiting factor when used at well lite physical space. All the participants (100%), however strongly agreed that the use of immersive technology improved their appreciation of design information much better than traditional approaches. According to participants, the use of the proposed system in the early design coordination workflow can enhance the coordination process reducing the amount of time by up to 50-80%. This is exemplified by quote for evaluator below: "we will take required actions to bring this kind of visualisation tools for all our projects at the earliest and this will be added as one of our requirement in our EIR (i.e. Employers Information Requirement) for all the projects" [Evaluator 1 -Client Representative].

Discussion
The prototypical MR model anchored to physical space revealed that the placements and installation details of the equipment and piping were very easy to visualize and understand rather did on a paper-based drawing. The participants had no previous experience using MR devices and they could adapt and explore the model in the MR environment quickly. Walkthrough using MR model revealed all the maintenance accessibilities issues that may have arisen only during installation or mostly during the operation phase of the facility, that would have led to rework, cost overrun and time. Further visualization of the model in the MR environment enabled the designers to understand the requirements of different stakeholders involved in the project, without putting many efforts. Also, it provided an opportunity for designers to try different installation methods and pipe routing which could reduce the cost at the same time keeping the health and safety of the installation and maintenance team in mind. However, few of the participants were concerned about the capability of using the device on well lite physical space due to the inherent brightness control limitations.

Conclusion and recommendations
In this paper, the authors propose a novel methodology for the application of MR in design coordination as well as investigates the impact of introducing MR into BIM workflow with a focus on the identification and avoidance of clashes. A prototypical model of the MR design coordination was presented and discussed. Findings indicate that MR improves design productivity and quality but also highlights potential infrastructure issues inhibiting the mainstreaming of MR for design practice. It is clear that integration of MR at the early design stage and active participation of all stakeholders in design coordination with the aid of MR devices can considerably reduce reworks, cost and time overruns at the same time enhancing the safety of the personals handling the task.