Exploring the Potential Utility of Unmanned Aerial Vehicles for Practical Bridge Inspection in Louisiana

Unmanned Aerial Vehicles (UAVs), especially those allowing programmed flight paths, with stops and hovering, e.g. the quad and hex rotor drones, have gathered attention in recently for as their potential use in certain envisioned applications that may be difficult, expensive, or dangerous to do manually. One poignant use case important to the State of Louisiana is that of bridge inspection. In the Lafayette District of LADOTD alone, more than 1600 bridges require annual inspection. In this paper, considerations and criteria important UAV-based bridge inspection in Louisiana are researched and analyzed; experiments with actual UAVs are conducted, and findings provide insight into the problem and potential solutions. The conclusion recommends further research items.


Introduction
Unmanned Aerial Vehicles (UAVs), i.e. capable of programmed flight paths and to stop and hover while carrying high-resolution cameras, e.g. quad and hexrotor aircraft, a.k.a. drones, are a relatively new technology sparking interest and investigation into this technology's possible applications. Potentially practical use cases abound. These range from inspecting the power grid or highways, and gathering traffic flow data for analysis, to inspection of high-rise towers, along with agricultural and hydrological surveys of farms and watersheds, to name a few. But one-use case gaining attention recently in state governments is the UAV's potential for bridge inspection.
America's bridges are aging. Inspectors must know which can be repaired if degradation is caught in time, and which must be replaced for public safety. On April 12 of this year, Mississippi's governor, Phil Bryant, declared a state of emergency, ordering Mississippi's Department of Transportation (MDOT) to immediately close 83 city and county bridges in response to a letter he received from the Federal Highway Administration (FHWA) recommending as many as 378 bridges be closed [1]. With competition for funding, states must allocate monies for bridge maintenance and replacement surgically, to get the best long-term value for taxpayers; but this requires information about bridge condition, i.e. only coming from repeated and detailed bridge inspections. Federal requirements drive states to inspect and report on all bridges annually. Currently, inspections are a manual process, coming with certain safety risks, and are costly in human and equipment resources. Some see using UAVs to inspect bridges as a potential solution. Traditionally inspecting bridges, especially those with large superstructures, requires getting equipment on the bridge deck, i.e. machines with long telescoping booms, called "reach-alls," to extend beneath the bridge deck and to the top of the superstructure. With the larger bridges, the boom may not extend high enough, meaning the inspector may not have easy or safe access to all parts of the bridge deck or superstructure for inspection [2]. So, some bridge sections may be hidden or missing from the data. Safety issues accompany traditional methods of "hands on" bridge inspection, e.g. working in the presence of vehicular traffic [2] and dangerous weather conditions, like lightning. Moreover, once problems are found, e.g. section loss in some member due to corrosion, it would be advantageous to revisit the "problem bridge" more frequently to monitor the problem's progression, allowing surgical funding and corrective action to be applied where problems are progressing fastest. However, with manual bridge inspection, it is difficult, i.e. with all the equipment setup time, to revisit bridges for example, quarterly, instead of annually. A nimbler approach using UAVs is seen by some as a potential solution to circumvent these logistical challenges, allowing more efficient and effective inspections of a wide variety of bridge structure types in challenging locations [3].
However, UAVs must prove themselves to the bridge inspection community, where many are not convinced that UAVs can assist in any meaningful way. A survey by the American Association of State Highway & Transportation Officials showed that 35 state departments of transportation are using UAVs, but only 10 are using them for bridge inspection [1]. Road & Bridges also found that Missouri Department of Transportation (MoDOT) isn't planning on owning UAVs in the coming years, but instead will call vendors who specialize in UAV technology. They consider many of their bridges to be fracture critical, requiring hands-on inspection. PennDOT owns a drone but is not using it for bridge inspection. They consider them best in emergencies to get a first view. "Drones are good for a first glance, but to do an in-depth inspection and to confirm the safety you want to have hands on as much as possible," said Kris Langer, assistant bridge engineer for PennDOT to Roads & Bridges [1]. This paper is based on a project funded by the TranSET Consortium to study the utility of UAVs for bridge inspection in Louisiana. Goals were to consider a UAV-based approach and to determine the advantages, disadvantages, and constraints, and to provide recommendation as to a system and conclusions for future research. The scope was not to do a business case analysis on the use of UAVs in Louisiana bridge inspection, but instead only an initial demonstration of the viability and potential for practical use of UAVs to assist bridge inspectors for more efficiency and safety. The rest of this paper proceeds as follows. Section 2, Research Done by Others, explores other research in the area of UAV-base bridge inspection. Section 3 describes the paper's research methodology and describes the challenge of doing UAV-based bridge inspection in Louisiana, recommends criteria for selecting aircraft and instrumentation, and selects an aircraft and its associated camera and instruments for experimentation and trial purposes in Louisiana. Section 4, Experimental Results, discusses experiments and results with the selected UAV. It examines the results to render findings as to the effectiveness and practical utility of UAVs for bridge inspection. Section 5 concludes recommending future research.

Literature review
Authors N. Hallermann, et al. [4], found that conventional bridge inspection is mostly based on visual investigation. Critical structural components and hot spots that are hard to reach or technically complex, are inspected by trained staff with large under-bridge platform machines. They concluded that remote controlled UAVs equipped with high definition photo and video cameras can simplify inspection tasks ( Figures  1 and 2) [4].
The authors discuss how the aircraft deals with lighting conditions and handles motion-induced blur from wind, as well as its response to GPS signal loss are important considerations for bridge inspection. They also conclude that visual inspection can be simplified and made safer and more efficient using UAVs. Moreover, they describe how bridge inspections can be pre-planned with the help of bridge inspectors and automated using UAVs with pre-programmed waypoint navigation software to get flight patterns of crucial bridge sections, ensuring equal and appropriate distances, for better image quality control. With pre-programmed UAVs, persons in the field don't necessarily have to be bridge inspection experts. While the authors provide some qualitative data that support the case for using UAVs in bridge inspection, they used a relatively costly solution, i.e. an ASCTec Falcon UAV, superior to many Commercial Off the Shelf (COTS) UAVs with a full 360 deg up and down pan capability for its 36-megapixel camera, but having a price tag of $36,000 [4].  The State of Minnesota Department of Transportation (MnDoT) conducted a demonstration project involving the use of a UAV to view four bridges at locations throughout Minnesota. It investigated the effectiveness of UAVs as compared to other access methods, and its use as a tool for interim and special inspections [3]. Different UAV technologies were investigated to evaluate current and future capabilities for bridge inspection. Additionally, current and proposed Federal Aviation Administration (FAA) rules were investigated to determine their constraints on inspection. Minnesota concluded the following: 1) The use of UAVs to assist in bridge inspection should be considered as a tool for a qualified Team Leader (certified by MnDoT to conduct inspections) when a hands-on inspection is not required. 2) UAVs should be used to aid in routine bridge inspections to improve inspection quality by obtaining information and detail that may not be easily obtained without expensive access methods. 3) UAVs should also be considered where they can increase personnel safety for inspectors and the travelling public.
The MnDoT recommended that a future researchdemonstration phase be considered and should include the following investigation topics: 1) Do a cost comparison with aerial work platforms and traffic control. 2) Explore inspection-specific UAV technology including the Sensfly eXom UAV. 3) a best practices document should be compiled. 4) Plan to incorporate UAV technology into an actual inspection. 5) Explore using a UAV to plan inspections. 6) Use a secondary display with the UAV, available for the bridge inspector Team Leader. 7) Use a zoom camera for surveys of the bridge deck. 8) Explore using UAV technology to perform inspections of culvert and box girder bridges, i.e. inside of the culvert or box girder, which does not require FAA approval. 9) It is recommended that infrared (IR) cameras be used to inspect bridges having known deck delaminations, and that inspections be done at dawn, so that the images show up better. 10) It is also recommended that UAVs be used to conduct paint assessments of existing bridges [3].

Methodology
Our research methodology included an investigation into the pertinent technologies, sciences, and intersecting areas for the problem at hand (i.e., determining the criteria, considerations, findings, and recommendations pertinent to practical bridge inspection in the State of Louisiana using UAVs). For example, much was learned through interviews and research of the literature on the process and science of bridge inspection itself, as well as the type of bridges and their important structural parts [5,2]. The science of crack detection in bridge concrete members was explored leading to some insight into the types of instrumentation needed and future research considerations [6].
Due to the nature of this research, much of the data and analysis was qualitative, rather than quantitative. Still, the plan was to analyze quantitative data in an objective way. It included an investigation of other similar investigations [3,7], appropriate instrumentation and the performance qualities of the UAVs themselves as they relate to the process of bridge inspection, and an investigation of FAA (Federal Aviation Administration) regulatory requirements with bearing on this topic [8]. Importance was placed on surveys and interviews with knowledgeable parties (e.g., the Louisiana Department of Transportation and Development, licensed and experienced pilots [9], and the use of a panel of engineering students, a UAV pilot, and UAV hobbyists to assist in considerations and analysis of the qualitative data gathered according to a developed rubric to allow that data to be objectively examined and compared. Further, the research included experimentation, results tabulation, an alternatives and tradeoffs analysis, and the selection UAV-instrumentation equipment for purchase appropriate to configuration and modification for application to bridge inspection, as well as a field-test demonstration experiment. A field-test demonstration experiment using the selected and purchased UAV to capture bridge images and video was conducted. Images and video were subsequently reviewed by bridge inspectors and LADOTD so as to provide an experienced recommendation, for a UAV system capable of practically assisting in bridge inspection for the State of Louisiana. In summary, the research included the following: 1) Surveys to gain understanding into bodies of knowledge pertinent to bridge inspecting with unmanned aerial vehicles in Louisiana, 2) an investigation of FAA regulations affecting UAVs in Louisiana Airspace, 3) the establishment of mostly qualitative functional and performance criteria for any UAV and associated cameras, and associated instrumentation system for inspecting bridges in Louisiana, 4) an alternatives and tradeoffs analysis of potential Commercial Off-the-Shelf (COTS) UAV-Instrumentation packages to render a choice of this equipment appropriate to the job at hand, 5) a field test demonstration of an actual bridge inspection with the selected UAV-Instrument equipment to gather qualitative and quantitative performance data, and 6) a review of the field demonstration data with UAV pilot, UAV hobbyists, and the LADOTD, to determine the effectiveness using UAVs for routine bridge inspection in Louisiana.

Criteria used to select COTS aircraftinstrumentation package for experimentation
A panel of engineering students, UAV hobbyists, and a licensed UAV pilot, working along with the PI was utilized to developed criteria for selecting aircraft and instrumentation that included 28 qualitative attributes to characterize the performance of each UAV alternative. Upon the completion of panel discussions pertinent to the review of aforementioned qualitative data gathered, the panel concluded that the following criteria were most important in the use of instrumented UAVs for bridge inspection in Louisiana. 1) Payload Attachment Points: The selected UAV should include sufficient payload attachment points to provide for the optional attachment of one or more cameras and other sensors, 2) Camera Stability: UAV camera(s) should include high definition video and still shot capability, i.e. with at least 4k 60 fps video and 20MP still shot, and have gimbaling capability and automatic camera stabilization, so as to eliminate vibration effects in the images and to increase the life of the camera, 3) First Person View: the UAV's camera and wireless remote controller should include First Person View (FPV) , so that the pilot can essentially see what the UAV camera is seeing, as this makes flight control easily supportive of image gathering 4) Flight Time: the selected UAV should sufficient flight time in minutes, i.e. at least 20 minutes of flight time on each of its interchangeable, rechargeable batteries, 5) Waypoints and GPS navigation: it should include self-stabilizing "fly-by-wire" automatic flight with Global Positioning System (GPS) navigation to allow for safe, stable flight, with the pilot only having to manually control or automatically program 3D flight path waypoints, as opposed to having to rapidly adjust the aircraft to account for drift and external forces like wind, 6) Fault Tolerance: there should be some degree of fault tolerance, e.g. with the capability of a six-prop UAV (hexacopter) flying on five propellers in the event one motor stops working, 7) Wind Stability and Resistance: the craft should have good flight stability even in strong wind (a strong concern), with sufficient range, e.g. 2km from the control transceiver, 8) Obstacle Avoidance: it should have obstacle avoidance capability to prevent the UAV from running into the bridge, and it should be considered a professional or semi-professional UAV with telemetry feedback about aircraft and camera status, and the capability of capturing 2D imagery which can then be used to create 3D models of the bridge, and 9) it should be available at a total cost of under $11,000, as compared to some professional package prices in excess of $140,000, so as to achieve a cost-effective solution where some aircraft losses would be expected due to weather and other potential mishaps, so as to be a viable long-term solution.

Selection of aircraft-instrumentation package
Alternatives and tradeoffs were done by assigning a qualitative metric rating to the attributes of each UAV that aligned with the panel's established criteria, and by assigning weights to each criteria, so as to allow a rubric for the calculation of a comparative ranking among all UAVs considered. To gain further insight, the panel performed some research on UAV aircraft and instruments, examining online reviews and in some cases testing certain aircraft available to the hobbyists. Research results were discussed at panel meetings in order to finalize the rubric.
Rankings for each instrumented UAV considered were calculated according to the following expression: (1) where is the calculated alternatives and tradeoffs overall ranking of a given UAV-instrument combination from the pool of considered alternatives, is the rating of the UAV-instrument combination with respect to metric , and is the weight, or importance assigned to metric for the UAV-Inspection package use for routine practical bridge inspection in Louisiana. We note here that each of the ranked attributes (ranked from 1 to 10) in the alternatives and tradeoffs rubric were not weighted equally, but the weighting scheme was developed via panel judgment, with the panel consisting of UAV enthusiasts and a UAV pilot. The weight for each attribute was assessed based on pilot experience and experimentation with UAVs, and online reviews. This resulted in a weighting factor ranging from 1 to 10 (with 10 being most important) assigned to each of 28 UAV attributes. The nine previously mentioned attributes were all deemed crucial to UAV performance sufficient for bridge inspection, and hence, assigned weights generally greater than 5, while many other attributes of lesser importance were given weights of 2 or 3, for instance. Specifically, weights of 5, 5, 5, 4, 6, 8, 10, and 8, where assigned to 1) Payload attachment points, 2) Camera gimballing and stability, 3) First Person View capability, 4) Flight time, 5) Waypoints capability, 6) Fault tolerance, 7) Wind stability, and 8) Obstacle avoidance respectively, while a price of under $11,000 was required. Of special importance were 7) i.e. stability in the wind, and 8) obstacle avoidance capability, with assigned weights of 10 and 8 respectively, deemed important for flight near bridge structures where turbulence and obstacles are both present. Ranking for each aircraft was based on panel judgment and experience, considering any experimental test results, online reviews, and aircraft and instrument characteristics, e.g. small cross-sectional area, significant density, and powerful motors, making the craft wind resistant. Similar qualitative analysis was brought to bear when ranking an aircraft according to each attribute.
Instrumented UAVs meeting important referenced criteria, included the DJI Mavic Pro, the GoPro Karma, the Parrot Bebop 2, the Yuneec Typhoon H, the DJU Phantom 4 Pro, and DJI Inspire, and the Yuneec H520. When evaluated according to the panel's comparison rubric, the Yuneec H520, with a of 1292 ranked best, while, in 2nd place was the Yuneec Typhoon H, followed by the DJI Inspire 2 and the DJI Phantom 4 Pro respectively, in 3rd and 4th places. Hence the instrumented UAV selected for experimental demonstration in bridge inspection was the Yuneec H520, a six-rotor craft with attachable E90 camera, rending 4k UHD 60 fps video and 20MP still shot resolution. The camera can be panned a full 360 degrees and up and down. The aircraft came with 3 batteries each with a 20-minute flight capacity allowing for a full 60 minutes flight time in the field before needing to recharge, all or a price of about $4,200.

Experimental results
A UAV equipped with instruments operating near a bridge always has the potential to encounter some mishap, i.e. crashing into a tree or the bridge structure itself, due to a wind gust or some onboard component failure. So, experiments were performed on the selected aircraft to determine its stability, when using floatation devices, especially gusty wind conditions up to 15 mph, common due to turbulence surrounding lofty bridge structures.
In earlier experiment, i.e. Experiment 4.4, it was determined at the sample bridge, that GPS coverage beneath the sample bridge deck was spotty, meaning that certain areas beneath the bridge deck had GPS coverage while others did not. Should the aircraft be flying beneath the bridge deck and lose GPS coverage, its programmed response is to gain altitude and try to return home. Such a response underneath the bridge deck would likely result in the aircraft crashing into the bridge deck as it attempted to gain altitude beneath the bridge. Hence, it was   During the bridge demonstration test, both the UAV's flight characteristics and its E90 camera performance and resolution were evaluated. Both the UAV pilot and independent observers on site noted that the aircraft was quite stable and controllable, allowing for stops to take photos. Subsequent testing of the aircraft showed its ability to follow programmed flight patterns when taking photos of the object (e.g., bridge) at equidistant points. Numerous photos and videos were captured during the inspection of the demo bridge. These reviewed with bridge inspectors from the Lafayette LADOTD. They agreed that the videos and photos taken were of sufficient quality to assist in bridge inspection. The reason bridge inspectors felt the photos and videos were useful in bridge inspection is because corrosion and cracks, e.g. shown in Figures. 4 and 5 could 1) be clearly seen, when zoomed in, even from 30 feet away, and 2) be measured using reference to other objects in the photo having known dimensions. Being able to get to places human inspectors can't reach was important to them. The capability of the UAV to reach those places and also take high-resolution photos at equal distance, and without wind-induced motion blur, was an important characteristic of UAV-assisted bridge inspection

Conclusions
The following findings were derived from the research and demonstration project.

Positives for UAV-based bridge inspection
1) With conventional methods, most of the inspector's work, involves visual inspection and knowing what to look for or, which bridge components to focus on. Aircraft are reasonably priced ($5000 or less) with onboard camera packages, capable of capturing and zooming in for more detail, sufficient for inspection, 2) Much of the inspection time involves setup, i.e. stopping traffic, getting telescoping boom equipment (reach-alls) onto the bridge deck to reach high on the superstructure, and under the bridge deck. A UAV with camera potentially may eliminate the need for a reach-all, saving most of this setup time, so bridges can be inspected more quickly. Traffic may need to be stopped only short intervals. 3) With programmed flight, the UAV can fly a path so photos of the bridge parts are taken at equidistant points, allowing images to be gathered with quality a human photographer can't achieve. Also, the equidistant images allow for accurate measurement of crack and corroded section loss. 4) With reduced setup time, repeat inspections become feasible and with equidistant photos, the progression of cracks and corrosion section loss can be measured. 5) On some tall bridges, boom equipment is not sufficient to reach to top of the superstructure. UAVs can reach these places for visual inspection, increasing safety.

Negatives for UAV-based bridge inspection
1) The UAVs can't easily access some areas, e.g. the dark underneath of the bridge deck, or below the water line, where the UAV can lose GPS navigation, potentially leading to a crash. Also, since most cameras are mounted under the UAV to capture footage looking down from above, it may not be possible on most COTS UAVs to capture images looking up towards the bottom of the bridge deck. 2) Portions of the inspector's work require hands-on physical checks like sounding, i.e. pinging the bridge with a hammer see if wooden structures are hollow inside. This function isn't easily done with a UAV.

Recommendations for future research
Recommendations for future research include 1) The development of an economical UAV system where many of the components can be 3D printed, as mishaps will occur, and these do-it-yourself systems will be highly customizable, accommodating top-mounted gimbals and cameras as well as spotlight illumination for inspection beneath the bridge deck. 2) A methodology using camera imagery of the bridge above the UAV should be researched as a means for navigation when the UAV loses GPS beneath the bridge deck. 3) A method for using small remote-controlled boats, to serve as UAV aircraft carriers is recommended for research, as some 68 captured during Experiment 4.5. These photos are shown in Figures 33 through 34, below.
Note: All of the bridge inspection photos captured during this experiment, as referenced herein, can be zoom-enlarged to a significant degree, digitally.
Digital Close-ups (enlargements), selected from bridge inspection photos referenced herein, were included in the PowerPoint presentation viewed at the DOTD meeting. Excerpts showing enlarged photos are included here, in the next few pages, with commentary. Note: That while some exterior rust is apparent in parts of the bridge superstructure members, the exposed side of the rivets in view appear to be in relatively good condition with no section loss.
bridge spans are quite long. This would save much needed battery capacity, allowing the aircraft carrier to ferry the UAV with a fresh supply of batteries to the point below the bridge immediately needing inspection, providing both a means of transport for the UAV to the worksite, a landing pad, and a visual marker for navigation purposes when the boat is anchored. 4) Finally, a comparison study evaluating the economics and ergonomics of using and not using the UAV to assist inspections should be done, followed by actual comparison test cases with actual bridge inspections, with evaluation of results by experts in bridge inspection, economics, UAV aircraft and instrumentation.