Category Archives: Robotics

I recently had the pleasure of meeting Dr. Robin Murphy when she participated in the 3-day Policy Forum on Humanitarian UAVs, which I organized and ran at the Rockefeller Center in Italy last month. Anyone serious about next generation humanitarian technology should read Robin’s book on rescue robotics. The book provides a superb introduction to the use of robotics in search and rescue missions and doubles as a valuable “how to manual” packed with deep insights, lessons learned and best practices. Rescue robots enable “responders and other stakeholders to sense and act at a distance from the site of a disaster or extreme incident.” While Robin’s focus is predominantly on the use of search-robots for rescue missions in the US, international humanitarian organizations should not overlook the important lessons learned from this experience.

As Robin rightly notes, ‘the impact of earthquakes, hurricanes, flooding […] is increasing, so the need for robots for all phases of a disaster, from prevention to response and recovery, will increase as well.” This is particularly true of aerial robots, or Unmanned Aerial Vehicles (UAVs), which represent the first wide-spread use of robotics in international humanitarian efforts. As such, this blog post relays some of the key insights from the field of rescue robots and aerial UAVs in particular. For another excellent book on the use of UAVs for search and rescue, please see Gene Robinson’s book entitled First to Deploy.

The main use-case for rescue robotics is data collection. “Rescue robots are a category of mobile robots that are generally small enough and portable enough to be transported, used and operated on demand by the group needing the information; such a robot is called a tactical, organic system […].” Tactical means that “the robot is directly controlled by stakeholders with ‘boots on the ground’—people who need to make fairly rapid decisions about the event. Organic means that the robot is deployed, maintained, transported, and tasked and directed by the stakeholder, though, of course, the information can be shared with other stakeholders […].” These mobile robots are “often referred to as unmanned systems to distinguish them from robots used for factory automation.”

There are three types or modalities of mobile robots: Unmanned Ground Vehicles (UGVs), Unmanned Marine Vehicles (UMVs) and Unmanned Aerial Vehicles (UAVs). UGVs are typically used to enter coal mines following cave-in’s or collapsed buildings to search for survivors. Indeed, “mine disasters are the most frequent users or requesters of rescue robots.” As an aside, I found it quite striking that “urban structures are likely to be manually inspected at least four times by different stakeholders” following a disaster. In any event, “few formal response organizations “own rescue robots, which explains the average lag time of 6.5 days for a robot to be used [in] disaster [response].” That said, Robin notes that this lag time is reduced to 0.5 day when a “command institution had a robot or an existing partnership with a group that had robots […].” While “robots are still far from perfect, they are useful.” Robin is careful to note that the failures and gaps described in her book “should not be used as reasons to reject use of a robot but rather as decision aids in selecting a currently available robot and for proactively preparing a field team for what to expect.”

The Florida State Emergency Response Team deployed the first documented use of small UAVs for disaster response following Hurricane Katrina in 2005. Robin Murphy’s Center for Robot-Assisted Search & Rescue (CRASAR) also flew two types of small UAVs to assist with the rescue phase: an AeroVironment Raven (fixed-wing UAV) and an iSENSYS T-Rex variant miniature helicopter (pictured below). Two flights were carried out to “determine whether people were stranded in the area around Pearlington, Mississippi, and if the cresting Pearl River was posing immediate threats.” These affected areas were “unreachable by truck due to trees in the road.” The Raven UAV unfortunately crashed “into a set of power lines […] while landing in a demolished neighborhood.” CRASAR subsequently carried out an additional 32 flights with an iSENSYS IP-3 miniature helicopter to examine “structural damage at seven multistory buildings.”

The second documented deployment of UAVs in Robin’s book occurs in 2009, when a quadrotor used by the Sapienza University of Rome in the aftermath of the L’Aquila earthquake in 2009. Members of the University’s Cognitive Cooperative Robotics Lab deployed the UAV on behalf of the L’Aquila Fire Department. “The deployment in the debris concentrated on demonstrating mobility to fire rescue agencies.” The third documented use of UAVs occurred in Haiti after the 2010 Earthquake. An Elbit Skylark (fixed-wing) UAV was used to survey the state of a distant orphanage near Leogane, just outside the capital.

Several UAV deployments occurred in 2011. After the Christchurch Earthquake in New Zealand, a consumer Parrot AR drone was initially used to fly into a cathedral to inspect the damage (aerial photo bellow). That same year, a Pelican UAV was used in response to the Japan Earthquake and Tsunami to “test multi-robot collaborative mapping in a damaged building at Tohoku University.” In this case, multirobot means that “the UAV was carried by a UGV” to get the former inside the rubble so it could fly inside the damaged building. At least two additional UAVs were used for the emergency at the Fukushima Daiichi nuclear power plant. Note that a “recording radiological sensor was zip tied to [one of the UAVs] in order to get low-altitude surveys.” Still in 2011, two UAVs were used in Cyprus after an explosion damaged a power plant. The UAVs were deployed to “inspect the damage and create a three-dimensional image of the power plant.” This mission “suggested that multiple UAVs could simultaneously map a face of the structure, [thus] accelerating the reconnaissance process. Finally, at least two multiple fixed-wing UAVs were used in Bangkok following the Great Thailand Flood in 2011. These aerial robots were used to “monitor large areas and allow disaster scientists to predict and prevent flooding.”

In 2012, a project funded by the European Union (EU) fielded to UAVs following earthquakes in Northern Italy to assess the exteriors of “two churches that had not been entered [for] safety reasons. The robots were successful and provided engineers and cultural historians with information that could not have been obtained otherwise.” UAV deployments following disasters in Haiti in 2012 and the Philippines in 2013 do not appear in the book, unfortunately. In any event, Robin notes that the main barrier to deploying UGVs, UMVs and UAVs “is not a technical issue but an administrative one.” I would add regulatory constraints as another major hurdle.

Robin’s book provides some excellent operational guidance on how to carry out rescue-robot missions successfully. These guidance notes also identify existing gaps in recent missions. One such gap is the “lack of ability to integrate UAV data with satellite imagery and other geographical sources,” an area that I’m actively working on (see MicroMappers). Robin makes an important observation on the gaps—or more precisely the data gaps that exist in the field of rescue robotics. “Surprisingly few deployments have been reported in the scientific or professional literature, and even fewer have been analyzed in any depth.” And even when “data are collected, many reports lack a unifying framework or conceptual model for analysis.”

This should not be surprising. Rescue robotics, and humanitarian UAVs in particular, “are new areas of discovery.” As such, “their newness means there is a lag in understanding how best to capture performance and even the dimensions that make up performance.” To be sure, “performance goes beyond simple binary declarations of mission success: it requires knowing what worked and why.” Furthermore, the use of UAVs in aid and development requires a “holistic evaluation of the technology in the larger socio-technical system.” I whole heartedly agree with Robin, which is precisely why I’ve been developing standardized indicators to assess the performance of humanitarian UAVs used for data collection, payload transportation and communication services in international humanitarian aid. Such standards are needed earlier rather than later since “the current state of reporting deployments is ad hoc,” which means “there is no guarantee that all deployments have been recorded, much less documented in a manner to support scientific understanding or improved devices and concepts of operations.” I’ll be writing more on the standardized indicators I’ve been developing in a future blog post.

As Robin also notes, “it is not easy to determine if a robot accomplished the mission optimally, was resilient to conditions it did not encounter, or missed an important cue of a victim or structural hazard.” What’s more, “good performance of a robot in one hurricane does not necessarily mean good performance in another hurricane because so many factors can be different.” Fact is, Rescue robotics have a “very small corpus of natural world observations […],” meaning that there is limited documentation based on direct observation of UAV missions in the field. This is also true of humanitarian UAVs. Unlike the science of rescue-robotics, many of the other sciences have a “large corpus of prior observations, and thus ideation may not require new fundamental observations of the natural world.” What does this mean for rescue robotics (and humanitarian UAVs)? According to Robin, the very small corpus of real world observations suggests that lab experimentation and simulations will have “limited utility as there is little information to create meaning models or to know what aspect of the natural world to duplicate.”

I’m still a strong proponent of simulations and disaster response exercises; they are key to catalyzing learning around emerging (humanitarian) technologies in non-high-stakes environments. But I certainly take Robin’s point. What’s very clear is that a lot more fieldwork is needed in rescue-robotics (and especially in the humanitarian UAV space). This fieldwork can be carried out in several ways:

• Controlled Experimentation
• Participation in an Exercise
• Concept Experimentation
• Participant-Observer Research

Controlled experimentation is “highly focused, either on testing a hypothesis or capturing a performance metric(s) […]. Participation in an exercise occurs in simulated-but-realistic environments. This type of fieldwork focuses on “reinforcing good practices […].” Concept experimentation can occur both in simulated environment and in the real world. “The experimentation is focused on generating concepts of how a new technology or protocol can be used […].” This type of experimentation also “identifies new uses or missions for the robot.” Lastly, “participant-observer” research is conducted while the robot is actually deployed to a disaster, and is a form of ethnography.” 

There are many more important, operational insights in Robin’s book. I highly recommend reading sections 3-6 in Chapter 6 since they provide very practical advice on how to carry out rescue-robotics missions. These section are packed with hands-on lessons learned and best practices, which very much mirror my own experience in the humanitarian UAV space, as documented in this best practices guide. For example, she emphasizes the critical importance of having a “Data Manager” as part of your deployment team. “The first priority of the data manager is to gather all the incoming data, and perform backups.” In addition, Robin Murphy strongly recommends that expert participant-observer researcher be embedded in the mission team—another suggestion I completely agree with. In terms of good etiquette, “Do not attempt first contact during a disaster,” is another suggestion that I wholeheartedly agree with. This is precisely why the UN asked UAV operators in Nepal to first check-in with the Humanitarian UAV Network (UAViators).

In closing, big thanks to Robin for writing this book and for participating in the recent Policy Forum on Humanitarian UAVs.