Using Crowd Computing to Analyze UAV Imagery for Search & Rescue Operations

My brother recently pointed me to this BBC News article on the use of drones for Search & Rescue missions in England’s Lake District, one of my favorite areas of the UK. The picture below is one I took during my most recent visit. In my earlier blog post on the use of UAVs for Search & Rescue operations, I noted that UAV imagery & video footage could be quickly analyzed using a microtasking platform (like MicroMappers, which we used following Typhoon Yolanda). As it turns out, an enterprising team at the University of Central Lancashire has been using microtasking as part of their UAV Search & Rescue exercises in the Lake District.

Every year, the Patterdale Mountain Rescue Team assists hundreds of injured and missing persons in the North of the Lake District. “The average search takes several hours and can require a large team of volunteers to set out in often poor weather conditions.” So the University of Central Lancashire teamed up with the Mountain Rescue Team to demonstrate that UAV technology coupled with crowdsourcing can reduce the time it takes to locate and rescue individuals.

The project, called AeroSee Experiment, worked as follows. The Mountain Rescue service receives a simulated distress call. As they plan their Search & Rescue operation, the University team dispatches their UAV to begin the search. Using live video-streaming, the UAV automatically transmits pictures back to the team’s website where members of the public can tag pictures that members of the Mountain Rescue service should investigate further. These tagged pictures are then forwarded to “the Mountain Rescue Control Center for a final opinion and dispatch of search teams.” Click to enlarge the diagram below.

Members of the crowd would simply log on to the AeroSee website and begin tagging. Although the experiment is over, you can still do a Practice Run here. Below is a screenshot of the microtasking interface (click to enlarge). One picture at a time is displayed. If the picture displays potentially important clues, then the digital volunteer points to said area of the picture and types in why they believe the clue they’re pointing at might be important.

The results were impressive. A total of 335 digital volunteers looked through 11,834 pictures and the “injured” walker (UAV image below) was found within 69 seconds of the picture being uploaded to microtasking website. The project team subsequently posted this public leaderboard to acknowledge all volunteers who participated, listing their scores and levels of accuracy for feedback purposes.

Upon further review of the data and results, the project team concluded that the experiment was a success and that digital Search & Rescue volunteers were able to “home in on the location of our missing person before the drones had even landed!” The texts added to the tagged images were also very descriptive, which helped the team “locate the casualty very quickly from the more tentative tags on other images.”

If the area being surveyed during a Search & Rescue operation is fairly limited, then using the crowd to process UAV images is a quick and straightforward, especially if the crowd is relatively large. We have over 400 digital humanitarian volunteers signed up for MicroMappers (launched in November 2013) and hope to grow this to 1,000+ in 2014. But for much larger areas, like Kruger National Park, one would need far more volunteers. Kruger covers 7,523 square miles compared to the Lake District’s 885 square miles.

One answer to this need for more volunteers could be the good work that my colleagues over at Zooniverse are doing. Launched in February 2009, Zooniverse has a unique volunteer base of one million volunteers. Another solution is to use machine computing to prioritize the flights paths of UAVs in the first place, i.e., use advanced algorithms to considerably reduce the search area by ruling out areas that missing people or other objects of interest (like rhinos in Kruger) are highly unlikely to be based on weather, terrain, season and other data.

This is the area that my colleague Tom Snitch works in. As he noted in this recent interview (PDF), “We want to plan a flight path for the drone so that the number of unprotected animals is as small as possible.” To do this, he and his team use “exquisite mathematics and complex algorithms” to learn how “animals, rangers and poachers move through space and time.” In the case Search & Rescue, ruling out areas that are too steep and impossible for humans to climb or walk through could go a long way to reducing the search area not to mention the search time.