Tag Archives: Search

Debrief: UAV/Drone Search and Rescue Challenge

I had the pleasure of helping co-organize the first UAV/Drone Search and Rescue Challenge in the DC Area last Saturday. This was the first time that members of the DC Area Drone User Group participated in an event like this, so it was an ideal opportunity for everyone involved to better understand how UAVs might be used in a real world emergency to support of professional first responders. The challenge was held at the 65-acre MadCap Farm in The Plains, Virginia. For perspective, 65 acres is equal to about 30 full-size football (soccer) fields.

Madcap Farm

Satellite view of MadCap Farm above versus aerial view below during the UAV Search and Rescue Challenge.

Madcap Farm 2

Big thanks to our host and to Timothy Reuter who organized and ran the challenge; and of course many thanks indeed to all five teams who participated in the challenge. One colleague even flew in from Texas to compete in the event, which was sponsored by UAViators, the Humanitarian UAV Network. I described the rules of the challenge in this post but let me briefly summarize these here. Teams were notified of the following “alert” the night before the challenge:

“We have received reports of three lost campers in the vicinity of MadCap Farms. Local Search & Rescue professionals have requested our help to find them. Please report to the front field of MadCap no later than 9:15am for additional details on the campers and efforts to locate them. You will receive a laminated map of the area upon your arrival as well as a wax pen. We ask that you use your drones to identify objects that may help local responders determine where the campers are and ideally find the campers themselves. You will mark on the maps you receive what items you find, their color, and any people you identify. If any of the campers are trapped, you may need to deliver some form of medicine or other relief to them in advance of first responders being able to aid them in person.”

UAVteams

Upon reporting to the farm the following morning, the teams (pictured above) were notified that the campers were teenagers who were carrying sleeping bags and tents. In reality, our three lost campers were the cardboard stand-ups below, but Timothy had already hidden these and scattered their belongings by the time participants arrived at the farm. By the way, notice all the clouds in the picture above? This would have hampered efforts to use satellite imagery in the search and rescue efforts. UAVs, in contrast, fly below the cloud canopy and can provide far cheaper and more up-to-date imagery at far higher spatial resolutions and available even using the best commercial satellites.

LostCampers2

As a side note, I was really pleased to see the Civilian Air Patrol (CAP) at the Search and Rescue Challenge. The Air Patrol is a federally supported non-profit volunteer-based organization that serves as the official civilian auxiliary of the US Air Force. CAP members share a passion for aviation and come from all backgrounds and walks of life.

UAV_CAP

Back to the Challenge. Each team had an hour to fly their UAVs and another hour to search through their aerial videos and/or images post-flight. Two of the five teams used fixed-wing UAVs, like the group below, which kicked off our Search & Rescue Challenge.

UAVgroup1

They decided to program their UAV for autonomous flight. You can see the flight path below with specified altitude and the different way points (numbers) in the top-right screen (click to enlarge).

UAVgroup1 autonomous

Here’s a short 20-second video of the hand-held launch of the fixed-wing UAV. Once airborne, the team simply switches to auto-pilot and the UAV does the rest, accurately following the pre-programmed flight path.

The team decided to analyze their streaming aerial video in real-time, as you can observe in the second video below. While this certainly expedites the analysis and the search for the missing campers, it is also challenging since the team has to pivot back and forth between the live video and the flight path of the UAV in order to pin-point the location of a potential camper or their tent. Unlike rotary-wing UAVs, fixed-wing UAVs obviously cannot hover over one area but need to circle back to fly over the same area.

My colleague Michael and his co-pilot programmed a quadcopter to fly to designated waypoints at a specified altitude. They too used live-streaming to look for clues that could reveal the location of the three missing campers. But they also recorded the video-feed for later analysis, which proved far more effective at identifying said clues. In any event, they used First Person View (FPV) goggles to see exactly what the quadcopter’s camera was seeing, as depicted below.

FPV Goggles Quadcopter

In addition to searching for the whereabouts of the missing campers, Timothy and I decided to add a bit more focus on the “rescue” part of Search & Rescue. My colleague Euan kindly gave us a number of his new payload units, which are basically a pair of magnets that can be demagnetized by passing a small electric current through said magnets, thus acting as a release mechanism. Euan posted this short video of his prototype payload units in action during a first test earlier this year. Competing teams could earn additional points if they were able to carry a small payload (of their choice) and release this near any of the cardboard stand-ups they could find.

UAV payload unit

Some teams used Euan’s units while other used their own, like the device pictured above. Here’s a short video of payload release (with parachute) during the competition.

At the end of the competition, we all came together for a full debrief and of course to count up points. Timothy asked each team to share what they felt went well and what some of the major challenges were. The picture below shows some of the items (sleeping bags, clothing, etc.) that were scattered around the farm.

UAV debrief

Perhaps the single biggest challenge was altitude. Given that we were in a valley surrounded by rolling hills, it was difficult for competing teams to judge at what altitude their UAVs should be programmed to fly since we couldn’t see over the next hill to determine whether there were taller trees in the distance.

UAVtrees

Flying too high would make it more difficult to identify the potential campers on the ground while flying too low would mean running into trees. Unfortunately, two teams encountered the latter problem but both UAVs were eventually recovered. This highlights the importance of developing automatic collision avoidance systems (ACAS) specifically for UAVs. In addition, if UAVs are to be used for Search and Rescue efforts in forested areas, it would pay to have a back-up quadcopter to rescue any UAVs caught in taller trees. One could attach a hanger to said quadcopter to unhook UAVs out of trees. The picture below is taken by a camera fixed to a quadcopter that hit the top of a tree. Yes, we all had a good laugh about the irony of sending UAVs to rescue other UAVs.

StuckQuadUAV

The debrief also revealed that most teams were able to find more items post-flight after going through their recorded video footage. My colleague Michael noted that finding signs of the campers was “like looking for a needle in a haystack.” One team noted that live video feeds have a limited range, which hampered their efforts. Another team remarked that one can never have enough batteries on hand. Indeed, wind conditions can very easily affect the endurance of UAV batteries, for example. The importance of pre-flight check-lists was reiterated as well as clearly spelling out safety protocols before a challenge.

UAViators Logo

I’ll be sharing this debrief and lessons learned with my humanitarian colleagues at the United Nations and the Red Cross; as well as with members of the Advisory Board of the Humanitarian UAV Network (UAViators). Keep in mind that groups like UNICEF, UNHCR and the UN Office for the Coordination for Humanitarian Affairs (OCHA) have not yet begun to experiment hands-on with UAVs to support their relief efforts, so all of the above will be very new to them, just as it was to most teams who participated in the challenge. So this kind of hands-on learning will be of interest to humanitarians groups looking to explore this space.

Counting Points UAVs

We counted up the points from each team’s map (like the one above) after the debrief and congratulated the winning team pictured below. They were the only team that found all three missing campers along with some of their belongings.

Winning UAV

Big thanks again to our hosts at MadCap Farm, to Timothy Reuter and all participants for spending a fun Saturday outdoors trying something new. We certainly learned some valuable lessons and in the process made new friends.

The short video above was produced by CCTV America, a news television channel that reported on the Search & Rescue Challenge.

bio

Acknowledgements: Many thanks to Timothy Reuter and Michael Ender for their feedback on an earlier draft of this blog post.

See also:

  • How UAVs are Making a Difference in Disaster Response [link]
  • Humanitarians Using UAVs for Post Disaster Recovery [link]
  • Grassroots UAVs for Disaster Response [link]
  • Using UAVs for Search & Rescue [link]
  • Crowdsourcing Analysis of UAV Imagery for Search/Rescue [link]

Launching a Search and Rescue Challenge for Drone / UAV Pilots

My colleague Timothy Reuter (of AidDroids fame) kindly invited me to co-organize the Drone/UAV Search and Rescue Challenge for the DC Drone User Group. The challenge will take place on May 17th near Marshall in Virginia. The rules for the competition are based on the highly successful Search/Rescue challenge organized by my new colleague Chad with the North Texas Drone User Group. We’ll pretend that a person has gone missing by scattering (over a wide area) various clues such as pieces of clothing & personal affects. Competitors will use their UAVs to collect imagery of the area and will have 45 minutes after flying to analyze the imagery for clues. The full set of rules for our challenge are listed here but may change slightly as we get closer to the event.

searchrescuedrones

I want to try something new with this challenge. While previous competitions have focused exclusively on the use of drones/UAVs for the “Search” component of the challenge, I want to introduce the option of also engaging in the “Rescue” part. How? If UAVs identify a missing person, then why not provide that person with immediate assistance while waiting for the Search and Rescue team to arrive on site? The UAV could drop a small and light-weight first aid kit, or small water bottle, or even a small walkie talkie. Enter my new colleague Euan Ramsay who has been working on a UAV payloader solution for Search and Rescue; see the video demo below. Euan, who is based in Switzerland, has very kindly offered to share several payloader units for our UAV challenge. So I’ll be meeting up with him next month to take the units back to DC for the competition.

Another area I’d like to explore for this challenge is the use of crowdsourcing to analyze the aerial imagery & video footage. As noted here, the University of Central Lancashire used crowdsourcing in their UAV Search and Rescue pilot project last summer. This innovative “crowdsearching” approach is also being used to look for Malaysia Flight 370 that went missing several weeks ago. I’d really like to have this crowdsourcing element be an option for the DC Search & Rescue challenge.

UAV MicroMappers

My team and I at QCRI have developed a platform called MicroMappers, which can easily be used to crowdsource the analysis of UAV pictures and videos. The United Nations (OCHA) used MicroMappers in response to Typhoon Yolanda last year to crowdsource the tagging pictures posted on Twitter. Since then we’ve added video tagging capability. So one scenario for the UAV challenge would be for competitors to upload their imagery/videos to MicroMappers and have digital volunteers look through this content for clues of our fake missing person.

In any event, I’m excited to be collaborating with Timothy on this challenge and will be share updates on iRevolution on how all this pans out.

bio

See also:

  • Using UAVs for Search & Rescue [link]
  • Crowdsourcing Analysis of UAV Imagery for Search and Rescue [link]
  • How UAVs are Making a Difference in Disaster Response [link]
  • Grassroots UAVs for Disaster Response [link]

Crowdsourcing the Search for Malaysia Flight 370 (Updated)

Early Results available here!

Update from Tomnod: The response has literally been overwhelming: our servers struggled to keep up all day.  We’ve been hacking hard to make some fixes and I think that the site is working now but I apologize if you have problems connecting: we’re getting up to 100,000 page views every minute! DigitalGlobe satellites are continuing to collect imagery as new reports about the possible crash sites come in so we’ll keep updating the site with new data.

Beijing-bound Flight 370 suddenly disappeared on March 8th without a trace. My colleagues at Tomnod have just deployed their satellite imagery crowdsourcing platform to support the ongoing Search & Rescue efforts. Using high-resolution satellite imagery from DigitalGlobe, Tomnod is inviting digital volunteers from around the world to search for any sign of debris from missing Boeing 777.

MH370

The DigitalGlobe satellite imagery is dated March 9th and covers over 1,000 square miles. What the Tomnod platform does is slice that imagery into many small squares like the one below (click to enlarge). Volunteers then tag one image at a time. This process is known as microtasking (or crowd computing). For quality control purposes, each image is shown to more than one volunteer. This consensus-based approach allows Tomnod to triangulate the tagging.

TomNod

I’ve long advocated for the use of microtasking to support humanitarian efforts. In 2010, I wrote about how volunteers used microtasking to crowdsource the search for Steve Fossett who had disappeared while flying a small single-engine airplane in Nevada. This was back in 2007. In 2011, I spearheaded a partnership with the UN Refugee Agency (UNCHR) in Somalia and used the Tomnod platform to crowdsource the search for internally displaced populations in the drought-stricken Afgooye Corridor. More here. I later launched a collaboration with Amnesty International in Syria to crowdsource the search for evidence of major human rights violations—again with my colleagues from Tomnod. Recently, my team and I at QCRI have been developing MicroMappers to support humanitarian efforts. At the UN’s request, MicroMappers was launched following Typhoon Yolanda to accelerate their rapid damage assessment. I’ve also written on the use of crowd computing for Search & Rescue operations.

TomnodSomalia

I’m still keeping a tiny glimmer of hope that somehow Malaysia Flight 370 was able to land somewhere and that there are survivors. I can only image what families, loved ones and friends must be going through. I’m sure they are desperate for information, one way or another. So please consider spending a few minutes of your time to support these Search and Rescue efforts. Thank you.

Bio

Note: If you don’t see any satellite imagery on the Tomnod platform for Flight 370, this means the team is busy uploading new imagery. So please check in again in a couple hours.

See also: Analyzing Tweets on Malaysia Flight #MH370 [link]

Using Crowd Computing to Analyze UAV Imagery for Search & Rescue Operations

My brother recently pointed me to this BBC News article on the use of drones for Search & Rescue missions in England’s Lake District, one of my favorite areas of the UK. The picture below is one I took during my most recent visit. In my earlier blog post on the use of UAVs for Search & Rescue operations, I noted that UAV imagery & video footage could be quickly analyzed using a microtasking platform (like MicroMappers, which we used following Typhoon Yolanda). As it turns out, an enterprising team at the University of Central Lancashire has been using microtasking as part of their UAV Search & Rescue exercises in the Lake District.

Lake District

Every year, the Patterdale Mountain Rescue Team assists hundreds of injured and missing persons in the North of the Lake District. “The average search takes several hours and can require a large team of volunteers to set out in often poor weather conditions.” So the University of Central Lancashire teamed up with the Mountain Rescue Team to demonstrate that UAV technology coupled with crowdsourcing can reduce the time it takes to locate and rescue individuals.

The project, called AeroSee Experiment, worked as follows. The Mountain Rescue service receives a simulated distress call. As they plan their Search & Rescue operation, the University team dispatches their UAV to begin the search. Using live video-streaming, the UAV automatically transmits pictures back to the team’s website where members of the public can tag pictures that members of the Mountain Rescue service should investigate further. These tagged pictures are then forwarded to “the Mountain Rescue Control Center for a final opinion and dispatch of search teams.” Click to enlarge the diagram below.

AeroSee

Members of the crowd would simply log on to the AeroSee website and begin tagging. Although the experiment is over, you can still do a Practice Run here. Below is a screenshot of the microtasking interface (click to enlarge). One picture at a time is displayed. If the picture displays potentially important clues, then the digital volunteer points to said area of the picture and types in why they believe the clue they’re pointing at might be important.

AeroSee MT2

The results were impressive. A total of 335 digital volunteers looked through 11,834 pictures and the “injured” walker (UAV image below) was found within 69 seconds of the picture being uploaded to microtasking website. The project team subsequently posted this public leaderboard to acknowledge all volunteers who participated, listing their scores and levels of accuracy for feedback purposes.

Aero MT3

Upon further review of the data and results, the project team concluded that the experiment was a success and that digital Search & Rescue volunteers were able to “home in on the location of our missing person before the drones had even landed!” The texts added to the tagged images were also very descriptive, which helped the team “locate the casualty very quickly from the more tentative tags on other images.”

If the area being surveyed during a Search & Rescue operation is fairly limited, then using the crowd to process UAV images is a quick and straightforward, especially if the crowd is relatively large. We have over 400 digital humanitarian volunteers signed up for MicroMappers (launched in November 2013) and hope to grow this to 1,000+ in 2014. But for much larger areas, like Kruger National Park, one would need far more volunteers. Kruger covers 7,523 square miles compared to the Lake District’s 885 square miles.

kruger-gate-sign

One answer to this need for more volunteers could be the good work that my colleagues over at Zooniverse are doing. Launched in February 2009, Zooniverse has a unique volunteer base of one million volunteers. Another solution is to use machine computing to prioritize the flights paths of UAVs in the first place, i.e., use advanced algorithms to considerably reduce the search area by ruling out areas that missing people or other objects of interest (like rhinos in Kruger) are highly unlikely to be based on weather, terrain, season and other data.

This is the area that my colleague Tom Snitch works in. As he noted in this recent interview (PDF), “We want to plan a flight path for the drone so that the number of unprotected animals is as small as possible.” To do this, he and his team use “exquisite mathematics and complex algorithms” to learn how “animals, rangers and poachers move through space and time.” In the case Search & Rescue, ruling out areas that are too steep and impossible for humans to climb or walk through could go a long way to reducing the search area not to mention the search time.

bio

See also:

  • Using UAVs for Search & Rescue [link]
  • MicroMappers: Microtasking for Disaster Response [link]
  • Results of MicroMappers Response to Typhoon Yolanda [link]
  • How UAVs are Making a Difference in Disaster Response [link]
  • Crowdsourcing Evaluation of Sandy Building Damage [link]

Using UAVs for Search & Rescue

UAVs (or drones) are starting to be used for search & rescue operations, such as in the Philippines following Typhoon Yolanda a few months ago. They are also used to find missing people in the US, which may explain why members of the North Texas Drone User Group (NTDUG) are organizing the (first ever?) Search & Rescue challenge in a few days. The purpose of this challenge is to 1) encourage members to build better drones and 2) simulate a real world positive application of civilian drones.

Drones for SA

Nine teams have signed up to compete in Saturday’s challenge, which will be held in a wheat field near Renaissance Fair in Waxahachie, Texas (satellite image below). The organizers have already sent these teams a simulated missing person’s report. This will include a mock photo, age, height, hair color, ethnicity, clothing and where/when this simulated lost person was last seen. Each drone must have a return to home function and failsafe as well as live video streaming.

Challenge location

When the challenge launches, each team will need to submit a flight plan to the contest’s organizers before being allowed to search for the missing items (at set times). An item is considered found when said item’s color or shape can be described and if the location of this item can be pointed to on a Google Map. These found objects then count as points. Points are also awarded for finding tracks made by humans or animals, for example. Points will be deducted for major crashes, for flying at an altitude above the 375 feet limit and risk disqualification for flying over people.

While I can’t make it to Waxahachie this weekend to observe the challenge first-hand, I’m thrilled that the DC Drones group (which I belong to), is preparing to host its own drones search & rescue challenge this Spring. So I hope to be closely involved with this event in the coming months.

Wildlife challenge

Although search & rescue is typically thought of as searching for people, UAVs are also beginning to appear in conversations about anti-poaching operations. At the most recent DC Drones MeetUp, we heard a presentation on the first ever Wildlife Conservation UAV Challenge (wcUAVc). The team has partnered with Krueger National Park to support their anti-poaching efforts in the face of skyrocketing Rhino poaching.

Rhino graph

The challenge is to “design low cost UAVs that can be deployed over the rugged terrain of Kruger, equipped with sensors able to detect and locate poachers, and communications able to relay accurate and timely intelligence to Park Rangers.” In addition, the UAVs will have to “collect RFID tag data throughout the sector; detect, classify, and tack all humans; regularly report on the location of all rhinos and humans; and receive commands to divert from general surveillance to support poacher engagement anywhere in the sector. They also need to be able to safely operate in same air space with manned helicopters, assisting special helicopter borne rangers engage poachers.” All this for under $3,000.

Why RFID tag data? Because rangers and tourists in Krueger National Park all carry RFID tags so they can be easily located. If a UAV automatically detects a group of humans moving through the bush and does not find an RFID signature for them, the UAV will automatically conclude that they may be poachers. When I spoke with one of the team members following the presentation, he noted that they were also interested in having UAVs automatically detect whether humans are carrying weapons. This is no small challenge, which explains why the total cash prize is $65,000 and an all-inclusive 10-day trip to Krueger National Park for the winning team.

I think it would be particularly powerful if the team could open up the raw footage for public analysis via microtasking, i.e., include a citizen science component to this challenge to engage and educate people from around the world about the plight of rhinos in South Africa. Participants would be asked to tag imagery that show rhinos and humans, for example. In so doing, they’d learn more about the problem, thus becoming better educated and possibly more engaged. Perhaps something along the lines of what we do for digital humanitarian response, as described here.

Drone Innovation Award

In any event, I’m a big proponent of using UAVs for positive social impact, which is precisely why I’m honored to be an advisor for the (first ever?) Drones Social Innovation Award. The award was set up by my colleague Timothy Reuter (founder of the the Drone User Group Network, DUGN). Timothy is also launching a startup, AirDroids, to further democratize the use of micro-copters. Unlike similar copters out there, these heavy-lift AirDroids are easier to use, cheaper and far more portable.

As more UAVs like AirDroids hit the market, we will undoubtedly see more and more aerial photo- and videography uploaded to sites like Flickr and YouTube. Like social media, I expect such user-generated imagery to become increasingly useful in humanitarian response operations. If users can simply slip their smartphones into their pocket UAV, they could provide valuable aerial footage for rapid disaster damage assessments purposes, for example. Why smart-phones? Because people already use their smartphones to snap pictures during disasters. In addition, relatively cheap hardware add-on’s can easily turn smartphones for LIDAR sensing and thermal imaging.

All this may eventually result in an overflow of potentially useful aerial imagery, which is where MicroMappers would come in. Digital volunteers could easily use MicroMappers to quickly tag UAV footage in support of humanitarian relief efforts. Of course, UAV footage from official sources will also continue to play a more important role in the future (as happened following Hurricane Sandy). But professional UAV teams are already outnumbered by DIY UAV users. They simply can’t be everywhere at the same time. But the crowd can. And in time, a bird’s eye view may become less important than a flock’s eye view, especially for search & rescue and rapid disaster assessments.

Bio

 See also:

  • How UAVs are Making a Difference in Disaster Response [link]
  • UN World Food Program to Use UAVs [link]
  • Drones for Human Rights: Brilliant or Foolish? [link]
  • The Use of Drones for Nonviolent Civil Resistance [link]