Category Archives: Robotics

Could These Swimming Robots Help Local Communities?

Flying robots are all over the news these days. But UAVs/drones are hardly the only autonomous robotics solutions out there. Driving robots like the one below by Starship Technologies has already driven more than 5,000 miles in 39 cities across 12 counties, gentling moving around hundreds of thousands of people in the process of delivering small packages. I’m already in touch with Starship and related companies to explore a range of humanitarian applications. Perhaps less well known, however, are the swimming robots that can be found floating and diving in oceans, lakes and rivers around the world.

These swimming robots are often referred to as maritime or marine robots, aquatic robots, remotely operated vehicles and autonomous surface water (or underwater) vehicles. I’m interested in swimming robots for the same reason I’m interested in flying and driving robots: they allow us to collect data, transport cargo and take samples in more efficient and productive ways. Flying Robots, for example, can be used to transport essential vaccines and medicines. They can also collect data by taking pictures to support precision agriculture and they can take air samples to test for pollution. The equivalent is true for swimming and diving robots.

So I’d like to introduce you to this cast of characters and will then elaborate on how they can and have been used to make a difference. Do please let me know if I’m missing any major ones—robots and use-cases.

OpenRov Trident
OpenROV_Trident
This tethered diving robot can reach depths of up to 100 meters with a maximum speed of 2 meters per second (7 km/hour). The Trident has a maximum run-time of 3 hours, weighs just under 3 kg and easily fits in a backpack. It comes with a 25 meter tether (although longer tethers are also available). The robot, which relays a live video feed back to the surface, can be programmed to swim in long straight lines (transects) over a given area to generate a continuous map of the seafloor. The OpenROV software is open source. More here.

MidWest ROV Screen Shot 2016-07-29 at 8.00.07 AM
This remotely operated swimming robot has an maximum cruising speed of just under 5km per hour and weighs 25kg. The ROV is a meter long and has a run time of approximately 4 hours. The platform has full digital and audio recording capabilities with a sonar a scanner that can record a swatch of ~60 meter wide at a depth of 180 meters. This sturdy robot has been swimming in one of the fastest changing glacier lakes in the Himalayas to assess flood hazards. More here. See also MarineTech.

Hydromea Vertex AUV

This small swimming robot can cover a volume of several square kilometers at a depth of up to 300 meters with a maximum speed of 1 meter per second (3.5 km/hour). The Vertex can automatically scan vertically and horizontally, or any other angle for that matter and from multiple locations. The platform, which only weighs 7 kg and has a length of 70 cm, can be used to create 3D scans of the seafloor with up to 10 robots operating in simultaneously in parallel thanks to communication and localization technology that enables them to cooperate as a team. More here.

Liquid Robotics Wave Glider
LiquidRobotics
The Wave Glider is an autonomous swimming robot powered by both wave and solar energy, enabling it to cruise at 5.5 km/hour. The surface component, which measures 3 meters in length, contains solar panels that power the platform and onboard sensors. The tether and underwater component enables the platform to use waves for thrust. This Glider operates individually or in fleets to deliver real-time data for up to a year with no fuel. The platform has already traveled well over one million kilometers and through a range of weather conditions including hurricanes and typhoons. More here.

SeaDrone
SeaDrone
This tethered robot weighs 5kg and can operate at a depth of 100 meters with a maximum of 1.5m per second (3.5km/hour). Tethers are available at a length of 30 meters to 50 meters. The platform has a battery life of 3 hours and provides a live, high-definition video feed. The SeaDrone platform can be easily controlled from an iOS tablet. More here.

Clear Path Robotics Heron
ClearPath
This surface water swimming robot can cruise at a maximum speed of 1.7 meters per second (6km/ hour) for around 2 hours. The Heron, which weighs 28kg, offers a payload bay for submerged sensors and a mounting system for those above water. The robot can carry a maximum payload of 10kg. A single operator can control multiple Herons simultaneously. The platform, like others described below is ideal for ecosystem assessments and bathymetry surveys (to map the topography of lakes and ocean floors). More here.

SailDrone
SailDrone
The Saildrone navigates to its destination using wind power alone, typically cruising at an average speed of 5.5 km/hour. The robot can then stay at a designated spot or perform survey patterns. Like other robots introduced here, the Saildrone can carry a range of sensors for data collection. The data is then transmitted back to shore via satellite. The Saildrone is also capable of carrying an additional 100 kg worth of payload. More here.

EMILY
Screen Shot 2016-07-29 at 2.29.07 PM
EMILY, an acronym for Emergency Integrated Lifesaving Lanyard, is a robotic device used by lifeguards for rescuing swimmers. It operates on battery power and is operated by remote control after being dropped into the water from shore, a boat or pier, or helicopter. EMILY has a maximum cruising speed of 35km per hour (much faster than a human lifeguard can swim) and function as a floatation device for up to 4-6 people. The platform was used in Greece to assist in ocean rescues of refugees crossing the Aegean Sea from Turkey. More here. The same company has also created Searcher, an autonomous marine robot that I hope to learn more about soon.

Platypus
Platypus
Platypus manufactures four different types of swimming robots one which is depicted above. Called the Serval, this platform has a maximum speed of 15 km/hour with a runtime of 4 hours. The Serval weighs 35kg and can carry a payload of 70kg. The Serval can use either wireless, 3G or Edge to communicate. Platypus also offers a base-station package that includes a wireless router and antenna with range up to 2.5 km. The Felis, another Playtpus robot, has a max speed of 30km/hour and a max payload of 200kg. The platform can operate for 12 hours. These platforms can be used for autonomous mapping. More here.

AquaBot
AquaBot
The aim of the AquaBot project is to develop an underwater tethered robot that automates the tasks of visual inspection of fish farm nets and mooring systems. There is little up-to-date information on this project so it is unclear how many prototypes and tests were carried out. Specs for this diving robot don’t seem to be readily available online. More here.

There are of course many more marine robots out there. Have a look at these other companies: Bluefin Robotics, Ocean Server, Riptide Autonomous Systems, Seabotix, Blue Robotics, YSI, AC-CESS and Juice Robotics, for example. The range of applications of maritime robotics can be applied to is also growing. At WeRobotics, we’re actively exploring a wide number of use-cases to determine if and where maritime robots might be able to add value to the work of our local partners in developing countries.

aquaculture

Take aquaculture (also known as aquafarming), for example. Aquaculture is the fastest growing sector of global food production. But many types of aquaculture remain labor intensive. In addition, a combination of “social and environmental pressures and biological necessities are creating opportunities for aquatic farms to locate in more exposed waters further offshore,” which increases both risks and costs, “particularly those associated with the logistics of human maintenance and intervention activities.” These and other factors make “this an excellent time to examine the possibilities for various forms of automation to improve the efficiency and cost-effectiveness of farming the oceans.”

Just like land-based agriculture, aquaculture can also be devastated by major disasters. To this end, aquaculture represents an important food security issue for local communities directly dependent on seafood for their livelihoods. As such, restoring aquafarms can be a vital element of disaster recovery. After the 2011 Japan Earthquake and Tsunami, for example, maritime robots were used to “remediate fishing nets and accelerate the restarting of the fishing economy.” As further noted in the book Disaster Robotics, the robots “cleared fishing beds from debris and pollution” by mapping the “remaining debris in the prime fishing and aquaculture areas, particularly looking for cars and boats leaking oil and gas and for debris that would snag and tear fishing nets.”

JapanTsunami

Ports and shipping channels in both Japan and Haiti were also reopened using marine robots following the major earthquakes in 2011 and 2010. They mapped the debris field that could damage or prevent ships from entering the port. The clearance of this debris allowed “relief supplies to enter devastated areas and economic activities to resume.” To be sure, coastal damage caused by earthquake and tsunamis can render ports inoperable. Marine robots can thus accelerate both response and recovery efforts by reopening ports that represent primary routes for relief supplies, as noted in the book Disaster Robotics

In sum, marine robotics can be used for aquaculture, structural inspections, estimation of debris volume and type, victim recovery, forensics, environmental monitoring as well as search, reconnaissance and mapping. While marine robots remain relatively expensive, new types of low-cost solutions are starting to enter the market. As these become cheaper and more sophisticated in terms of their autonomous capabilities, I am hopeful that they will become increasingly useful and accessible to local communities around the world. Check out WeRobotics to learn about how appropriate robotics solutions can support local livelihoods.

Reverse Robotics: A Brief Thought Experiment

Imagine a world in which manually controlled technologies simply do not exist. The very thought of manual technologies is, in actual fact, hardly conceivable let alone comprehensible. Instead, this seemingly alien world is seamlessly powered by intelligent and autonomous robotics systems. Lets call this world Planet AI.

PlanetAI

Planet AI’s version of airplanes, cars, trains and ships are completely unmanned. That is, they are fully autonomous—a silent symphony of large and small robots waltzing around with no conductor in sight. On one fateful night, a young PhD student awakens in a sweat unable to breathe, momentarily. The nightmare: all the swirling robots of Planet AI were no longer autonomous. Each of them had to be told exactly what to do by the Planet’s inhabitants. Madness.

She couldn’t go back to sleep. The thought of having to tell her robotics transport unit (RTU) in the morning how to get from her studio to the university gave her a panic attack. She would inevitably get lost or worse yet crash, maybe even hurt someone. She’d need weeks of practice to manually control her RTU. And even if she could somehow master manual steering, she wouldn’t be able to steer and work on her dissertation at the same time during the 36-minute drive. What’s more, that drive would easily become a 100-minute drive since there’s no way she would manually steer the RTU at 100 kilometers an hour—the standard autonomous speed of RTUs; more like 30km/h.

And what about the other eight billion inhabits of Planet AI? The thought of having billions of manually controlled RTUs flying, driving & swimming through the massive metropolis of New AI was surely the ultimate horror story. Indeed, civilization would inevitably come to an end. Millions would die in horrific RTU collisions. Transportation would slow to a crawl before collapsing. And the many billions of hours spent working, resting or playing in automated RTU’s every day would quickly evaporate into billions of hours of total stress and anxiety. The Planet’s Global GDP would free fall. RTU’s carrying essential cargo automatically from one side of the planet to the other would need to be steered manually. Where would those millions of jobs require such extensive manual labor come from? Who in their right mind would even want to take such a dangerous and dull assignment? Who would provide the training and certification? And who in the world would be able to pay for all the salaries anyway?

At this point, the PhD student was on her feet. “Call RTU,” she instructed her personal AI assistant. An RTU swung by while she as putting on her shoes on. Good, so far so good, she told herself. She got in slowly and carefully, studying the RTU’s behavior suspiciously. No, she thought to herself, nothing out of the ordinary here either. It was just a bad dream. The RTU’s soft purring power source put her at ease, she had always enjoyed the RTU’s calming sound. For the first time since she awoke from her horrible nightmare, she started to breathe more easily. She took an extra deep and long breath.

starfleet

The RTU was already waltzing with ease at 100km per hour through the metropolis, the speed barely noticeable from inside the cocoon. Forty-six, forty-seven and forty-eight; she was counting the number of other RTU’s that were speeding right alongside her’s, below and above as well. She arrived on campus in 35 minutes and 48 seconds—exactly the time it had taken the RTU during her 372 earlier rides. She breathed a deep sigh of relief and said “Home Please.” It was just past 3am and she definitely needed more sleep.

She thought of her fiancée on the way home. What would she think about her crazy nightmare given her work in the humanitarian space? Oh no. Her heart began to race again. Just imagine the impact that manually steered RTUs would have on humanitarian efforts. Talk about a total horror story. Life-saving aid, essential medicines, food, water, shelter; each of these would have to be trans-ported manually to disaster-affected communities. The logistics would be near impossible to manage manually. Everything would grind and collapse to a halt. Damage assessments would have to be carried manually as well, by somehow steering hundreds of robotics data units (RDU’s) to collect data on affected areas. Goodness, it would take days if not weeks to assess disaster damage. Those in need would be left stranded. “Call Fiancée,” she instructed, shivering at the thought of her fiancée having to carry out her important life-saving relief work entirely manually.


The point of this story and thought experiment? While some on Planet Earth may find the notion of autonomous robotics system insane and worry about accidents, it is worth noting that a future world like Planet AI would feel exactly the same way with respect to our manually controlled technologies. Over 80% of airplane accidents are due to human pilot error and 90% of car accidents are the result of human driver error. Our PhD student on Planet AI would describe our use of manually controlled technologies a suicidal, not to mention a massive waste of precious human time.

Screen Shot 2016-07-07 at 11.11.08 AM

An average person in the US spends 101 minutes per day driving (which totals to more than 4 years in their life time). There are 214 million licensed car drivers in the US. This means that over 360 million hours of human time in the US alone is spent manually steering a car from point A to point B every day. This results in more than 30,000 people killed per year. And again, that’s just for the US. There are over 1 billion manually controlled motor vehicles on Earth. Imagine what we could achieve with an additional billion hours every day if we had Planet AI’s autonomous systems to free up this massive cognitive surplus. And lets not forget the devastating environmental impact of individually-owned, manually controlled vehicles.

If you had the choice, would you prefer to live on Earth or on Planet AI if everything else were held equal?

Humanitarian Robotics: The $15 Billion Question?

The International Community spends around $25 Billion per year to provide life saving assistance to people devastated by wars and natural disasters. According to the United Nations, this is $15 Billion short of what is urgently needed; that’s $15 Billion short every year. So how do we double the impact of humanitarian efforts and do so at half the cost?

wp1

Perhaps one way to deal with this stunning 40% gap in funding is to scale the positive impact of the aid industry. How? By radically increasing the efficiency (time-savings) and productivity (cost-savings) of humanitarian efforts. This is where Artificial Intelligence (AI) and Autonomous Robotics come in. The World Economic Forum refers to this powerful new combination as the 4th Industrial Revolution. Amazon, Facebook, Google and other Top 100 Fortune companies are powering this revolution with billions of dollars in R&D. So whether we like it or not, the robotics arms race will impact the humanitarian industry just like it is impacting other industries: through radical gains in efficiency & productivity.

Take Amazon, for example. The company uses some 30,000 Kiva robots in its warehouses across the globe (pictured below). These ground-based, terrestrial robotics solutions have already reduced Amazon’s operating expenses by no less than 20%. And each new warehouse that integrates these self-driving robots will save the company around $22 million in fulfillment expenses alone. According to Deutsche Bank, “Bringing the Kivas to the 100 or so distribution centers that still haven’t implemented the tech would save Amazon a further $2.5 billion.” As is well known, the company is also experimenting with aerial robotics (drones). A recent study by PwC (PDF) notes that “the labor costs and services that can be replaced by the use of these devices account for about $127 billion today, and that the main sectors that will be affected are infrastructure, agriculture, and transportation.” Meanwhile, Walmart and others are finally starting to enter the robotics arms race. The former is using ground-based robots to ship apparel and is actively exploring the use of aerial robotics to “photograph ware-house shelves as part of an effort to reduce the time it takes to catalogue inventory.”

Amazon Robotics

What makes this new industrial revolution different from those that preceded it is the fundamental shift from manually controlled technologies—a world we’re all very familiar with—to a world powered by increasingly intelligent and autonomous systems—an entirely different kind of world. One might describe this as a shift towards extreme automation. And whether extreme automation powers aerial robotics, terrestrial robotics or maritime robots (pictured below) is besides the point. The disruption here is the one-way shift towards increasingly intelligent and autonomous systems.

All_Robotics

Why does this fundamental shift matter to those of us working in humanitarian aid? For at least two reasons: the collection of humanitarian information and the transportation of humanitarian cargo. Whether we like it or not, the rise of increasingly autonomous systems will impact both the way we collect data and transport cargo by making these processes faster, safer and more cost-effective. Naturally, this won’t happen overnight: disruption is a process.

Humanitarian organizations cannot stop the 4th Industrial Revolution. But they can apply their humanitarian principles and ideals to inform how autonomous robotics are used in humanitarian contexts. Take the importance of localizing aid, for example, a priority that gained unanimous support at the recent World Humanitarian Summit. If we apply this priority to humanitarian robotics, the question becomes: how can access to appropriate robotics solutions be localized so that local partners can double the positive impact of their own humanitarian efforts? In other words, how do we democratize the 4th Industrial Revolution? Doing so may be an important step towards closing the $15 billion gap. It could render the humanitarian industry more efficient and productive while localizing aid and creating local jobs in new industries.

This is What Happens When You Send Flying Robots to Nepal

In September 2015, we were invited by our partner Kathmandu University to provide them and other key stakeholders with professional hands-on training to help them scale the positive impact of their humanitarian efforts following the devastating earthquakes. More specifically, our partners were looking to get trained on how to use aerial robotics solutions (drones) safely and effectively to support their disaster risk reduction and early recovery efforts. So we co-created Kathmandu Flying Labs to ensure the long-term sustainability of our capacity building efforts. Kathmandu Flying Labs is kindly hosted by our lead partner, Kathmandu University (KU). This is already well known. What is hardly known, however, is what happened after we left the country.

Screen Shot 2015-11-02 at 5.17.58 PM

Our Flying Labs are local innovation labs used to transfer both relevant skills and appropriate robotics solutions sustainably to outstanding local partners who need these the most. The co-creation of these Flying Labs include both joint training and applied projects customized to meet the specific needs & priorities of our local partners. In Nepal, we provided both KU and Kathmandu Living Labs (KLL) with the professional hands-on training they requested. What’s more, thanks to our Technology Partner DJI, we were able to transfer 10 DJI Phantoms (aerial robotics solutions) to our Nepali partners (6 to KU and 4 to KLL). In addition, thanks to another Technology Partner, Pix4D, we provided both KU and KLL with free licenses of the Pix4D software and relevant training so they could easily process and analyze the imagery they captured using their DJI platforms. Finally, we carried out joint aerial surveys of Panga, one of the towns hardest-hit by the 2015 Earthquake. Joint projects are an integral element of our capacity building efforts. These projects serve to reinforce the training and enable our local partners to create immediate added value using aerial robotics. This important phase of Kathmandu Flying Labs is already well documented.

WP15

What is less known, however, is what KU did with the technology and software after we left Nepal. Indeed, the results of this next phase of the Flying Labs process (during which we provide remote support as needed) has not been shared widely, until now. KU’s first order of business was to actually finish the joint project we had started with them in Panga. It turns out that our original aerial surveys there were actually incomplete, as denoted by the red circle below.

Map_Before

But because we had taken the time to train our partners and transfer both our skills and the robotics technologies, the outstanding team at KU’s School of Engineering returned to Panga to get the job done without needing any further assistance from us at WeRobotics. They filled the gap:

Map_After

The KU team didn’t stop there. They carried out a detailed aerial survey of a nearby hospital to create the 3D model below (at the hospital’s request). They also created detailed 3D models of the university and a nearby temple that had been partially damaged by the 2015 earthquakes. Furthermore, they carried out additional disaster damage assessments in Manekharka and Sindhupalchowk, again entirely on their own.

Yesterday, KU kindly told us about their collaboration with the World Wildlife Fund (WWF). Together, they are conducting a study to determine the ecological flow of Kaligandaki river, one of the largest rivers in Nepal. According to KU, the river’s ecosystem is particularly “complex as it includes aquatic invertebrates, flora, vertebrates, hydrology, geo-morphology, hydraulics, sociology-cultural and livelihood aspects.” The Associate Dean at KU’s School of Engineering wrote “We are deploying both traditional and modern technology to get the information from ground including UAVs. In this case we are using the DJI Phantoms,” which “reduced largely our field investigation time. The results are interesting and promising.” I look forward to sharing these results in a future blog post.

kali-gandaki-river

Lastly, KU’s Engineering Department has integrated the use of the robotics platforms directly into their courses, enabling Geomatics Engineering students to use the robots as part of their end-of-semester projects. In sum, KU has done truly outstanding work following our capacity building efforts and deserve extensive praise. (Alas, it seems that KLL has made little to no use of the aerial technologies or the software since our training 10 months ago).

Several months after the training in Nepal, we were approached by a British company that needed aerial surveys of specific areas for a project that the Nepal Government had contracted them to carry out. So they wanted to hire us for this project. We proposed instead that they hire our partners at Kathmandu Flying Labs since the latter are more than capable to carry out the surveys themselves. In other words, we actively drive business opportunities to Flying Labs partners. Helping to create local jobs and local businesses around robotics as a service is one of our key goals and the final phase of the Flying Labs framework.

So when we heard last week that USAID’s Global Development Lab was looking to hire a foreign company to carry out aerial surveys for a food security project in Nepal, we jumped on a call with USAID to let them know about the good work carried out by Kathmandu Flying Labs. We clearly communicated to our USAID colleagues that there are perfectly qualified Nepali pilots who can carry out the same aerial surveys. USAID’s Development Lab will be meeting with Kathmandu Flying Labs during their next visit in September.

thumb_IMG_4591_1024

On a related note, one of the participants who we trained in September was hired soon after by Build Change to support the organization’s shelter programs by producing Digital Surface Models (DSMs) from aerial images captured using DJI platforms. More recently, we heard from another student who emailed us with the following: “I had an opportunity to participate in the Humanitarian UAV Training mission in Nepal. It’s because of this training I was able learn how to fly drones and now I can conduct aerial Survey on my own with any hardware.  I would like to thank you and your team for the knowledge transfer sessions.”

This same student (who graduated from KU) added: “The workshop that your team did last time gave us the opportunity to learn how to fly and now we are handling some professional works along with major research. My question to you is ‘How can young graduates from developing countries like ours strengthen their capacity and keep up with their passion on working with technology like UAVs […]? The immediate concern for a graduate in Nepal is a simple job where he can make some money for him and prove to his family that he has done something in return for all the investments they have been doing upon him […]’.

KU campus sign

This is one of several reasons why our approach at WeRobotics is not limited to scaling the positive impact of local humanitarian, development, environmental and public health projects. Our demand-driven Flying Labs model goes the extra (aeronautical) mile to deliberately create local jobs and businesses. Our Flying Labs partners want to make money off the skills and technologies they gain from WeRobotics. They want to take advantage of the new career opportunities afforded by these new AI-powered robotics solutions. And they want their efforts to be sustainable.

In Nepal, we are now interviewing the KU graduate who posed the question above because we’re looking to hire an outstanding and passionate Coordinator for Kathmandu Flying Labs. Indeed, there is much work to be done as we are returning to Nepal in coming months for three reasons: 1) Our local partners have asked us to provide them with the technology and training they need to carry out large scale mapping efforts using long-distance fixed-wing platforms; 2) A new local partner needs to create very high-resolution topographical maps of large priority areas for disaster risk reduction and planning efforts, which requires the use of a fixed-wing platform; 3) We need to meet with KU’s Business Incubation Center to explore partnership opportunities since we are keen to help incubate local businesses that offer robotics as a service in Nepal.

How to Democratize Humanitarian Robotics

Our world is experiencing an unprecedented shift from manually controlled technologies to increasingly intelligent and autonomous systems powered by artificial intelligence (AI). I believe that this radical shift in both efficiency and productivity can have significant positive social impact when it is channeled responsibly, locally and sustainably.

WeRobotics_Logo_New

This is why my team and I founded WeRobotics, the only organization fully dedicated to accelerating and scaling the positive impact of humanitarian, development and environmental projects through the appropriate use of AI-powered robotics solutions. I’m thrilled to announce that the prestigious Rockefeller Foundation shares our vision—indeed, the Foundation has just awarded WeRobotics a start-up grant to take Humanitarian Robotics to the next level. We’re excited to leverage the positive power of robotics to help build a more resilient world in line with Rockefeller’s important vision.

Print

Aerial Robotics (drones/UAVs) represent the first wave of robotics to impact humanitarian sectors by disrupting traditional modes of data collection and cargo delivery. Both timely data and the capacity to act on this data are integral to aid, development and environmental projects. This is why we are co-creating and co-hosting global network of “Flying Labs”; to transfer appropriate aerial robotics solutions and relevant skills to outstanding local partners in developing countries who need these the most.

Our local innovation labs also present unique opportunities for our Technology Partners—robotics companies and institutes. Indeed, our growing network of Flying Labs offer a multitude of geographical, environmental and social conditions for ethical social good projects and responsible field-testing; from high-altitude glaciers and remote archipelagos experiencing rapid climate change to dense urban environments in the tropics subject to intense flooding and endangered ecosystems facing cascading environmental risks.

The Labs also provide our Technology Partners with direct access to local knowledge, talent and markets, and in turn provide local companies and entrepreneurs with facilitated access to novel robotics solutions. In the process, our local partners become experts in different aspects of robotics, enabling them to become service providers and drive new growth through local start-up’s and companies. The Labs thus seek to offer robotics-as-a-service across multiple local sectors. As such, the Labs follow a demand-driven social entrepreneurship model designed to catalyze local businesses while nurturing learning and innovation.

Of course, there’s more to robotics than just aerial robotics. This is why we’re also exploring the use of AI-powered terrestrial and maritime robotics for data collection and cargo delivery. We’ll add these solutions to our portfolio as they become more accessible in the future. In the meantime, sincerest thanks to the Rockefeller Foundation for their trust and invaluable support. Big thanks also to our outstanding Board of Directors and to key colleagues for their essential feed-back and guidance.

Humanitarian Cargo Delivery via Aerial Robotics is Not Science Fiction (Updated)

I had the opportunity to visit Zipline’s field-testing site in San Francisco last year after the company participated in an Experts Meeting on Humanitarian UAVs (Aerial Robotics) that I co-organized at MIT. The company has finally just gone public about their good work in Rwanda, so I’m at last able to blog about it on iRevolutions. When I write “finally”, this is not meant to be a complaint; in fact, one aspect that really drew me to Zipline in the first place is the team’s genuine down-to-earth, no-hype mantra. So, I use the word finally since I now finally have public evidence to backup many conversations I’ve had with humanitarian partners on the topic of cargo delivery via aerial robotics.

Zip Delivery

As I had signed an NDA, I was (and still am) only allowed to discuss information that is public, which was basically nothing until today. So below is a summary of what is at last publicly known about Zipline’s pioneering aerial robotics efforts in Rwanda. I’ve also added videos at the end.

Zipic

  • Zipline’s Mission: to deliver critical medical products to health centers and hospitals that are either difficult or impossible to reach via traditional modes of transportation
  • Zipline Fleet: 15 aerial robotics platforms (UAVs) in Rwanda.
  • Aerial Robotics platform: Fixed-wing.
  • Weight of each platform: 10-kg.
  • Power: Battery-operated twin-electric motors.
  • Payload capacity: up to 1.5kg.
  • Cargo: Blood and essential medicines (small vials) to begin with. Eventually cargo will extend to lifesaving vaccines, treatments for HIV/AIDS, malaria, tuberculosis, etc.
  • Range: Up to 120 km.
  • Flight Plans: Pre-programmed and monitored on the ground via tablets. Individual plans are stored on SIM cards.

gallery5

 

 

 

 

 

 

 

  • Flight Navigation: GPS using the country’s cellular network.
  • Launch Mechanism: Via catapult.
  • Maximum Speed: Around 100 km/hour.
  • Landings: Zipline’s aerial robot does not require a runway.
  • Delivery Mechanism: Fully autonomous, low altitude drop via simple paper parachute. Onboard computers determine appropriate parameters (taking into account winds, etc) to ensure that the cargo accurately lands on it’s dedicated delivery site called a “mailbox”.
  • Delivery Sites: Dedicated drop sites at 21 health facilities that can carry out blood transfusions. These cover more than half of Rwanda.

Screen Shot 2016-04-05 at 4.22.02 PM

 

 

 

 

 

 

 

 

 

  • Takeoff Sites: Modified shipping containers located next to existing medical warehouses.
  • Delivery Time: Each cargo is delivered within 1 hour. The aerial robot takes about 1/2 hour reach a delivery site.
  • Flight Frequency: Eventually up to 150 flights per day.
  • Weather: Fixed-wings can operate in ~50km/hour winds.
  • Regulatory Approval: Direct agreements already secured with the Government of Rwanda and country’s Civil Aviation Authority.

Sources:

Think Global, Fly Local: The Future of Aerial Robotics for Disaster Response

First responders during disasters are not the United Nations or the Red Cross. The real first responders, by definition, are the local communities; always have been, always will be. So the question is: can robotics empower local communities to respond and recover both faster and better? I believe the answer is Yes.

But lets look at the alternative. As we’ve seen from recent disasters, the majority of teams that deploy with aerial robotics (UAVs) do so from the US, Europe and Australia. The mobilization costs involved in flying a professional team across the world—not to mention their robotics equipment—is not insignificant. And this doesn’t even include the hotel costs for a multi-person team over the course of a mission. When you factor in these costs on top of the consulting fees owed to professional international robotics teams, then of course the use of aerial robotics versus space robotics (satellites) becomes harder to justify.

There is also an important time factor. The time it takes for international teams to obtain the necessary export/import permits and customs clearance can be highly unpredictable. More than one international UAV team that (self) deployed to Nepal after the tragic 2015 Earthquake had their robotics platforms held up in customs for days. And of course there’s the question of getting regulatory approval for robotics flights. Lastly, international teams (especially companies and start-up’s) may have little to no prior experience working in the country they’re deploying to; they may not know the culture or speak the language. This too creates friction and can slow down a humanitarian robotics mission.

WeR PM Blog pic 2

What if you had fully trained teams on the ground already? Not an international team, but a local expert robotics team that obviously speaks the local language, understands local customs and already has a relationship with the country’s Civil Aviation Authority. A local team does not need to waste time with export/import permits or customs clearance; doesn’t need expensive international flights or weeks’ worth of hotel accommodations. They’re on site, and ready to deploy at a moment’s notice. Not only would this response be faster, it would be orders of magnitudes cheaper and more sustainable to carry through to the recovery and reconstruction phase.

In sum, we need to co-create local Flying Labs with local partners including universities, NGOs, companies and government partners. Not only would these Labs be far more agile and rapid vis-a-vis disaster response efforts, they would also be far more sustainable and their impact more scalable than deploying international robotics teams. This is one of the main reasons why my team and I at WeRobotics are looking to co-create and connect a number of Flying Labs in disaster prone countries across Asia, Africa and Latin America. With these Flying Labs in place, the cost of rapidly acquiring high quality aerial imagery will fall significantly. Think Global, Fly Local.

Aerial Robotics for Payload Delivery in Developing Countries: Open Questions

Should developing countries seek to manufacture their own robotics solutions in order to establish payload delivery services? What business models make the most sense to sustain these services? Do decision-support tools already exist to determine which delivery routes are best served by aerial robots (drones) rather than traditional systems (such as motorbikes)? And what mechanisms should be in place to ensure that the impact of robotics solutions on local employment is one of net job creation rather than job loss?

Screen Shot 2016-03-17 at 3.34.52 PM

There are some of the questions I’ve been thinking about and discussing with various colleagues over the past year vis-a-vis humanitarian applications. So let me take the first 2 questions and explore these further here. I’ll plan on writing a follow up post in the near future to address the other two questions.

First, should developing countries take advantage of commercial solutions that already exist to build their robotics delivery infrastructure? Or should they seek instead to manufacture these robotics platforms locally instead? The way I see it, this does not have to be an either/or situation. Developing countries can both benefit from the robust robotics technologies that already exist and take steps to manufacture their own solutions over time.

This is not a hypothetical debate. I’ve spent the past few months going back and forth with a government official in a developing country about this very question. The official is not interested in leveraging existing commercial solutions from the West. As he rightly notes, there are many bright engineers in-country who are able and willing to build these robotics solutions locally.

Here’s the rub, however, this official has no idea just how much work, time and money is needed to develop robust, reliable and safe robotics solutions. In fact, many companies in both Europe and the US have themselves completely under-estimated just how technically challenging (and very expensive) it is to develop reliable aerial robotics solutions to delivery payloads. This endeavor easily takes years and millions of dollars to have a shot at success. It is far from trivial.

The government official in question wants his country’s engineers to build these solutions locally in order to transport essential medicines and vaccines between health clinics and remote villages. Providing this service is relatively urgent because existing delivery mechanisms are slow, unreliable and at times danger-ous. So this official will have to raise a substantial amount of funds to pay local engineers to build home-grown robotics solutions and iterate accordingly. This could take years (with absolutely no guarantee of success mind you).

On the other hand, this same official could decide to welcome the use of existing commercial solutions as part of field-tests in-country. The funding for this would not have to come from the government and the platforms could be field-tested as early as this summer. Not only would this provide local engineers with the ability to learn from the tests and gain important engineering insights, they could also be hired to actually operate the cargo delivery services over the long-term, thus gaining the skills to maintain and fix the platforms. Learning by doing would give these engineers practical training that they could use to build their own home-grown solutions.

One could be even more provocative: Why invest so much time and effort in local manufacturing when in-country engineers and entrepreneurs could simply use commercial solutions that already exist to make money sooner rather than later by providing robotics as a service? We’ve seen, historically, the transition from manufacturing to service-based economies. There’s plenty of profit to be made from the latter with a lot less start-up time and capital required. And again, one strategy does not preclude the other, so why forgo both early training and business opportunities when these same opportunities could help develop and fund the local robotics industry?

Admittedly, I’m somewhat surprised by the official’s zero tolerance for the use of foreign commercial technology to improve his country’s public health services; that same official is using computers, phones, cars, televisions, etc., that are certainly not made in-country. He does not have a background in robotics, so perhaps he assumes that building robust robotics solutions is relatively easy. Simply perusing the past 2 years of crowdfunded aerial robotics projects will clearly demonstrate that most have resulted in complete failure despite raising millions of dollars. That robotics graveyard keeps growing.

But I fully respect the government official’s position even if I disagree with it. In my most recent exchange with said official, I politely re-iterated that one strategy (local manufacturing) does not preclude the other (local business opportunities around robotics as service using foreign commercial solutions). Surely, the country in question can both leverage foreign technology while also building a local manufacturing base to produce their own robotics solutions.

show-me-the-money

Second, on business models, which models can provide sustainability by having aerial delivery services be profitable earlier rather than later? I was recently speaking to a good colleague of mine who works for a very well-respected humanitarian group about their plans to pilot the use of aerial robotics for the delivery of essential medicines. When I asked him about his organization’s business model for sustaining these delivery services, he simply said there was no model, that his humanitarian organization would simply foot the bill.

Surely we can do better. Just think how absurd it would be for a humanitarian organization to pay for their own 50 kilometer paved road to transport essential medicines by truck and decide not to recoup those major costs. You’ve paid for a perfectly good road that only gets used a few times a day by your organization. But 80% of the time there is no one else on that road. That would be absurd. Humanitarians who seek to embark on robotics delivery projects should really take the time to understand local demand for transportation services and use-cases to explore strategies to recoup part of their investments in building the aerial robotics infrastructure.

Surely remote communities who are disconnected from health services are also disconnected from access to other commodities. Of course, these local villages may not benefit from high levels of income; but I’m not suggesting that we look for high margins of return. Point is, if you’ve already purchased an aerial robot (drone) and it spends 80% of its time on the ground, then talk about a missed opportunity. Take commercial aviation as an analogy. Airlines do not make money when their planes are parked at the gate. They make money when said planes fly from point A to point B. The more they fly, the more they transport, the more they profit. So pray tell what is the point of investing in aerial robots only to have them spend most of their lives on the ground? Why not “charter” these robots for other purposes when they’re not busy flying medicines?

The fixed costs are the biggest hurdle with respect to aerial robotics, not the variable costs. Autonomous flights themselves cost virtually nothing; only 1-2 person’s time to operate the robot and swap batteries & payloads. Just like their big sisters (manually piloted aircraft), aerial robots should be spending the bulk of their time in the sky. So humanitarian organizations really ought to be thinking earlier rather than later about how to recoup part of their fixed costs by offering to transport other high-demand goods. For example, by allowing local businesses to use existing robotics aircraft and routes to transport top-up cards or SIM cards for mobile phones. What is the weight of 500 top-up or SIM cards? Around 0.5kg, which is easily transportable via aerial robot. Better yet, identify perishable commodities with a short shelf-life and allow business to fly those via aerial robot.

The business model that I’m most interested in at the moment is a “Per Flight Savings” model. One reason to introduce robotics solutions is to save on costs—variable costs in particular. Lest say that the variable cost of operating robotics solutions is 20% lower than the costs of traditional delivery mechanisms (per flight versus per drive, for example). You offer the client a 10% cost saving and pocket the other 10% as revenue. Over time, with sufficient flights (transactions) and growing demand, you break even and start to create a profit. I realize this is a hugely simplistic description; but this need not be unnecessarily complicated either.  The key will obviously be the level of demand for these transactions.

The way I see it, regardless of the business model, there will be a huge first-mover advantage in developing countries given the massive barriers to entry. Said barriers are primarily due to regulatory issues and air traffic management challenges. For example, once a robotics company manages to get regulatory approval and specific flight permissions for designated delivery routes to supply essential medicines, a second company that seeks to enter the market may face even greater barriers. Why? Because managing aerial robotics platforms from one company and segregating that airspace from manned aircraft can already be a challenge (not to mention a source of concern for Civil Aviation Authorities).

So adding new (and different types of) robots from a second company requires new communication protocols between the different robotics platforms operated by the 2 different companies. In sum, the challenges become more complex more quickly as new competitors seek entry. And for an Aviation Authority that may already be weary of flying robots, the proposal of adding a second fleet from a different company in order to increase competition around aerial deliveries may take said Authority some time to digest. Of course, if these companies can each operate in completely different parts of a given country, then technically this is an easier challenge to manage (and less anxiety provoking for authorities).

But said barriers do not only include technical (though surmountable) barriers. They also include identifying those (few?) use-cases that clearly make the most business sense to recoup one’s investments earlier rather than later given the very high start-up fixed costs associated with developing robotics platforms. Identifying these business cases is typically not something that’s easily done remotely. A considerable amount of time and effort must be spent on-site to identify and meet possible stakeholders in order to brainstorm and discover key use-cases. And my sense is that aerial robots often need to be designed to meet a specific use-case. So even when new use-cases are identified, there may still be the need for Research and Development (R&D) to modify a given robotics platform so it can most efficiently cater to new use-cases.

There are other business models worth thinking through for related services, such as those around the provision of battery-charging services, for example. The group Mobisol has installed solar home systems on the roofs of over 40,000 households in Rwanda and Tanzania to tackle the challenge of energy poverty. Mobisol claims to already cover much of Tanzania with solar panels that are no more than 5 kilometers apart. This could enabling aerial robots (UAVs) to hop from recharging station to recharging station, an opportunity that Mobisol is already actively exploring. Practical challenges aside, this network of charging stations could lead to an interesting business model around the provision of aerial robotics services.

As the astute reader will have gathered, much of the above is simply a written transcript me thinking out load. So I’d very much welcome some intellectual company here along with constructive feedback. What am I missing? Is my logic sound? What else should I be taking into account?

UN Crisis Map of Fiji Uses Aerial Imagery (Updated)

Update 1: The Crisis Map below was produced pro bono by Tonkin + Taylor so they should be credited accordingly.

Update 2: On my analysis of Ovalau below, I’ve been in touch with the excellent team at Tonkin & Taylor. It would seem that the few images I randomly sampled were outliers since the majority of the images taken around Ovalau reportedly show damage, hence the reason for Tonkin & Taylor color-coding the island red. Per the team’s explanation: “[We] have gone through 40 or so photographs of Ovalau. The area is marked red because the majority of photographs meet the definition of severe, i.e.,: 1) More than 50% of all buildings sustaining partial loss of amenity/roof; and 2) More than 20% of damaged buildings with substantial loss of amenity/roof.” Big thanks to the team for their generous time and for their good work on this crisis map.


Fiji Crisis Map

Fiji recently experienced the strongest tropical cyclone in its history. Named Cyclone Winston, the Category 5 Cyclone unleashed 285km/h (180 mph) winds. Total damage is estimated at close to half-a-billion US dollars. Approximately 80% of the country’s population lost power; 40,000 people required immediate assistance; some 24,000 homes were damaged or destroyed leaving around 120,000 people in need of shelter assistance; 43 people tragically lost their lives.

As a World Bank’s consultant on UAVs (aerial robotics), I was asked to start making preparations for the possible deployment of a UAV team to Fiji should an official request be made. I’ve therefore been in close contact with the Civil Aviation Authority of Fiji; and several professional and certified UAV teams as well. The purpose of this humanitarian robotics mission—if requested and authorized by relevant authorities—would be to assess disaster damage in support of the Post Disaster Needs Assessment (PDNA) process. I supported a similar effort last year in neighboring Vanuatu after Cyclone Pam.

World Bank colleagues are currently looking into selecting priority sites for the possible aerial surveys using a sampling method that would make said sites representative of the disaster’s overall impact. This is an approach that we were unable to take in Vanuatu following Cyclone Pam due to the lack of information. As part of this survey sampling effort, I came across the United Nations Office for the Coordination of Humanitarian Affairs (UN/OCHA) crisis map below, which depicts areas of disaster damage.

Fiji Crisis Map 2

I was immediately struck by the fact that the main dataset used to assess the damage depicted on this map comes from (declassified) aerial imagery provided by the Royal New Zealand Air Force (RNZAF). Several hundred high-resolution oblique aerial images populate the crisis map along with dozens of ground-based photographs like the ones below. Note that the positional accuracy of the aerial images is +/- 500m (meaning not particularly accurate).

Fiji_2

Fiji_!

I reached out to OCHA colleagues in Fiji who confirmed that they were using the crisis map as one source of information to get a rough idea about which areas were the most affected.  What makes this data useful, according to OCHA, is that it had good coverage over a large area. In contrast, satellite imagery could only provide small snapshots of random villages which were not as useful for trying to understand the scale and scope of a disasters. The limited value added of satellite imagery was reportedly due to cloud cover, which is typical after atmospheric hazards like Cyclones.

Below is the damage assessment methodology used vis-a-vis the interpret the aerial imagery. Note that this preliminary assessment was not carried out by the UN but rather an independent company.

Fiji Crisis Map 3

  • Severe Building Damage (Red): More than 50% of all buildings sustaining partial loss of amenity/roof or more than 20% of damaged buildings with substantial loss of amenity/roof.
  • Moderate Building Damage (Orange): Damage generally exceeding minor [damage] with up to 50% of all buildings sustaining partial loss of amenity/roof and up to 20% of damaged buildings with substantial loss of amenity/roof.
  • Minor Building Damage (Blue):  Up to 5% of all buildings with partial loss of amenity/roof or up to 1% of damaged buildings with substantial loss of amenity/roof.

The Fiji Crisis Map includes an important note: The primary objective of this preliminary assessment was to communicate rapid high-level building damage trends on a regional scale. This assessment has been undertaken on a regional scale (generally exceeding 100 km2) and thus may not accurately reflect local variation in damage. I wish more crisis maps provided qualifiers like the above. That said, while I haven’t had the time to review the hundreds of aerial images on the crisis map to personally assess the level of damage depicted in each, I was struck by the assessment of Ovalau, which I selected at random.

Fiji Crisis Map 4

As you’ll note, the entire island is color coded as severe damage. But I selected several aerial images at random and none showed severe building damage. The images I reviewed are included below.

Ovalau0 Ovalau1 Ovalau2 Ovalau3

This last one may seem like there is disaster damage but a closer inspection by zooming in reveals that the vast majority of buildings are largely intact.

Ovalau5

I shall investigate this further to better understand the possible discrepancy. In any event, I’m particularly pleased to see the UN (and others) make use of aerial imagery in their disaster damage assessment efforts. I’d also like to see the use of aerial robotics for the collection of very high resolution, orthorectified aerial imagery. But using these robotics solutions to their full potential for damage assessment purposes requires regulatory approval and robust coordination mechanisms. Both are absolutely possible as we demonstrated in neighboring Vanuatu last year.

Aerial Robotics for Search & Rescue: State of the Art?

WeRobotics is co-creating a global network of labs to transfer robotics solutions to those who need them most. These “Flying Labs” take on different local flavors based on the needs and priorities of local partners. Santiago Flying Labs is one of the labs under consideration. Our local partners in Chile are interested in the application of robotics for disaster preparedness and Search & Rescue (SaR) operations. So what is the state of the art in rescue robotics?

WP0

One answer may lie several thousand miles away in Lushan, China, which experienced a 7.0 magnitude earthquake in 2013. The mountainous area made it near impossible for the Chinese International Search and Rescue Team (CISAR) to implement a rapid search and post-seismic evaluation. So State Key Robotics Lab at Shenyang Institute of Automation offered aerial support to CISAR. They used their aerial robot (UAV) to automatically and accurately detect collapsed buildings for ground rescue guidance. This saved the SaR teams considerable time. Here’s how.

WP5

A quicker SaR response leads to a higher survival rate. The survival rate is around 90% within the first 30 minutes but closer to 20% by day four. “In traditional search methods, ground rescuers are distributed to all possible places, which is time consuming and inefficient.” An aerial inspection of the disaster damage can help accelerate the ground search for survivors by prioritizing which areas to search first.

WP1

State Key Labs used a ServoHeli aerial robot to capture live video footage of the damaged region. And this is where it gets interesting. “Because the remains of a collapsed building often fall onto the ground in arbitrary directions, their shapes will exhibit random gradients without particular orientations. Thus, in successive aerial images, the random shape of a collapsed building will lead to particular motion features that can be used to discriminate collapsed from non-collapsed buildings.”

WP2

These distinct motion features can be quantified using a histogram of oriented gradient (HOG) as depicted here (click to enlarge):

WP4
As is clearly evident from the histograms, the “HOG variation of a normal building will be much larger than that of a collapsed one.” The team at State Key Labs had already employed this technique to train and test their automated feature-detection algorithm using aerial video footage from the 2010 Haiti Earthquake. Sample results of this are displayed below. Red rectangles denote where the algorithm was successful in identifying damage. Blue rectangles are false alarms while orange rectangles are missed detections.

Screen Shot 2016-02-07 at 2.55.14 AM

Screen Shot 2016-02-07 at 2.54.14 AM

Screen Shot 2016-02-07 at 2.55.57 AM

While the team achieved increasingly accurate detection rates for Haiti, the initial results for Lushan were not as robust. This was due to the fact that Lushan is far more rural than Port-au-Prince, which tripped up the algorithm. Eventually, the software achieved an accurate rate of 83.4% without any missed collapses, however. The use of aerial robotics and automated feature detection algorithms in Xinglong Village (9.5 sq. km) enabled CISAR to cut their search time in half. In sum, the team concluded that videos are more valuable for SaR operations than static images.

Screen Shot 2016-02-07 at 2.53.05 AM

Screen Shot 2016-02-07 at 2.52.26 AM

To learn more about this deployment, see the excellent write-up “Search and Rescue Rotary-Wing UAV and Its Application to the Lushan Ms 7.0 Earthquake” published in the Journal of Field Robotics. I wish all robotics deployments were this well documented. Another point that I find particularly noteworthy about this operation is that it was conducted three years ago already. In other words, real-time feature detection of disaster damage from live aerial video footage was already used operationally years ago.

WP3
What’s more, this paper published in 2002 (!) used computer vision to detect with a 90% accuracy rate collapsed buildings in aerial footage of the 1995 Kobe earthquake captured by television crews in helicopters. Perhaps in the near future we’ll have automated feature detection algorithms for disaster damage assessments running on live video footage from news channels and aerial robots. These could then be complemented by automated change-detection algorithms running on satellite imagery. In any event, the importance of applied research is clearly demonstrated by the Lushan deployments. This explains why WeRobotics
always aims to have local universities involved in Flying Labs.


Thanks to the ICARUS Team for pointing me to this deployment.