Category Archives: Digital Activism

Using Sound and Artificial Intelligence to Detect Human Rights Violations

Video continues to be a powerful way to capture human rights abuses around the world. Videos posted to social media can be used to hold perpetrators of gross violations accountable. But video footage poses a “Big Data” challenge to human rights organizations. Two billion smartphone users means almost as many video cameras. This leads to massive amounts of visual content of both suffering and wrong-doing during conflict zones. Reviewing these videos manually is a very labor intensive, time consuming, expensive and often traumatic task. So my colleague Jay Aronson at CMU has been exploring how artificial intelligence and in particular machine learning might solve this challenge.

blog1pic

As Jay and team rightly note in a recent publication (PDF), “the dissemination of conflict and human rights related video has vastly outpaced the ability of researchers to keep up with it – particularly when immediate political action or rapid humanitarian response is required.” The consequences of this are similar to what I’ve observed in humanitarian aid: At some point (which will vary from organization to organization), time and resource limitations will necessitate an end to the collection, archiving, and analysis of user generated content unless the process can be automated.” In sum, information overload can “prevent human rights researchers from uncovering widely dispersed events taking place over long periods of time or large geographic areas that amount to systematic human rights violations.”

jayblog1

To take on this Big Data challenge, Jay and team have developed a new machine learning-based audio processing system that “enables both synchronization of multiple audio-rich videos of the same event, and discovery of specific sounds (such as wind, screaming, gunshots, airplane noise, music, and explosions) at the frame level within a video.” The system basically “creates a unique “soundprint” for each video in a collection, synchronizes videos that are recorded at the same time and location based on the pattern of these signatures, and also enables these signatures to be used to locate specific sounds precisely within a video. The use of this tool for synchronization ultimately provides a multi-perspectival view of a specific event, enabling more efficient event reconstruction and analysis by investigators.”

Synchronizing image features is far more complex than synchronizing sound. “When an object is occluded, poorly illuminated, or not visually distinct from the background, it cannot always be detected by computer vision systems. Further, while computer vision can provide investigators with confirmation that a particular video was shot from a particular location based on the similarity of the background physical environment, it is less adept at synchronizing multiple videos over time because it cannot recognize that a video might be capturing the same event from different angles or distances. In both cases, audio sensors function better so long as the relevant videos include reasonably good audio.”

Ukrainian human rights practitioners working with families of protestors killed during the 2013-2014 Euromaidan Protests recently approached Jay and company to analyze videos from those events. They wanted to “ locate every video available in their collection of the moments before, during, and just after a specific set of killings. They wanted to extract information from these videos, including visual depictions of these killings, whether the protesters in question were an immediate and direct threat to the security forces, plus any other information that could be used to corroborate or refute other forms of evidence or testimony available for their cases.”

jayblog2

Their plan had originally been to manually synchronize more than 65 hours of video footage from 520 videos taken during the morning of February 20, 2014. But after working full-time over several months, they were only able to stitch together about 4 hours of the total video using visual and audio cues in the recording.” So Jay and team used their system to make sense of the footage. They were able to automatically synchronize over 4 hours of the footage. The figure above shows an example of video clips synchronized by the system.

Users can also “select a segment within the video containing the event they are interested in (for example, a series of explosions in a plaza), and search in other videos for a similar segment that shows similar looking buildings or persons, or that contains a similar sounding noise. A user may for example select a shooting scene with a significant series of gunshots, and may search for segments with a similar sounding series of gunshots. This method increases the chances for finding video scenes of an event displaying different angles of the scene or parallel events.”

Jay and team are quick to emphasize that their system “does not  eliminate human involvement in the process because machine learning systems provide probabilistic, not certain, results.” To be sure, “the synchronization of several videos is noisy and will likely include mistakes—this is precisely why human involvement in the process is crucial.”

I’ve been following Jay’s applied research for many years now and continue to be a fan of his approach given the overlap with my own work in the use of machine learning to make sense of the Big Data generated during major natural disasters. I wholeheartedly agree with Jay when he reflected during a recent call that the use of advanced techniques alone is not the answer. Effective cross-disciplinary collaboration between computer scientists and human rights (or humanitarian) practitioners is really hard but absolutely essential. This explains why I wrote this practical handbook on how to create effective collaboration and successful projects between computer scientists and humanitarian organizations.

Counter-Mapping the State with UAVs

Want a piece of Indonesia? The country’s government is busy implementing an “accelerated development program” in which “different provinces are assigned different development foci,” like “food and energy for Papua, palm oil processing for North Sumatra, mining for Central Kalimantan etc.” Critics describe this program as “a national, state-coordinated program of land grabs.” An important component of “this development plan is the commoditization of space by spatial planning,” which is “supposed to be open, transparent and participatory.” The reality is very different. “Maps are made by consultants and government offices favoring the interests of capital and local elites.” As a result, “concessions are given mostly without the consent (and often without the knowledge) of local communities.” These quotes are taken from a brilliant new study (PDF) written by Irendra Radjawali and Oliver Pye. The study describes the use of Unmanned Aerial Vehicles (UAVs) to “generate high-quality community controlled maps to challenge spatial planning from above,” which is “revolutionizing the counter-mapping movement in Indonesia.”

Screen Shot 2015-07-31 at 2.46.00 PM

“Challenging state power over maps and its categorization of land uses by counter-mapping indigenous and local claims to territory has developed into an important movement in Indonesia.” As the authors of the new study rightly note, “Mapping needs to be understood as a political process rather than a merely technical tool. Mapping is not only an act of how to produce maps, it is important to always ask who produces the maps, how people can access the maps and how the maps can be used for emancipatory purposes.” Counter-mapping is thus a political process as well. And this counter-mapping movement is now experimenting with “grassroots UAVs” (or community drones) to bolster their political actions.

Activists in Indonesia initially used their UAV to capture “high quality and high-resolution spatial data in areas where access was restricted by company security and police.” Where exactly did they get their UAV from? They built one from scratch: “Irendra Radjawali built the first drone without any former training, by using the Internet and the online forum. He also sourced much of the material second hand via ebay.” The advantage of this DIY approach is the relatively low costs involved. This UAV, coupled with a mapping camera, came to just over USD 500.

Screen Shot 2015-07-31 at 3.35.58 PM

Irendra and team subsequently few their UAVs over oil palm plantations where a company had taken lands from local communities who had no idea that their lands had been parceled off to said company. The team managed to fly their UAVs “at several places, capturing several community’s areas which have been grabbed by the company, including the customary area.” It is worth emphasizing that “community members very rarely have access to the spatial plan documents, and so could hardly ever actively participate in the spatial planning process. The opportunity to produce high-quality and precise maps is seen by community members as the chance to claim and to re-claim their lands.”

The team also flew over an area that was directly “affected by the expansion of large scale open mining for bauxite.” The water from the river became unsafe to drink; fishing grounds vanished; the nearby lake dried up. Local communities repeatedly protested the irreversible destruction of their ecosystem but this hasn’t stopped mining companies from expanding their activities. Irendra and team were able to take aerial photographs of the affected areas. One of the “high-quality and precise maps” that they were able to generate with these photos has since “been used as an evidence to disclose illegal mining company exploiting bauxites operating outside of their concession area.” These aerial counter-maps are thus “being used to provide evidence against the mining company,” and they also support local community’s efforts to protect their existing lands and forest.”

Screen Shot 2015-07-31 at 3.45.19 PM

Irendra and his colleagues took a direct, community-driven approach to these counter-mapping projects: “Community members are involved in establishing the community drone and in deciding who will be responsible to perform drone mapping activities. […] Village meetings also discussed the plans and strategies to perform mapping activities at various different villages with different challenges and contexts. One part of village meetings was training on mapping and drones where participants were informed about participatory counter-mapping techniques as well as the use and the operation of drones to support rapid participatory counter-mapping for high-quality spatial data. A meeting in Subah village agreed to fund the mapping themselves by a monthly contribution of [50 USD] from each [sub-village].”

In sum, co-authors Irendra and Oliver write that UAVs are “very empowering.” “The sense of power and achievement when community members themselves fly the drone is substantial. The empowerment impact that comes with the knowledge that these images are of greater quality than the concession maps and that they have been acknowledged by the Constitutional Court is even greater.”

It is worth noting that the land-use planning maps controlled by the government and companies were made on “the basis of satellite imagery,” which means that “small hamlets [are] not visible. In the process of map-making by the State, the hamlets literally disappeared, losing any rights to their land in the process. With high-resolution drone maps, however, residential areas, farming, fruit tree forests and other long-term uses of the land are rendered visible. Furthermore, local communities require high-quality maps to re-claim those residential areas which now are ‘officially’ part of company’s concessions. These maps are used to support their arguments to halt new concessions for mining and for oil palm.”

Screen Shot 2015-07-31 at 4.39.47 PM

Not surprisingly, perhaps, “the counter-mapping process also uncovered simmering territorial conflicts.” In one of these conflicts, “it emerged that the unsettled village border is one the problems.” Irendra and fellow co-author Oliver write that “One of the aims of community drones is to map the area of several villages […] and to confirm village borders.”

The team’s use of UAVs for counter-mapping resulted in a number of political victories that went beyond the local level. In one case, for example, a counter-map was “used as legal evidence at the Constitutional Court trial on the 1st September 2014, providing the chance for drone counter maps to be recognized by the Indonesian legal system in the future.” In another case, counter maps were combined with other evidence to “challenge the provincial government to accept what the civil society organizations demand. Some of their demands were achieved and accepted, including: (1) Recognition of community-managed lands, (2) Recognition of customary community rights, and (3) active community engagement in the spatial planning process. These demands had not been addressed before.” In yet a third case, “Maps made by drones were used to support […] arguments that often mining activities are causing detrimental social and ecological effects. The Constitutional Court ruled against the mining corporations [as a result], upholding the obligation of mining companies to install smelters and to process raw minerals and coal before exporting them.”

Screen Shot 2015-07-31 at 4.37.53 PM

These projects have generated a growing interest in UAVs, which is why the local Swandiri Institute recently established a “drones school” where “civil society organizations and community activists who are interested in learning and using drones for mapping and for advocacy work could join and participate.” A second drones school was also launched by other partners to “focus on using drones at village levels to map village areas and to confirm village borders.”

The authors conclude that “the appropriation of drone technology by community activists has the potential to improve the situation with regard to inclusion, transparency, and empowerment. […] Nowadays, younger members of local communities are computer literate. After a mapping flight, images and videos can be directly downloaded on to a laptop, giving instant transparency to village meetings during the mapping project. The resolution is so high that individual houses, trees, etc. can be clearly identified, also increasing transparency and the potential to include just about everybody in territorial discussions.”

But of course, to state the obvious: UAVs are not a silver bullet or “magic wand that can conjure away hierarchies and power structures at the local level or in wider society.” Irendra and team were “unable to use drones in those areas where local elites were in cahoots with plantation and mining companies and controlled traditional institutions such as customary councils and where opposition was marginalized.” In other areas, “hierarchical gender relations […], power dynamics, and territorial disputes between different villages were replicated in the mapping process.” At the same time, the UAV revolution does have “the potential—together with campaigning and political pressure—to force through the recognition of community counter-maps in the spatial planning process […].” To this end, “if embedded within political action, drone technology can revolutionize counter-mapping and become an effective weapon in the struggle against land grabs.” And in this context, “community drones for counter-mapping could well become a technology of the masses, by the masses, and for the masses.”

Indigenous Community in Guyana Builds Drones for Good

If you find yourself in the middle of the jungle somewhere in South America and come across this indigenous community, then you’re probably in Guyana:

guyana-tessa-soldering-800

Screen Shot 2015-02-05 at 8.31.01 AM

Screen Shot 2015-02-05 at 8.33.02 AM

guyana-flight-simulator-800

Screen Shot 2015-02-05 at 8.25.35 AM

Screen Shot 2015-02-05 at 8.32.34 AM

Screen Shot 2015-02-05 at 10.09.10 AM

I’ve been an avid fan of Digital Democracy since 2008 and even had the honor of serving on their Advisory Board during the early days. So I was thrilled when friends Emily Jacobi and Gregor MacLennan told me they were interested in using drones/UAVs for their projects. Six months later, the pictures above explain my excitement.

When Gregor traveled down to Guyana a few months ago, he didn’t bring a drone; he simply brought a bunch of parts and glue, lots of glue. “We didn’t want to just fly into Guyana and fly a drone over the local villages,” writes Gregor. “Our interest was whether this technology could be something that can be used and controlled by the commumunities themselves, and become a tool of em-powerment for helping them have more of a say in their own future. We wanted the Wapichana to be able to repair it themselves, fly it themselves, and process the images to use for their own means.” Oh, and by the way, Gregor had never built a drone before.

And that’s the beauty of Digital Democracy’s approach: co-learning, co-creation and co-experimentation. Moreover, Emily & Gregor didn’t turn to drones simply because it’s the latest fad. They tried using satellite imagery to document illegal logging and deforestation in Guyana but the resolution of said imagery was limited. So they figured drones might do the trick instead. Could this technology be a “tool for positive change in the hands of indigenous communities?” Could local communities in Guyana use flying robots to create maps and thus monitor illegal logging and deforestation?

Building the drone was truly a community effort. “When the motor mount broke, the team scoured the village for different types of plastic, and fashioned a new mount from an old beer crate. The drone was no longer a foreign, mysterious piece of technology, but something they owned, built, & therefore understood.” And that is what it’s all about. Check out the neat video above to see the team in action and the 3D results below based on the data collected.

So what’s next? The Wapichana UAV Team have demonstrated “that a remote indigenous community with no prior engineering experience can build and fly a complex drone and make a detailed map.” The team has already been discussing the multiple ways they want to use their UAVs: “to monitor deforestation of bush islands over time; creating high-resolution maps of villages to use as a basis for resource-management discussions; and flying over logging camps in the forest to document illegal deforestation.” You can make sure this happens by donating to the cause (like I just did). That way, Gregor can continue the training and get “the whole team comfortable with flying and to streamline the process from mission planning to processing imagery.”


Meanwhile, back in Congo-Brazzaville…

braza1

… another team was learning about Drones for Good.

Drones for Good: Technology, Social Movements and The State

Discussions surrounding use of drones, or UAVs, have typically “centered on their use by governments, often for the purpose of surveillance and warfare.” But as colleague Austin Choi-Fitzpatrick rightly notes in his new study, “[t]his focus on the state’s use obscures the opportunity for civil society actors, including social movements, to make use of these technologies.” Austin thus seeks to high-light civil society uses, “ranging from art to digital disruption.” The latter is what I am particularly interested given my previous writings on the use of non-lethal UAVs for civil resistance and for peacebuilding.

Screen Shot 2015-02-02 at 7.50.33 AM

When I began writing my doctoral dissertation some 7 years ago, scholars and activists were particularly interested in measuring the impact of mobile phones on social movements and civil resistance. Today, civil society is also turning to UAVs as evidenced during the recent protests in Hong Kong, Turkey, Poland, Ukraine and Ferguson. “This innovation represents a technological shift in scale for citizen journalists, human rights advocates, and social movement actors,” writes Austin. “As such, it requires a sophisticated assessment of the ethical issues and policy terrain surrounding its use.”

The most disruptive aspect of today’s small, personal UAVs, “is the fundamental break between the camera and the street level. […] The most memorable photographs of violent conflict, social protest and natural disasters have almost all been taken by a person present on the ground. […] UAVs relocate the boundary between what is public and what is private, because camera-equipped UAVs more the line of sight from the street to the air. This simple shift effectively pushes public space from the sidewalk to the stairwell, courtyard, rooftop, and so forth.” As Austin rightly concludes, “‘Open air’ and ‘free space’ are no longer as ‘open’ or ‘free’ as they once were. They are instead now occupied or vulnerable to occupation.” The use of the words “occupied” and “occupation” here is indeed intentional. Austin also makes another crucial point: UAVs  represent a type of innovation that is a “hallmark of asymmetrical warfare.”

One of my favorite books, Wasp, illustrates this asymmetry perfectly; as does the Syria Air Lift project. The latter seeks to fly swarms of UAVs to deliver aid to civilians caught in conflict zones. Little surprise, then, that the State is clamping down on civil society uses of UAVs. At times, they even shoot the UAVs down, as evidenced when “police in Istanbul shot down a camera-equipped UAV while it was monitoring large anti-government protests […].” Authorities would not be shooting down UAVs if they did not pose some form of (real or imagined) threat. And even when they pose no direct threat, UAVs are clearly annoying enough to react to (like a wasp or annoying mosquito). Annoyance is a key tactic in civil guerrilla warfare and civil resistance.

SAP_ReutersPic

Austin goes on to propose a “broad framework to guide a range of non-state and non-commercial actor uses of drones.” This framework is comprised of the following 6 principles:

1. Subsidiarity: decision-making and problem solving should occur at the lowest and least sophisticated level possible. I take this to mean that decisions surrounding the use of drones should be taken at the local level (implying local ownership) and that drones should “only be used to address situations for which there is not a less sophisticated, invasive, or novel use.”

2. Physical and material security: self-explanatory – “care must be taken so that these devices do not collide with people or with one another.”

3. Do no harm: emphasizes a “rights-based approach as found in the development and humanitarian aid communities. “The principle is one of proportionality, in which the question to be answered is, ‘Are the risks of using UAVs in a given humanitarian setting outweighed by the expected benefits?'”

4. Public interest: also self-explanatory but “especially sensitive to the importance of investigative journalism that holds to account the powerful and well-resourced, despite attempts by established interests to discredit these efforts.” Public interest should also include the interests of the local community.

5. Privacy: straightforward issue but not easily resolved: “creating a [privacy] framework that applies in all circumstances is nearly impossible in an era in which digital privacy appears to be a mirage […].

6. Data protection: of paramount importance. Aerial footage of protests can be used by governments to “create a database of known activists.” As such, “[c]ontext specific protocols must ensure the security of data, thereby protecting against physical or digital theft or corruption.”

Are there other principles that should factor into the “Drones for Good” frame-work? If so, what are they? I’ll also be asking these questions in Dubai this week where I’m speaking at the Drones for Good Festival.

Reflections on Digital Humanitarians – The Book

In January 2014, I wrote this blog post announcing my intention to write a book on Digital Humanitarians. Well, it’s done! And launches this week. The book has already been endorsed by scholars at Harvard, MIT, Stanford, Oxford, etc; by practitioners at the United Nations, World Bank, Red Cross, USAID, DfID, etc; and by others including Twitter and National Geographic. These and many more endorsements are available here. Brief summaries of each book chapter are available here; and the short video below provides an excellent overview of the topics covered in the book. Together, these overviews make it clear that this book is directly relevant to many other fields including journalism, human rights, development, activism, business management, computing, ethics, social science, data science, etc. In short, the lessons that digital humanitarians have learned (often the hard way) over the years and the important insights they have gained are directly applicable to fields well beyond the humanitarian space. To this end, Digital Humanitarians is written in a “narrative and conversational style” rather than with dense, technical language.

The story of digital humanitarians is a multifaceted one. Theirs is not just a story about using new technologies to make sense of “Big Data”. For the most part, digital humanitarians are volunteers; volunteers from all walks of life and who occupy every time zone. Many are very tech-savvy and pull all-nighters, but most simply want to make a difference using the few minutes they have with the digital technologies already at their fingertips. Digital humanitarians also include pro-democracy activists who live in countries ruled by tyrants. This story is thus also about hope and humanity; about how technology can extend our humanity during crises. To be sure, if no one cared, if no one felt compelled to help others in need, or to change the status quo, then no one even would bother to use these new, next generation humanitarian technologies in the first place.

I believe this explains why Professor Leysia Palen included the following in her very kind review of my book: “I dare you to read this book and not have both your heart and mind opened.” As I reflected to my editor while in the midst of book writing, an alternative tag line for the title could very well be “How Big Data and Big Hearts are Changing the Face of Humanitarian Response.” It is personally and deeply important to me that the media, would-be volunteers  and others also understand that the digital humanitarians story is not a romanticized story about a few “lone heroes” who accomplish the impossible thanks to their super human technical powers. There are thousands upon thousands of largely anonymous digital volunteers from all around the world who make this story possible. And while we may not know all their names, we certainly do know about their tireless collective action efforts—they mobilize online from all corners of our Blue Planet to support humanitarian efforts. My book explains how these digital volunteers do this, and yes, how you can too.

Digital humanitarians also include a small (but growing) number of forward-thinking professionals from large and well-known humanitarian organizations. After the tragic, nightmarish earthquake that struck Haiti in January 2010, these seasoned and open-minded humanitarians quickly realized that making sense of “Big Data” during future disasters would require new thinking, new risk-taking, new partnerships, and next generation humanitarian technologies. This story thus includes the invaluable contributions of those change-agents and explains how these few individuals are enabling innovation within the large bureaucracies they work in. The story would thus be incomplete without these individuals; without their appetite for risk-taking, their strategic understanding of how to change (and at times circumvent) established systems from the inside to make their organizations still relevant in a hyper-connected world. This may explain why Tarun Sarwal of the International Committee of the Red Cross (ICRC) in Geneva included these words (of warning) in his kind review: “For anyone in the Humanitarian sector — ignore this book at your peril.”

bookcover

Today, this growing, cross-disciplinary community of digital humanitarians are crafting and leveraging ingenious crowdsourcing solutions with trail-blazing insights from advanced computing and artificial intelligence in order to make sense of “Big Data” generated during disasters. In virtually real-time, these new solutions (many still in early prototype stages) enable digital volunteers to make sense of vast volumes of social media, SMS and imagery captured from satellites & UAVs to support relief efforts worldwide.

All of this obviously comes with a great many challenges. I certainly don’t shy away from these in the book (despite my being an eternal optimist : ). As Ethan Zuckerman from MIT very kindly wrote in his review of the book,

“[Patrick] is also a careful scholar who thinks deeply about the limits and potential dangers of data-centric approaches. His book offers both inspiration for those around the world who want to improve our disaster response and a set of fertile challenges to ensure we use data wisely and ethically.”

Digital humanitarians are not perfect, they’re human, they make mistakes, they fail; innovation, after all, takes experimenting, risk-taking and failing. But most importantly, these digital pioneers learn, innovate and over time make fewer mistakes. In sum, this book charts the sudden and spectacular rise of these digital humanitarians and their next generation technologies by sharing their remarkable, real-life stories and the many lessons they have learned and hurdles both cleared & still standing. In essence, this book highlights how their humanity coupled with innovative solutions to “Big Data” is changing humanitarian response forever. Digital Humanitarians will make you think differently about what it means to be humanitarian and will invite you to join the journey online. And that is what it’s ultimately all about—action, responsible & effective action.

Why did I write this book? The main reason may perhaps come as a surprise—one word: hope. In a world seemingly overrun by heart-wrenching headlines and daily reminders from the news and social media about all the ugly and cruel ways that technologies are being used to spy on entire populations, to harass, oppress, target and kill each other, I felt the pressing need to share a different narrative; a narrative about how selfless volunteers from all walks of life, from all ages, nationalities, creeds use digital technologies to help complete strangers on the other side of the planet. I’ve had the privilege of witnessing this digital good-will first hand and repeatedly over the years. This goodwill is what continues to restore my faith in humanity and what gives me hope, even when things are tough and not going well. And so, I wrote Digital Humanitarians first and fore-most to share this hope more widely. We each have agency and we can change the world for the better. I’ve seen this and witnessed the impact first hand. So if readers come away with a renewed sense of hope and agency after reading the book, I will have achieved my main objective.

For updates on events, talks, trainings, webinars, etc, please click here. I’ll be organizing a Google Hangout on March 5th for readers who wish to discuss the book in more depth and/or follow up with any questions or ideas. If you’d like additional information on this and future Hangouts, please click on the previous link. If you wish to join ongoing conversations online, feel free to do so with the FB & Twitter hashtag #DigitalJedis. If you’d like to set up a book talk and/or co-organize a training at your organization, university, school, etc., then do get in touch. If you wish to give a talk on the book yourself, then let me know and I’d be happy to share my slides. And if you come across interesting examples of digital humanitarians in action, then please consider sharing these with other readers and myself by using the #DigitalJedis hashtag and/or by sending me an email so I can include your observation in my monthly newsletter and future blog posts. I also welcome guest blog posts on iRevolutions.

Naturally, this book would never have existed were it for digital humanitarians volunteering their time—day and night—during major disasters across the world. This book would also not have seen the light of day without the thoughtful guidance and support I received from these mentors, colleagues, friends and my family. I am thus deeply and profoundly grateful for their spirit, inspiration and friendship. Onwards!

Digital Jedis: There Has Been An Awakening…

May the Crowd Be With You

Three years ago, 167 digital volunteers and I combed through satellite imagery of Somalia to support the UN Refugee Agency (UNHCR) on this joint project. The purpose of this digital humanitarian effort was to identify how many Somalis had been displaced (easily 200,000) due to fighting and violence. Earlier this year, 239 passengers and crew went missing when Malaysia Flight 370 suddenly disappeared. In response, some 8 million digital volunteers mobilized as part of the digital search & rescue effort that followed.

May the Crowd be With You

So in the first case, 168 volunteers were looking for 200,000+ people displaced by violence and in the second case, some 8,000,000 volunteers were looking for 239 missing souls. Last year, in response to Typhoon Haiyan, digital volunteers spent 200 hours or so tagging social media content in support of the UN’s rapid disaster damage assessment efforts. According to responders at the time, some 11 million people in the Philippines were affected by the Typhoon. In contrast, well over 20,000 years of volunteer time went into the search for Flight 370’s missing passengers.

What to do about this heavily skewed distribution of volunteer time? Can (or should) we do anything? Are we simply left with “May the Crowd be with You”?The massive (and as yet unparalleled) online response to Flight 370 won’t be a one-off. We’re entering an era of mass-sourcing where entire populations can be mobilized online. What happens when future mass-sourcing efforts ask digital volunteers to look for military vehicles and aircraft in satellite images taken of a mysterious, unnamed “enemy country” for unknown reasons? Think this is far-fetched? As noted in my forthcoming book, Digital Humanitarians, this online, crowdsourced military surveillance operation already took place (at least once).

As we continue heading towards this new era of mass-sourcing, those with the ability to mobilize entire populations online will indeed yield an impressive new form of power. And as millions of volunteers continue tagging, tracing various features, this volunteer-generated data combined with machine learning will be used to automate future tagging and tracing needs of militaries and multi-billion dollar companies, thus obviating the need for large volumes of volunteers (especially handy should volunteers seek to boycott these digital operations).

At the same time, however, the rise of this artificial intelligence may level the playing field. But few players out there have ready access to high resolution satellite imagery and the actual technical expertise to turn volunteer-generated tags/traces into machine learning classifiers. To this end, perhaps one way forward is to try and “democratize” access to both satellite imagery and the technology needed to make sense of this “Big Data”. Easier said than done. But maybe less impossible than we may think. Perhaps new, disruptive initiatives like Planet Labs will help pave the way forward.

bio

From Russia with Love: A Match.com for Disaster Response

I’ve been advocating for the development of a “Match.com” for disaster response since early 2010. Such a platform would serve to quickly match hyperlocal needs with relevant resources available at the local and national level, thus facilitating and accelerating self-organization following major disasters. Why advocate for a platform modeled after an online dating website? Because self-organized mutual-aid is an important driver of community resilience.

Russian Bell

Obviously, self-organization is not dependent on digital technology. The word Rynda, for example, is an old Russian word for a “village bell” which was used by local communities to self-organize during emergencies. Interestingly, Rynda became a popular meme on social media during fires in 2010. As my colleague Gregory Asmolov notes in his brilliant new study, a Russian blogger at the time of the fires “posted an emotional open letter to Prime Minister Putin, describing the lack of action by local authorities and emergency services.” In effect, the blogger demanded a “return to an old tradition of self-organization in local communities,” subsequently exclaiming “bring back the Rynda!” This demand grew into a popular meme symbolizing the catastrophic failure of the formal system’s response to the massive fires.

At the time, my colleagues Gregory, Alexey Sidorenko & Glafira Parinos launched the Help Map above in an effort to facilitate self-organization and mutual aid. But as Gregory notes in his new study, “The more people were willing to help, the more difficult it was to coordinate the assistance and to match resources with needs.” Moreover, the Help Map continued to receive reports on needs and offers-of-help after the fires had subsided. To be sure, reports of flooding soon found their way to the map, for example. Gregory, Alexey, Glarifa and team thus launched “Virtual Rynda: The Help Atlas” to facilitate self-help in response to a variety of situations beyond sudden-onset crises.

“We believed that in order to develop the capacity and resilience to respond to crisis situations we would have to develop the potential for mutual aid in everyday life. This would rely on an idea that emergency and everyday-life situations were interrelated. While people’s motivation to help one another is lower during non-emergency situations, if you facilitate mutual aid in everyday life and allow people to acquire skills in using Internet-based technologies to help one another or in asking for assistance, this will help to create an improved capacity to fulfill the potential of mutual aid the next time a disaster happens. […] The idea was that ICTs could expand the range within which the tolling of the emergency bell could be heard. Everyone could ‘ring’ the ‘Virtual Rynda’ when they needed help, and communication networks would magnify the sound until it reached those who could come and help.”

In order to accelerate and scale the matching of needs & resources, Gregory and team (pictured below) sought to develop a matchmaking algorithm. Rynda would ask users to specify what the need was, where (geographically) the need was located and when (time-wise) the need was requested. “On the basis of this data, computer-based algorithms & human moderators could match offers with requests and optimize the process of resource allocation.” Rynda also included personal profiles, enabling volunteers “to develop an online reputation and increase trust between those needing help and those who could offer assistance. Every volunteer profile included not only personal information, but also a history of the individual’s previous activities within the platform.” To this end, in addition to “Help Requests” & “Help Offers,” Rynda also included an entry for “Help Provided” to close the feedback loop.

Asmolov1

As Gregory acknowledges, the results were mixed but certainly interesting and insightful. “Most of the messages [posted to the Rynda platform dealt] with requests for various types of social help, like clothing and medical equipment for children, homes for orphans, people with limited capabilities, or families in need. […]. Some requests from environmental NGOs were related to the mobilization of volunteers to fight against deforestation or to fight wildfires. […]. In another case, a volunteer who responded to a request on the platform helped to transport resources to a family with many children living far from a big city. […]. Many requests concern[ed] children or disabled people. In one case, Rynda found a volunteer who helped a young woman leave her flat for walks, something she could not do alone. In some cases, the platform helped to provide medicine.” In any event, an analysis of the needs posted to Rynda suggests that “the most needed resource is not the thing itself, but the capacity to take it to the person who needs it. Transportation becomes a crucial resource, especially in a country as big as Russia.”

Alas, “Despite the efforts to create a tool that would automatically match a request with a potential help provider, the capacity of the algorithm to optimize the allocation of resources was very limited.” To this end, like the Help Map initiative, digital volunteers who served as social moderators remained pivotal to the Virtual Ryndal platform. As Alexey notes, “We’ve never even got to the point of the discussion of more complex models of matching.” Perhaps Rynda should have included more structured categories to enable more automated-matching since the volunteer match-makers are simply not scalable. “Despite the intention that the ‘matchmaking’ algorithm would support the efficient allocation of resources between those in need and those who could help, the success of the ‘matchmaking’ depended on the work of the moderators, whose resources were limited. As a result, a gap emerged between the broad issues that the project could address and the limited resources of volunteers.”

To this end, Gregory readily admits that “the initial definition of the project as a general mutual aid platform may have been too broad and unspecific.” I agree with this diagnostic. Take the online dating platform Match.com for example. Match.com’s sole focus is online dating; Airbnb’s sole purpose is to match those looking for a place to stay with those offering their places; Uber’s sole purpose is matching those who need to get somewhere with a local car service. To this end, matching platform for mutual-aid may indeed been too broad—at least to begin with. Amazon began with books, but later diversified.

In any case, as Gregory rightly notes, “The relatively limited success of Rynda didn’t mean the failure of the idea of mutual aid. What […] Rynda demonstrates is the variety of challenges encountered along the way of the project’s implementation.” To be sure, “Every society or community has an inherent potential mutual aid structure that can be strengthened and empowered. This is more visible in emergency situations; however, major mutual aid capacity building is needed in everyday, non-emergency situations.” Thanks to Gregory and team, future digital matchmakers can draw on the above insights and Rynda’s open source code when designing their own mutual-aid and self-help platforms.

For me, one of the key take-aways is the need for a scalable matching platform. Match.com would not be possible if the matching were done primarily manually. Nor would Match.com work as well if the company sought to match interests beyond the romantic domain. So a future Match.com for mutual-aid would need to include automated matching and begin with a very specific matching domain. 

Bio

 

See also:

  • Using Waze, Uber, AirBnB, SeeClickFix for Disaster Response [link]
  • MatchApp: Next Generation Disaster Response App? [link]
  • A Marketplace for Crowdsourcing Crisis Response [link]

Results of the Crowdsourced Search for Malaysia Flight 370 (Updated)

Update: More than 3 million volunteers thus far have joined the crowdsourcing efforts to locate the missing Malaysian Airlines plane. These digital volunteers have viewed over a quarter-of-a-billion micro-maps and have tagged almost 3 million features in these satellite maps. Source of update.

Malaysian authorities have now gone on record to confirm that Flight 370 was hijacked, which reportedly explains why contact with the passenger jet abruptly ceased a week ago. The Search & Rescue operations now involve 13 countries around the world and over 100 ships, helicopters and airplanes. The costs of this massive operation must easily be running into the millions of dollars.

FlightSaR

Meanwhile, a free crowdsourcing platform once used by digital volunteers to search for Genghis Khan’s Tomb and displaced populations in Somalia (video below) has been deployed to search high-resolution satellite imagery for signs of the missing airliner. This is not the first time that crowdsourced satellite imagery analysis has been used to find a missing plane but this is certainly the highest profile operation yet, which may explain why the crowdsourcing platform used for the search (Tomnod) reportedly crashed for over a dozen of hours since the online search began. (Note that Zooniverse can easily handle this level of traffic). Click on the video below to learn more about the crowdsourced search for Genghis Khan and displaced peoples in Somalia.

NatGeoVideo

Having current, high-resolution satellite imagery is almost as good as having your own helicopter. So the digital version of these search operations includes tens of thousands of digital helicopters, whose virtual pilots are covering over 2,000 square miles of Thailand’s Gulf right from their own computers. They’re doing this entirely for free, around the clock and across multiple time zones. This is what Digital Humanitarians have been doing ever since the 2010 Haiti Earthquake, and most recently in response to Typhoon Yolanda.

Tomnod has just released the top results of the crowdsourced digital search efforts, which are displayed in the short video below. Like other microtasking platforms, Tomnod uses triangulation to calculate areas of greatest consensus by the crowd. This is explained further here. Note: The example shown in the video is NOT a picture of Flight 370 but perhaps of an airborne Search & Rescue plane.

While looking for evidence of the missing airliner is like looking for the proverbial needle in a massive stack of satellite images, perhaps the biggest value-added of this digital search lays in identifying where the aircraft is most definitely not located—that is, approaching this crowdsourced operation as a process of elimination. Professional imagery analysts can very easily and quickly review images tagged by the crowd, even if they are mistakenly tagged as depicting wreckage. In other words, the crowd can provide the first level filter so that expert analysts don’t waste their time looking at thousands of images of bare oceans. Basically, if the mandate is to leave no stone unturned, then the crowd can do that very well.

In sum, crowdsourcing can reduce the signal to noise ratio so that experts can focus more narrowly on analyzing the potential signals. This process may not be perfect just yet but it can be refined and improved. (Note that professionals also get it wrong, like Chinese analysts did with this satellite image of the supposed Malaysian airliner).

If these digital efforts continue and Flight 370 has indeed been hijacked, then this will certainly be the first time that crowdsourced satellite imagery analysis is used to find a hijacked aircraft. The latest satellite imagery uploaded by Tomnod is no longer focused on bodies of water but rather land. The blue strips below (left) is the area that the new satellite imagery covers.

Tomnod New Imagery 2

Some important questions will need to be addressed if this operation is indeed extended. What if the hijackers make contact and order the cessation of all offline and online Search & Rescue operations? Would volunteers be considered “digital combatants,” potentially embroiled in political conflict in which the lives of 227 hostages are at stake?

bio

Note: The Google Earth containing the top results of the search is available here.

See also: Analyzing Tweets on Malaysia Flight #MH370 [link]

Yes, I’m Writing a Book (on Digital Humanitarians)

I recently signed a book deal with Taylor & Francis Press. The book, which is tentatively titled “Digital Humanitarians: How Big Data is Changing the Face of Disaster Response,” is slated to be published next year. The book will chart the rise of digital humanitarian response from the Haiti Earthquake to 2015, highlighting critical lessons learned and best practices. To this end, the book will draw on real-world examples of digital humanitarians in action to explain how they use new technologies and crowdsourcing to make sense of “Big (Crisis) Data”. In sum, the book will describe how digital humanitarians & humanitarian technologies are together reshaping the humanitarian space and what this means for the future of disaster response. The purpose of this book is to inspire and inform the next generation of (digital) humanitarians while serving as a guide for established humanitarian organizations & emergency management professionals who wish to take advantage of this transformation in humanitarian response.

2025

The book will thus consolidate critical lessons learned in digital humanitarian response (such as the verification of social media during crises) so that members of the public along with professionals in both international humanitarian response and domestic emergency management can improve their own relief efforts in the face of “Big Data” and rapidly evolving technologies. The book will also be of interest to academics and students who wish to better understand methodological issues around the use of social media and user-generated content for disaster response; or how technology is transforming collective action and how “Big Data” is disrupting humanitarian institutions, for example. Finally, this book will also speak to those who want to make a difference; to those who of you who may have little to no experience in humanitarian response but who still wish to help others affected during disasters—even if you happen to be thousands of miles away. You are the next wave of digital humanitarians and this book will explain how you can indeed make a difference.

The book will not be written in a technical or academic writing style. Instead, I’ll be using a more “storytelling” form of writing combined with a conversational tone. This approach is perfectly compatible with the clear documentation of critical lessons emerging from the rapidly evolving digital humanitarian space. This conversational writing style is not at odds with the need to explain the more technical insights being applied to develop next generation humanitarian technologies. Quite on the contrary, I’ll be using intuitive examples & metaphors to make the most technical details not only understandable but entertaining.

While this journey is just beginning, I’d like to express my sincere thanks to my mentors for their invaluable feedback on my book proposal. I’d also like to express my deep gratitude to my point of contact at Taylor & Francis Press for championing this book from the get-go. Last but certainly not least, I’d like to sincerely thank the Rockefeller Foundation for providing me with a residency fellowship this Spring in order to accelerate my writing.

I’ll be sure to provide an update when the publication date has been set. In the meantime, many thanks for being an iRevolution reader!

bio