Category Archives: Social Media

The Future of Crisis Mapping is Finally Here

Posted on June 7, 2017 | 1 comment

In 2010, I had the opportunity to participate in the very first Disaster Response Working Group meeting held at Facebook. The digital humanitarian response to the tragic Haiti earthquake months earlier was the main point of discussion. Digital Humanitarians at the time had crowdsourced social media monitoring and satellite imagery analysis to create a unique set of crisis maps used by a range of responders. Humanitarian organizations to this day point to the Haiti response as a pivotal milestone in the history of crisis mapping. Today marks an equally important milestone thanks to three humanitarian groups and Facebook.

Facebook just announced a new partnership with UNICEF, the International Federation of the Red Cross (IFRC), American Red Cross (ARC) and the World Food Program (WFP) to begin sharing actionable, real-time data that will fill critical data gaps that exist in the first hours of a sudden onset disaster. UNICEF, IFRC, ARC and WFP deserve considerable praise in partnering on such an innovative effort. As the IFRC’s World Disaster Report noted in 2005, having access to information during disasters is equally important as having access to food, water and medicine. But unlike these other commodities, information has a far shorter shelf life. In other words, the value of information depreciates very quickly; information rots fast.

Disaster responders need information that is both reliable and timely. Both are typically scarce after disasters. Saving time can make all the difference. The faster responders get reliable information, the faster they can prioritize and mobilize relief efforts based on established needs. Information takes time to analyze, however, especially unstructured information. Digital Humanitarians encountered this Big Data challenge first hand during the Haiti Earthquake response, and after most disasters since then. Still, online data has the potential to fill crucial data gaps. This is especially true if this data is made available in a structured and responsible way by a company like Facebook; a platform that reaches nearly 2 billion people around the world. And by listening to what aid organizations need, Facebook is providing this information in a format that is actually usable and useful.

Listening to Humanitarian Needs

In early 2016, I began consulting with Facebook on their disaster mapping initiative. One of our first orders of business was to reach out to subject matter experts around the world. It is all too easy for companies in Silicon Valley to speculate about solutions that could be useful to humanitarian organizations. The problem with that approach is that said companies almost never consult seasoned humanitarian professionals in the process. Facebook took a different approach. They spent well over half-a-year meeting with and listening to humanitarian professionals across a number of different aid organizations. Then, they co-developed the solution together with experts from UNCIEF, IFRC, ARC, WFP and myself. This process insured that they built solutions that are actually needed by the intended end users. Other Silicon Valley companies really ought to take the same approach when seeking to support social good efforts in a meaningful manner.

UNICEF, IFRC, ARC and WFP bring extensive expertise and global reach to this new partnership with Facebook. They have both the capacity and strong interest to fully leverage the new disaster maps being made available. And each of these humanitarian organizations have spent a considerable amount of time and energy collaborating with Facebook to iterate on the disaster maps. This type of commitment, partnership and leadership from the humanitarian sector is vital and indeed absolutely necessary to innovate and scale innovation.

One of the areas in which Facebook exercised great care was in applying protection standards. This was another area in which I provided guidance, along with colleagues at the International Committee of the Red Cross (ICRC). We worked closely with Facebook to ensure that their efforts followed established protection protocols in the humanitarian sector. In September 2016, for example, three Facebookers and I participated in a full-day protection workshop organized the ICRC. Facebook presented on the new mapping project – still in its very early stages – and actively solicited feedback from the ICRC and a dozen other humanitarian organizations that participated in the workshop. Facebook noted upfront that they didn’t have all the answers and welcomed as much input as humanitarian professionals could give. As it turns out, they were already well on their way to being fully in line with the ICRC’s own protection protocols.

Facebook also worked with its own internal privacy, security and legal teams to ensure that the datasets it produced were privacy-preserving and consistent with legal standards around the world. This process took a long time. Some insight from the “inside”: I began joking that this process makes the UN look fast. But the fact that Facebook was so careful and meticulous when it came to data privacy was certainly reassuring. To be sure, Facebook developed a rigorous review process to ensure that our applied research was carried out responsibly and ethically. This demonstrates that using data for high-impact, social good projects need not be at odds with privacy—we can achieve both. By using data aggregating and spatial smoothing, for example, we can reduce noise in the data and identify important trends while following its data privacy standards.

Another important area of collaboration very early on focused specifically on data bias. The team at Facebook was careful to emphasize that their data was not a silver bullet – it is representative of people who use Facebook on mobile with Location Services enabled. To this end, one of the areas I worked on closely with Facebook was validation. For example, in an early iteration of the maps, I analyzed mainstream media news reports on the Fort McMurray Fires in Canada and matched them with specific patterns we had observed on Facebook’s maps. The results suggested that Facebook’s geospatial data was providing reliable insights about evacuation and safety on the ground albeit in real time compared to the media reports which were published many hours later.

Facebook Safety Check

Within 24 hours of activating Safety Check, we see that there are far fewer people than usual in the town of Fort McMurray. Areas that are color-coded red reflect much lower numbers of Facebook users there compared to the same time the week before. This makes sense since these locations are affected by the wildfires and have thus been evacuated.

We can use Facebook’s Safety Check data to create live disaster maps that quickly highlight where groups of users are checking in safe, and also where they are not checking in safe. This could provide a number of important proxies such as disaster damage, for example.

Facebook Location Maps

We see that before the crisis began (left plot) people were located in the town in expected numbers, but quickly vacated over the next 24 hour period (map turning red). Even within just an hour and half into the crisis we can tell that users are evacuating the town (the red color indicating low values of people present compared to baseline data). This signal becomes even more clear and consistent as the crisis progresses.

Population here refers to the population of Facebook users. These aggregated maps can provide a proxy for population density and movement before, during and after humanitarian disasters.

In the above video, the blue line that stretches diagonally across the map is Highway 63, which was the primary evacuation route for many in McMurray. The video shows where the general population of Facebook users is moving over time at half-hour intervals. Notice that the blue line becomes even denser between 1 and 3 A.M. local time. Reports from the mainstream media published that afternoon revealed that many drivers ended up having to “camp” along the highway overnight.

Take the map below of the Kaikoura Earthquake in New Zealand as another example. The disaster maps for the earthquake show the location and movement of people in Kaikoura following the disaster. One day after the earthquake, we notice that the population begins to evacuate the city. Using news articles, we can cross validate that residents of Kaikoura were evacuated to Christchurch, 200 kilometers away. Several days later, we notice from the Facebook maps that individuals are starting to return to Kaikoura, presumably to repair and rebuild their community.

It’s still early days, and Facebook plans to work closely alongside their partners to better understand and report biases in the data. This is another reason why Facebook’s partnership with UNICEF, IFRC, ARC and WFP is so critical. These groups have the capacity to compare the disaster maps with other datasets, validate the maps with field surveys, and support Facebook in understanding how to address issues of representativeness. One approach they are exploring is to compare the disaster maps to the population density datasets that Facebook has already open-sourced. By making this comparison, we can clearly communicate any areas that are likely to be inadequately covered by the disaster data. They are also working with Facebook’s Connectivity Lab to develop bias-correcting solutions based on maps of cell phone connectivity. For more on social media, bias and crisis mapping, see Chapter 2 of Digital Humanitarians.

Moving Forward

Our humanitarian partners are keen to use Facebook’s new solution in their relief efforts. Thanks to Facebook’s data, we can create a series of unique maps that in turn provide unique insights and do so in real-time. These maps can be made available right away and updated at 15 minute intervals if need be. Let me repeat that: every 15 minutes. This is the first time in history that humanitarian organizations will have access to such high frequency, privacy-preserving structured data powered by some 1.86 billion online users.

There is no doubt that responders would’ve had far more situational awareness and far more quickly had these crisis maps existed in the wake of Haiti’s tragic earthquake in 2010. Since the maps aggregate Facebook data to administrative boundaries, humanitarian partners can also integrate this unique dataset into their own systems. During the first Facebook Disaster Working Group meeting back in 2010, we asked ourselves how Facebook might leverage it’s own data to create unique maps to help aid organizations reduce suffering and loss of life. Today, not only do we have an answer to this question, we also have the beginnings of an operational solution that humanitarians can use directly.

Facebook’s new disaster mapping solution is not a silver bullet, however; all my colleagues at Facebook recognize this full well, as do our humanitarian partners. These maps simply serve as new, unique and independent sources of real-time data and insights for humanitarian organizations. The number of Facebook users has essentially doubled since the Haiti Earthquake, nearing 2 billion users today. The more people around the planet connect and share on Facebook, the more insights responders gain on how best to carry out relief efforts during major disasters. This information is a public good that has the potential to save lives, and it’s crucial that insights derived from the data be made available to those who can put it to use. I sincerely hope that other Silicon Valley companies take note of these efforts and following in Facebook’s footsteps.

As a next step, Facebook is looking to both international and local humanitarian partners to help improve, validate and measure the impact of these new disaster maps. As the Facebook team works to validate the maps with the humanitarian community, they also hope to make the maps available to aid organizations though a dedicated API and Visualization tool. Interested organizations will be asked to follow a simple application process to gain access to the disaster maps.

Facebook disaster maps are really unique and we’ve only begun to scratch the surface vis-à-vis the different humanitarian efforts these maps can inform. For example, my team and I at WeRobotics were recently in the Dominican Republic (DR) where we ran a full-fledged disaster response exercise with the country’s Emergency Operations Center (EOC) and the World Food Program (WFP). The purpose of the simulation—which focused on searching for survivors and assessing disaster damage—was to develop and test coordination mechanisms to facilitate the rapid deployment of small drones or Unmanned Aerial Vehicles (UAVs). As the drone pilots began to program their drones to carry out the aerial surveys, I turned to my WFP colleague Gabriela and said:

“What if, during the next disaster, we used Facebook’s Safety Check Map to prioritize which areas the drones should search? What if we used Facebook’s Population Map to prioritize aerial surveys of areas that are being abandoned, possibly to due to collapsed buildings or other types of infrastructure damage? Since the Facebook maps are available in near real-time, we could program the drone flights within minutes of a disaster. What do you think?”

Gaby looked back at the drones and said:

“Wow. This would change everything.”

1 Comment

Posted in Crisis Mapping, Humanitarian Technologies, Social Computing, Social Media

Tagged Check, Displacement, Facebook, maps, Movement, Population, Safety

Using Sound and Artificial Intelligence to Detect Human Rights Violations

Posted on September 24, 2016 | 2 comments

Video continues to be a powerful way to capture human rights abuses around the world. Videos posted to social media can be used to hold perpetrators of gross violations accountable. But video footage poses a “Big Data” challenge to human rights organizations. Two billion smartphone users means almost as many video cameras. This leads to massive amounts of visual content of both suffering and wrong-doing during conflict zones. Reviewing these videos manually is a very labor intensive, time consuming, expensive and often traumatic task. So my colleague Jay Aronson at CMU has been exploring how artificial intelligence and in particular machine learning might solve this challenge.

As Jay and team rightly note in a recent publication (PDF), “the dissemination of conflict and human rights related video has vastly outpaced the ability of researchers to keep up with it – particularly when immediate political action or rapid humanitarian response is required.” The consequences of this are similar to what I’ve observed in humanitarian aid: At some point (which will vary from organization to organization), time and resource limitations will necessitate an end to the collection, archiving, and analysis of user generated content unless the process can be automated.” In sum, information overload can “prevent human rights researchers from uncovering widely dispersed events taking place over long periods of time or large geographic areas that amount to systematic human rights violations.”

To take on this Big Data challenge, Jay and team have developed a new machine learning-based audio processing system that “enables both synchronization of multiple audio-rich videos of the same event, and discovery of specific sounds (such as wind, screaming, gunshots, airplane noise, music, and explosions) at the frame level within a video.” The system basically “creates a unique “soundprint” for each video in a collection, synchronizes videos that are recorded at the same time and location based on the pattern of these signatures, and also enables these signatures to be used to locate specific sounds precisely within a video. The use of this tool for synchronization ultimately provides a multi-perspectival view of a specific event, enabling more efficient event reconstruction and analysis by investigators.”

Synchronizing image features is far more complex than synchronizing sound. “When an object is occluded, poorly illuminated, or not visually distinct from the background, it cannot always be detected by computer vision systems. Further, while computer vision can provide investigators with confirmation that a particular video was shot from a particular location based on the similarity of the background physical environment, it is less adept at synchronizing multiple videos over time because it cannot recognize that a video might be capturing the same event from different angles or distances. In both cases, audio sensors function better so long as the relevant videos include reasonably good audio.”

Ukrainian human rights practitioners working with families of protestors killed during the 2013-2014 Euromaidan Protests recently approached Jay and company to analyze videos from those events. They wanted to “ locate every video available in their collection of the moments before, during, and just after a specific set of killings. They wanted to extract information from these videos, including visual depictions of these killings, whether the protesters in question were an immediate and direct threat to the security forces, plus any other information that could be used to corroborate or refute other forms of evidence or testimony available for their cases.”

Their plan had originally been to manually synchronize more than 65 hours of video footage from 520 videos taken during the morning of February 20, 2014. But after working full-time over several months, they were only able to stitch together about 4 hours of the total video using visual and audio cues in the recording.” So Jay and team used their system to make sense of the footage. They were able to automatically synchronize over 4 hours of the footage. The figure above shows an example of video clips synchronized by the system.

Users can also “select a segment within the video containing the event they are interested in (for example, a series of explosions in a plaza), and search in other videos for a similar segment that shows similar looking buildings or persons, or that contains a similar sounding noise. A user may for example select a shooting scene with a significant series of gunshots, and may search for segments with a similar sounding series of gunshots. This method increases the chances for finding video scenes of an event displaying different angles of the scene or parallel events.”

Jay and team are quick to emphasize that their system “does not eliminate human involvement in the process because machine learning systems provide probabilistic, not certain, results.” To be sure, “the synchronization of several videos is noisy and will likely include mistakes—this is precisely why human involvement in the process is crucial.”

I’ve been following Jay’s applied research for many years now and continue to be a fan of his approach given the overlap with my own work in the use of machine learning to make sense of the Big Data generated during major natural disasters. I wholeheartedly agree with Jay when he reflected during a recent call that the use of advanced techniques alone is not the answer. Effective cross-disciplinary collaboration between computer scientists and human rights (or humanitarian) practitioners is really hard but absolutely essential. This explains why I wrote this practical handbook on how to create effective collaboration and successful projects between computer scientists and humanitarian organizations.

2 Comments

Posted in Big Data, Digital Activism, Information Forensics, Social Media

Tagged AI, Learning, Machine, video

The Value of Timely Information During Disasters (Measured in Hours)

Posted on April 6, 2016 | 14 comments

In the 2005 World Disaster Report (PDF), the International Federation of the Red Cross states unequivocally that access to information during disasters is equally important as access to food, water, shelter and medication. Of all these commodities, however, crisis information is the most perishable. In other words, the “sell-by” or “use-by” date of information for decision-making during crisis is very short. Put simply: information rots fast, especially in the field (assuming that information even exists in the first place). But how fast exactly as measured in hours and days?

Enter this handy graph by FEMA, which is based on a large survey of emergency management professionals across the US. As you’ll note, there is a very clear cut-off at 72 hours post-disaster by which time the value of information for decision making purposes has depreciated by 60% to 85%. Even at 48 hours, information has lost 35% to 65% of its initial tactical value. Disaster responders don’t have the luxury of waiting around for actionable information to inform their decisions during the first 24-72 hours after a disaster. So obviously they’ll take those decisions whether or not timely data is available to guide said decisions.

In a way, the graph also serves as a “historical caricature” of the availability of crisis information over the past 25 years:

During the early 1990s, when the web and mobile phones were still in their infancy, it often took weeks to collect detailed information on disaster damage and needs following major disasters. Towards the end of the 2000’s, thanks to the rapid growth in smartphones, social media and the increasing availability of satellite imagery plus improvements in humanitarian information management systems, the time it took to collect crisis information was shortened. One could say we crossed the 72-hour time barrier on January 12, 2010 when a devastating earthquake struck Haiti. Five years later, the Nepal earthquake in April 2015 may have seen a number of formal responders crossing the 48-hour threshold.

While these observations are at best the broad brushstrokes of a caricature, the continued need for timely information is very real, especially for tactical decision making in the field. This is why we need to shift further left in the FEMA graph. Of course, information that is older than 48 hours is still useful, particularly for decision-makers at headquarters who do not need to make tactical decisions.

In fact, the real win would be to generate and access actionable information within the first 12- to 24-hour mark. By the end of the 24-hours, the value of information has “only” depreciated by 10% to 35%. So how do we get to the top left corner of the graph? How do we get to “Win”?

By integrating new and existing sensors and combining these with automated analysis solutions. New sensors: like Planet Lab’s growing constellation of micro-satellites, which will eventually image the entire planet once every 24 hours at around 3-meter resolution. And new automated analysis solutions: powered by crowdsourcing and artificial intelligence (AI), and in particular deep learning techniques to process the Big Data generated by these “neo-sensors” in near real-time, including multimedia posted to social media sites and the Web in general.

And the need for baseline data is no less important for comparative analysis and change detection purposes. As a colleague of mine recently noted, the value of baseline information before a major disaster is at an all time high but then itself depreciates as well post-disaster.

Of course, access to real-time information does not make a humanitarian organization a real-time response organization. There are always delays regard-less of how timely (or not) the information is (assuming it is even available). But the real first responders are the local communities. So the real win here would be to make make this real-time analysis directly available to local partners in disaster prone countries. They often have more of an immediate incentive to generate and consume timely, tactical information. I described this information flow as “crowdfeeding” years ago.

In sum, the democratization of crisis information is key (keeping in mind data-protection protocols). But said democratization isn’t enough. The know-how and technologies to generate and analyze crisis information during the first 12-24 hours must also be democratized. The local capacity to respond quickly and effectively must exist; otherwise timely, tactical information will just rot away.

I’d be very interested to hear from human rights practitioners to get their thoughts on how/when the above crisis information framework does, and does not, apply when applied to human rights monitoring.

14 Comments

Posted in Satellite Imagery, Social Media

Tagged Disasters, Information, management

New Findings: Rapid Assessment of Disaster Damage Using Social Media

Posted on March 17, 2016 | 2 comments

The latest peer-reviewed, scientific research on social media & crisis computing has just been published in the prestigious journal, Science. The authors pose a question that many of us in the international humanitarian space have been asking, debating and answering since 2009: Can social media data aid in disaster response and damage assessment?

To answer this question, the authors of the new study carry out “a multiscale analysis of Twitter activity before, during, and after Hurricane Sandy” and “examine the online response of 50 metropolitan areas of the US.” They find a “strong relationship between proximity to Sandy’s path and hurricane-related social media activity.” In addition, they “show that real and perceived threats, together with physical disaster effects, are directly observable through the intensity and composition of Twitter’s message stream.”

What’s more, they actually “demonstrate that per-capita Twitter activity strongly correlates with the per-capita economic damage inflicted by the hurricane.” The authors found these results to hold true for a “wide range of [US-based] disasters and suggest that massive online social networks can be used for rapid assessment of damage caused by a large-scale disaster.”

Unlike the vast majority of crisis computing studies in the scientific literature, this is one of the few (perhaps the only?) study of its kind that uses actual post-disaster damage data, i.e. actual ground-truthing, to demonstrate that “the per-capita number of Twitter messages corresponds directly to disaster-inflicted monetary damage.” What’s more, “The correlation is especially pronounced for persistent post-disaster activity and is weakest at the peak of the disaster.”

The authors thus conclude that social media is a “viable platform for preliminary rapid damage assessment in the chaotic time immediately after a disaster.” As such, their results suggest that “officials should pay attention to normalized activity levels, rates of original content creation, and rates of content rebroadcast to identify the hardest hit areas in real time. Immediately after a disaster, they should focus on persistence in activity levels to assess which areas are likely to need the most assistance.”

In sum, the authors found that “Twitter activity during a large-scale natural disaster—in this instance Hurricane Sandy—is related to the proximity of the region to the path of the hurricane. Activity drops as the distance from the hurricane increases; after a distance of approximately 1200 to 1500 km, the influence of proximity disappears. High-level analysis of the composition of the message stream reveals additional findings. Geo-enriched data (with location of tweets inferred from users’ profiles) show that the areas close to the disaster generate more original content […].”

Five years ago, professional humanitarians were still largely dismissive of social media’s added value in disasters. Three years ago, it was the turn of ivory tower academics in the social sciences to dismiss the value added of social media for disaster response. The criticisms focused on the notion that reports posted on social media were simply untrustworthy and hardly representative. The above peer-reviewed scientific study dismisses these limitations as inconsequential.

2 Comments

Posted in Humanitarian Technologies, Social Computing, Social Media

Tagged Correlation, Damage, Disasters, Twitter

QED – Goodbye Doha, Hello Adventure!

Posted on October 14, 2015 | 5 comments

Quod Erat Demonstrandum (QED) is Latin for “that which had to be proven.” This abbreviation was traditionally used at the end of mathematical proofs to signal the completion of said proofs. I joined the Qatar Computing Research Institute (QCRI) well over 3 years ago with a very specific mission and mandate: to develop and deploy next generation humanitarian technologies. So I built the Institute’s Social Innovation Program from the ground up and recruited the majority of the full-time experts (scientists, engineers, research assistants, interns & project manager) who have become integral to the Program’s success. During these 3+years, my team and I partnered directly with humanitarian and development organizations to empirically prove that methods from advanced computing can be used to make sense of Big (Crisis) Data. The time has thus come to add “QED” to the end of that proof and move on to new adventures. But first a reflection.

Over the past 3.5 years, my team and I at QCRI developed free and open source solutions powered by crowdsourcing and artificial intelligence to make sense of Tweets, text messages, pictures, videos, satellite and aerial imagery for a wide range of humanitarian and development projects. We co-developed and co-deployed these platforms (AIDR and MicroMappers) with the United Nations and the World Bank in response to major disasters such as Typhoons Haiyan and Ruby, Cyclone Pam and both the Nepal & Chile Earthquakes. In addition, we carried out peer-reviewed, scientific research on these deployments to better understand how to meet the information needs of our humanitarian partners. We also tackled the information reliability question, experimenting with crowd-sourcing (Verily) and machine learning (TweetCred) to assess the credibility of information generated during disasters. All of these initiatives were firsts in the humanitarian technology space.

We later developed AIDR-SMS to auto-classify text messages; a platform that UNICEF successfully tested in Zambia and which the World Food Program (WFP) and the International Federation of the Red Cross (IFRC) now plan to pilot. AIDR was also used to monitor a recent election, and our partners are now looking to use AIDR again for upcoming election monitoring efforts. In terms of MicroMappers, we extended the platform (considerably) in order to crowd-source the analysis of oblique aerial imagery captured via small UAVs, which was another first in the humanitarian space. We also teamed up with excellent research partners to crowdsource the analysis of aerial video footage and to develop automated feature-detection algorithms for oblique imagery analysis based on crowdsourced results derived from MicroMappers. We developed these Big Data solutions to support damage assessment efforts, food security projects and even this wildlife protection initiative.

In addition to the above accomplishments, we launched the Internet Response League (IRL) to explore the possibility of leveraging massive multiplayer online games to process Big Crisis Data. Along similar lines, we developed the first ever spam filter to make sense of Big Crisis Data. Furthermore, we got directly engaged in the field of robotics by launching the Humanitarian UAV Network (UAViators), yet another first in the humanitarian space. In the process, we created the largest repository of aerial imagery and videos of disaster damage, which is ripe for cutting-edge computer vision research. We also spearheaded the World Bank’s UAV response to Category 5 Cyclone Pam in Vanuatu and also directed a unique disaster recovery UAV mission in Nepal after the devastating earthquakes. (I took time off from QCRI to carry out both of these missions and also took holiday time to support UN relief efforts in the Philippines following Typhoon Haiyan in 2013). Lastly, on the robotics front, we championed the development of international guidelines to inform the safe, ethical & responsible use of this new technology in both humanitarian and development settings. To be sure, innovation is not just about the technology but also about crafting appropriate processes to leverage this technology. Hence also the rationale behind the Humanitarian UAV Experts Meetings that we’ve held at the United Nations Secretariat, the Rockefeller Foundation and MIT.

All of the above pioneering-and-experimental projects have resulted in extensive media coverage, which has placed QCRI squarely on the radar of international humanitarian and development groups. This media coverage has included the New York Times, Washington Post, Wall Street Journal, CNN, BBC News, UK Guardian, The Economist, Forbes and Times Magazines, New Yorker, NPR, Wired, Mashable, TechCrunch, Fast Company, Nature, New Scientist, Scientific American and more. In addition, our good work and applied research has been featured in numerous international conference presentations and keynotes. In sum, I know of no other institute for advanced computing research that has contributed this much to the international humanitarian space in terms of thought-leadership, strategic partnerships, applied research and operational expertise through real-world co-deployments during and after major disasters.

There is, of course, a lot more to be done in the humanitarian technology space. But what we have accomplished over the past 3 years clearly demonstrates that techniques from advanced computing can indeed provide part of the solution to the pressing Big Data challenge that humanitarian & development organizations face. At the same time, as I wrote in the concluding chapter of my new book, Digital Humanitarians, solving the Big Data challenge does not alas imply that international aid organizations will actually make use of the resulting filtered data or any other data for that matter—even if they ask for this data in the first place. So until humanitarian organizations truly shift towards both strategic and tactical evidence-based analysis & data-driven decision-making, this disconnect will surely continue unabated for many more years to come.

Reflecting on the past 3.5 years at QCRI, it is crystal clear to me that the number one most important lesson I (re)learned is that you can do anything if you have an outstanding, super-smart and highly dedicated team that continually goes way above and beyond the call of duty. It is one thing for me to have had the vision for AIDR, MicroMappers, IRL, UAViators, etc., but vision alone does not amount to much. Implementing said vision is what delivers results and learning. And I simply couldn’t have asked for a more talented & stellar team to translate these visions into reality over the past 3+years. You each know who you are, partners included; it has truly been a privilege and honor working with you. I can’t wait to see what you do next at/with QCRI. Thank you for trusting me; thank you for sharing my vision; thanks for your sense of humor, and thank you for your dedication and loyalty to science and social innovation.

So what’s next for me? I’ll be lining up independent consulting work with several organizations (likely including QCRI). In short, I’ll be open for business. I’m also planning to work on a new project that I’m very excited about, so stay tuned for updates; I’ll be sure to blog about this new adventure when the time is right. For now, I’m busy wrapping up my work as Director of Social Innovation at QCRI and working with the best team there is. QED.

5 Comments

Posted in Big Data, Crowdsourcing, Drones/UAVs, Humanitarian Technologies, Information Forensics, Satellite Imagery, Social Computing, Social Media

Tagged QCRI

Social Media for Disaster Response – Done Right!

Posted on June 29, 2015 | 3 comments

To say that Indonesia’s capital is prone to flooding would be an understatement. Well over 40% of Jakarta is at or below sea level. Add to this a rapidly growing population of over 10 million and you have a recipe for recurring disasters. Increasing the resilience of the city’s residents to flooding is thus imperative. Resilience is the capacity of affected individuals to self-organize effectively, which requires timely decision-making based on accurate, actionable and real-time information. But Jakarta is also flooded with information during disasters. Indeed, the Indonesian capital is the world’s most active Twitter city.

So even if relevant, actionable information on rising flood levels could somehow be gleaned from millions of tweets in real-time, these reports could be inaccurate or completely false. Besides, only 3% of tweets on average are geo-located, which means any reliable evidence of flooding reported via Twitter is typically not actionable—that is, unless local residents and responders know where waters are rising, they can’t take tactical action in a timely manner. These major challenges explain why most discount the value of social media for disaster response.

But Digital Humanitarians in Jakarta aren’t your average Digital Humanitarians. These Digital Jedis recently launched one of the most promising humanitarian technology initiatives I’ve seen in years. Code named Peta Jakarta, the project takes social media and digital humanitarian action to the next level. Whenever someone posts a tweet with the word banjir (flood), they receive an automated tweet reply from @PetaJkt inviting them to confirm whether they see signs of flooding in their area: “Flooding? Enable geo-location, tweet @petajkt #banjir and check petajakarta.org.” The user can confirm their report by turning geo-location on and simply replying with the keyword banjir or flood. The result gets added to a live, public crisis map, like the one below.

Over the course of the 2014/2015 monsoon season, Peta Jakarta automatically sent 89,000 tweets to citizens in Jakarta as a call to action to confirm flood conditions. These automated invitation tweets served to inform the user about the project and linked to the video below (via Twitter Cards) to provide simple instructions on how to submit a confirmed report with approximate flood levels. If a Twitter user forgets to turn on the geo-location feature of their smartphone, they receive an automated tweet reminding them to enable geo-location and resubmit their tweet. Finally, the platform “generates a thank you message confirming the receipt of the user’s report and directing them to PetaJakarta.org to see their contribution to the map.” Note that the “overall aim of sending programmatic messages is not to simply solicit a high volume of replies, but to reach active, committed citizen-users willing to participate in civic co-management by sharing nontrivial data that can benefit other users and government agencies in decision-making during disaster scenarios.”

A report is considered verified when a confirmed geo-tagged tweet includes a picture of the flooding, like in the tweet below. These confirmed and verified tweets get automatically mapped and also shared with Jakarta’s Emergency Management Agency (BPBD DKI Jakarta). The latter are directly involved in this initiative since they’re “regularly faced with the difficult challenge of anticipating & responding to floods hazards and related extreme weather events in Jakarta.” This direct partnership also serves to limit the “Data Rot Syndrome” where data is gathered but not utilized. Note that Peta Jakarta is able to carry out additional verification measures by manually assessing the validity of tweets and pictures by cross-checking other Twitter reports from the same district and also by monitoring “television and internet news sites, to follow coverage of flooded areas and cross-check reports.”

During the latest monsoon season, Peta Jakarta “received and mapped 1,119 confirmed reports of flooding. These reports were formed by 877 users, indicating an average tweet to user ratio of 1.27 tweets per user. A further 2,091 confirmed reports were received without the required geolocation metadata to be mapped, highlighting the value of the programmatic geo-location ‘reminders’ […]. With regard to unconfirmed reports, Peta Jakarta recorded and mapped a total of 25,584 over the course of the monsoon.”

The Live Crisis Maps could be viewed via two different interfaces depending on the end user. For local residents, the maps could be accessed via smartphone with the visual display designed specifically for more tactical decision-making, showing flood reports at the neighborhood level and only for the past hour.

For institutional partners, the data is visualized in more aggregate terms for strategic decision-making based trends-analysis and data integration. “When viewed on a desktop computer, the web-application scaled the map to show a situational overview of the city.”

Peta Jakarta has “proven the value and utility of social media as a mega-city methodology for crowdsourcing relevant situational information to aid in decision-making and response coordination during extreme weather events.” The initiative enables “autonomous users to make independent decisions on safety and navigation in response to the flood in real-time, thereby helping increase the resilience of the city’s residents to flooding and its attendant difficulties.” In addition, by “providing decision support at the various spatial and temporal scales required by the different actors within city, Peta Jakarta offers an innovative and inexpensive method for the crowdsourcing of time-critical situational information in disaster scenarios.” The resulting confirmed and verified tweets were used by BPBD DKI Jakarta to “cross-validate formal reports of flooding from traditional data sources, supporting the creation of information for flood assessment, response, and management in real-time.”

My blog post is based several conversations I had with Peta Jakarta team and on this white paper, which was just published a week ago. The report runs close to 100 pages and should absolutely be considered required reading for all Digital Humanitarians and CrisisMappers. The paper includes several dozen insights which a short blog post simply cannot do justice to. If you can’t find the time to read the report, then please see the key excerpts below. In a future blog post, I’ll describe how the Peta Jakarta team plans to leverage UAVs to complement social media reporting.

Extracting knowledge from the “noise” of social media requires designed engagement and filtering processes to eliminate unwanted information, reward valuable reports, and display useful data in a manner that further enables users, governments, or other agencies to make non-trivial, actionable decisions in a time-critical manner.

While the utility of passively-mined social media data can offer insights for offline analytics and derivative studies for future planning scenarios, the critical issue for frontline emergency responders is the organization and coordination of actionable, real-time data related to disaster situations.

User anonymity in the reporting process was embedded within the Peta Jakarta project. Whilst the data produced by Twitter reports of flooding is in the public domain, the objective was not to create an archive of users who submitted potentially sensitive reports about flooding events, outside of the Twitter platform. Peta Jakarta was thus designed to anonymize reports collected by separating reports from their respective users. Furthermore, the text content of tweets is only stored when the report is confirmed, that is, when the user has opted to send a message to the @petajkt account to describe their situation. Similarly, when usernames are stored, they are encrypted using a one-way hash function.

In developing the Peta Jakarta brand as the public face of the project, it was important to ensure that the interface and map were presented as community-owned, rather than as a government product or academic research tool. Aiming to appeal to first adopters—the young, tech-savvy Twitter-public of Jakarta—the language used in all the outreach materials (Twitter replies, the outreach video, graphics, and print advertisements) was intentionally casual and concise. Because of the repeated recurrence of flood events during the monsoon, and the continuation of daily activities around and through these flood events, the messages were intentionally designed to be more like normal twitter chatter and less like public service announcements.

It was important to design the user interaction with PetaJakarta.org to create a user experience that highlighted the community resource element of the project (similar to the Waze traffic app), rather than an emergency or information service. With this aim in mind, the graphics and language are casual and light in tone. In the video, auto-replies, and print advertisements, PetaJakarta.org never used alarmist or moralizing language; instead, the graphic identity is one of casual, opt-in, community participation.

The most frequent question directed to @petajkt on Twitter was about how to activate the geo-location function for tweets. So far, this question has been addressed manually by sending a reply tweet with a graphic instruction describing how to activate geo-location functionality.

Critical to the success of the project was its official public launch with, and promotion by, the Governor. This endorsement gave the platform very high visibility and increased legitimacy among other government agencies and public users; it also produced a very successful media event, which led substantial media coverage and subsequent public attention.

The aggregation of the tweets (designed to match the spatio-temporal structure of flood reporting in the system of the Jakarta Disaster Management Agency) was still inadequate when looking at social media because it could result in their overlooking reports that occurred in areas of especially low Twitter activity. Instead, the Agency used the @petajkt Twitter stream to direct their use of the map and to verify and cross-check information about flood-affected areas in real-time. While this use of social media was productive overall, the findings from the Joint Pilot Study have led to the proposal for the development of a more robust Risk Evaluation Matrix (REM) that would enable Peta Jakarta to serve a wider community of users & optimize the data collection process through an open API.

Developing a more robust integration of social media data also means leveraging other potential data sets to increase the intelligence produced by the system through hybridity; these other sources could include, but are not limited to, government, private sector, and NGO applications (‘apps’) for on- the-ground data collection, LIDAR or UAV-sourced elevation data, and fixed ground control points with various types of sensor data. The “citizen-as- sensor” paradigm for urban data collection will advance most effectively if other types of sensors and their attendant data sources are developed in concert with social media sourced information.

3 Comments

Posted in Big Data, Crisis Mapping, Crowdsourcing, Disaster Resilience, Humanitarian Technologies, Social Computing, Social Media

Tagged flooding, Indonesia, Jakarta, PetaJakarta, Twitter

A Force for Good: How Digital Jedis are Responding to the Nepal Earthquake (Updated)

Posted on April 27, 2015 | 18 comments

Digital Humanitarians are responding in full force to the devastating earthquake that struck Nepal. Information sharing and coordination is taking place online via CrisisMappers and on multiple dedicated Skype chats. The Standby Task Force (SBTF), Humanitarian OpenStreetMap (HOT) and others from the Digital Humanitarian Network (DHN) have also deployed in response to the tragedy. This blog post provides a quick summary of some of these digital humanitarian efforts along with what’s coming in terms of new deployments.

Update: A list of Crisis Maps for Nepal is available below.

At the request of the UN Office for the Coordination of Humanitarian Affairs (OCHA), the SBTF is using QCRI’s MicroMappers platform to crowdsource the analysis of tweets and mainstream media (the latter via GDELT) to rapidly 1) assess disaster damage & needs; and 2) Identify where humanitarian groups are deploying (3W’s). The MicroMappers CrisisMaps are already live and publicly available below (simply click on the maps to open live version). Both Crisis Maps are being updated hourly (at times every 15 minutes). Note that MicroMappers also uses both crowdsourcing and Artificial Intelligence (AIDR).

Update: More than 1,200 Digital Jedis have used MicroMappers to sift through a staggering 35,000 images and 7,000 tweets! This has so far resulted in 300+ relevant pictures of disaster damage displayed on the Image Crisis Map and over 100 relevant disaster tweets on the Tweet Crisis Map.

Live CrisisMap of pictures from both Twitter and Mainstream Media showing disaster damage:

Live CrisisMap of Urgent Needs, Damage and Response Efforts posted on Twitter:

Note: the outstanding Kathmandu Living Labs (KLL) team have also launched an Ushahidi Crisis Map in collaboration with the Nepal Red Cross. We’ve already invited invited KLL to take all of the MicroMappers data and add it to their crisis map. Supporting local efforts is absolutely key.

The Humanitarian UAV Network (UAViators) has also been activated to identify, mobilize and coordinate UAV assets & teams. Several professional UAV teams are already on their way to Kathmandu. The UAV pilots will be producing high resolution nadir imagery, oblique imagery and 3D point clouds. UAViators will be pushing this imagery to both HOT and MicroMappers for rapid crowdsourced analysis (just like was done with the aerial imagery from Vanuatu post Cyclone Pam, more on that here). A leading UAV manufacturer is also donating several UAVs to UAViators for use in Nepal. These UAVs will be sent to KLL to support their efforts. In the meantime, DigitalGlobe, Planet Labs and SkyBox are each sharing their satellite imagery with CrisisMappers, HOT and others in the Digital Humanitarian Network.

There are several other efforts going on, so the above is certainly not a complete list but simply reflect those digital humanitarian efforts that I am involved in or most familiar with. If you know of other major efforts, then please feel free to post them in the comments section. Thank you. More on the state of the art in digital humanitarian action in my new book, Digital Humanitarians.

List of Nepal Crisis Maps

Please add to the list below by posting new links in this Google Spreadsheet. Also, someone should really create 1 map that pulls from each of the listed maps.

Code for Nepal Casualty Crisis Map:
http://bit.ly/1IpUi1f

DigitalGlobe Crowdsourced Damage Assessment Map:
http://goo.gl/bGyHTC

Disaster OpenRouteService Map for Nepal:
http://www.openrouteservice.org/disaster-nepal

ESRI Damage Assessment Map:
http://arcg.is/1HVNNEm

Harvard WorldMap Tweets of Nepal:
http://worldmap.harvard.edu/maps/nepalquake

Humanitarian OpenStreetMap Nepal:
http://www.openstreetmap.org/relation/184633

Kathmandu Living Labs Crowdsourced Crisis Map: http://www.kathmandulivinglabs.org/earthquake

MicroMappers Disaster Image Map of Damage:
http://maps.micromappers.org/2015/nepal/images/#close

MicroMappers Disaster Damage Tweet Map of Needs:
http://maps.micromappers.org/2015/nepal/tweets

NepalQuake Status Map:
http://www.nepalquake.org/status-map

UAViators Crisis Map of Damage from Aerial Pics/Vids:
http://uaviators.org/map (takes a while to load)

Visions SDSU Tweet Crisis Map of Nepal:
http://vision.sdsu.edu/ec2/geoviewer/nepal-kathmandu#

18 Comments

Posted in Big Data, Crisis Mapping, Drones/UAVs, Humanitarian Technologies, Satellite Imagery, Social Computing, Social Media

Tagged Digital, Disaster, earthquake, Humanitarians, Nepal, Response

Artificial Intelligence for Monitoring Elections (AIME)

Posted on April 13, 2015 | Leave a comment

I published a blog post with the same title a good while back. Here’s what I wrote at the time:

Citizen-based, crowdsourced election observation initiatives are on the rise. Leading election monitoring organizations are also looking to leverage citizen-based reporting to complement their own professional election monitoring efforts. Meanwhile, the information revolution continues apace, with the number of new mobile phone subscriptions up by over 1 billion in just the past 36 months alone. The volume of election-related reports generated by “the crowd” is thus expected to grow significantly in the coming years. But international, national and local election monitoring organizations are completely unprepared to deal with the rise of Big (Election) Data.

I thus introduced a new project to “develop a free and open source platform to automatically filter relevant election reports from the crowd.” I’m pleased to report that my team and I at QCRI have just tested AIME during an actual election for the very first time—the 2015 Nigerian Elections. My QCRI Research Assistant Peter Mosur (co-author of this blog post) collaborated directly with Oludotun Babayemi from Clonehouse Nigeria and Chuks Ojidoh from the Community Life Project & Reclaim Naija to deploy and test the AIME platform.

AIME is a free and open source (experimental) solution that combines crowd-sourcing with Artificial Intelligence to automatically identify tweets of interest during major elections. As organizations engaged in election monitoring well know, there can be a lot chatter on social media as people rally behind their chosen candidates, announce this to the world, ask their friends and family who they will be voting for, and updating others when they have voted while posting about election related incidents they may have witnessed. This can make it rather challenging to find reports relevant to election monitoring groups.

Election monitors typically monitor instances of violence, election rigging, and voter issues. These incidents are monitored because they reveal problems that arise with the elections. Election monitoring initiatives such as Reclaim Naija & Uzabe also monitor several other type of incidents but for the purposes of testing the AIME platform, we selected three types of events mentioned above. In order to automatically identify tweets related to these events, one must first provide AIME with example tweets. (Of course, if there is no Twitter traffic to begin with, then there won’t be much need for AIME, which is precisely why we developed an SMS extension that can be used with AIME).

So where does the crowdsourcing comes in? Users of AIME can ask the crowd to tag tweets related to election-violence, rigging and voter issues by simply clicking on tagging tweets posted to the AIME platform with the appropriate event type. (Several quality control mechanisms are built in to ensure data quality. Also, one does not need to use crowdsourcing to tag the tweets; this can be done internally as well or instead). What AIME does next is use a technique from Artificial Intelligence (AI) called statistical machine learning to understand patterns in the human-tagged tweets. In other words, it begins to recognize which tweets belong in which category type—violence, rigging and voter issues. AIME will then auto-classify new tweets that are related to these categories (and can auto-classify around 2 millions tweets or text messages per minute).

Before creating our automatic classifier for the Nigerian Elections, we first needed to collect examples of tweets related to election violence, rigging and voter issues in order to teach AIME. Oludotun Babayemi and Chuks Ojidoh kindly provided the expert local knowledge needed to identify the keywords we should be following on Twitter (using AIME). They graciously gave us many different keywords to use as well as a list of trusted Twitter accounts to follow for election-related messages. (Due to difficulties with AIME, we were not able to use the trusted accounts. In addition, many of the suggested keywords were unusable since words like “aggressive”, “detonate”, and “security” would have resulted in large amount of false positives).

Here is the full list of keywords used by AIME:

Nigeria elections, nigeriadecides, Nigeria decides, INEC, GEJ, Change Nigeria, Nigeria Transformation, President Jonathan, Goodluck Jonathan, Sai Buhari, saibuhari, All progressives congress, Osibanjo, Sambo, Peoples Democratic Party, boko haram, boko, area boys, nigeria2015, votenotfight, GEJwinsit, iwillvoteapc, gmb2015, revoda, thingsmustchange, and march4buhari

Out of this list, “NigeriaDecides” was by far the most popular keyword used in the elections. It accounted for over 28,000 Tweets of a batch of 100,000. During the week leading up to the elections, AIME collected roughly 800,000 Tweets. Over the course of the elections and the few days following, the total number of collected Tweets jumped to well over 4 million.

We sampled just a handful of these tweets and manually tagged those related to violence, rigging and other voting issues using AIME. “Violence” was described as “threats, riots, arming, attacks, rumors, lack of security, vandalism, etc.” while “Election Rigging” was described as “Ballot stuffing, issuing invalid ballot papers, voter impersonation, multiple voting, ballot boxes destroyed after counting, bribery, lack of transparency, tampered ballots etc.” Lastly, “Voting Issues” was defined as “Polling station logistics issues, technical issues, people unable to vote, media unable to enter, insufficient staff, lack of voter assistance, inadequate voting materials, underage voters, etc.”

Any tweet that did not fall into these three categories was tagged as “Other” or “Not Related”. Our Election Classifiers were trained with a total of 571 human-tagged tweets which enabled AIME to automatically classify well over 1 million tweets (1,263,654 to be precise). The results in the screenshot below show accurate AIME was at auto-classifying tweets based on the different event types define earlier. AUC is what captures the “overall accuracy” of AIME’s classifiers.

AIME was rather good at correctly tagging tweets related to “Voting Issues” (98% accuracy) but drastically poor at tagging related to “Election Rigging” (0%). This is not AIME’s fault : ) since it only had 8 examples to learn from. As for “Violence”, the accuracy score was 47%, which is actually surprising given that AIME only had 14 human-tagged examples to learn from. Lastly, AIME did fairly well at auto-classifying unrelated tweets (accuracy of 86%).

Conclusion: this was the first time we tested AIME during an actual election and we’ve learned a lot in the process. The results are not perfect but enough to press on and experiment further with the AIME platform. If you’d like to test AIME yourself (and if you fully recognize that the tool is experimental and still under development, hence not perfect), then feel free to get in touch with me here. We have 2 slots open for testing. In the meantime, big thanks to my RA Peter for spearheading both this deployment and the subsequent research.

Artificial Intelligence Powered by Crowdsourcing: The Future of Big Data and Humanitarian Action

Posted on March 16, 2015 | 7 comments

There’s no point spewing stunning statistics like this recent one from The Economist, which states that 80% of adults will have access to smartphones before 2020. The volume, velocity and variety of digital data will continue to skyrocket. To paraphrase Douglas Adams, “Big Data is big. You just won’t believe how vastly, hugely, mind-bogglingly big it is.”

And so, traditional humanitarian organizations have a choice when it comes to battling Big Data. They can either continue business as usual (and lose) or get with the program and adopt Big Data solutions like everyone else. The same goes for Digital Humanitarians. As noted in my new book of the same title, those Digital Humanitarians who cling to crowdsourcing alone as their pièce de résistance will inevitably become the ivy-laden battlefield monuments of 2020.

Big Data comprises a variety of data types such as text, imagery and video. Examples of text-based data includes mainstream news articles, tweets and WhatsApp messages. Imagery includes Instagram, professional photographs that accompany news articles, satellite imagery and increasingly aerial imagery as well (captured by UAVs). Television channels, Meerkat and YouTube broadcast videos. Finding relevant, credible and actionable pieces of text, imagery and video in the Big Data generated during major disasters is like looking for a needle in a meadow (haystacks are ridiculously small datasets by comparison).

Humanitarian organizations, like many others in different sectors, often find comfort in the notion that their problems are unique. Thankfully, this is rarely true. Not only is the Big Data challenge not unique to the humanitarian space, real solutions to the data deluge have already been developed by groups that humanitarian professionals at worst don’t know exist and at best rarely speak with. These groups are already using Artificial Intelligence (AI) and some form of human input to make sense of Big Data.

How does it work? And why do you still need some human input if AI is already in play? The human input, which can be via crowdsourcing or a few individuals is needed to train the AI engine, which uses a technique from AI called machine learning to learn from the human(s). Take AIDR, for example. This experimental solution, which stands for Artificial Intelligence for Disaster Response, uses AI powered by crowdsourcing to automatically identify relevant tweets and text messages in an exploding meadow of digital data. The crowd tags tweets and messages they find relevant and the AI engine learns to recognize the relevance patterns in real-time, allowing AIDR to automatically identify future tweets and messages.

As far as we know, AIDR is the only Big Data solution out there that combines crowdsourcing with real-time machine learning for disaster response. Why do we use crowdsourcing to train the AI engine? Because speed is of the essence in disasters. You need a crowd of Digital Humanitarians to quickly tag as many tweets/messages as possible so that AIDR can learn as fast as possible. Incidentally, once you’ve created an algorithm that accurately detects tweets relaying urgent needs after a Typhoon in the Philippines, you can use that same algorithm again when the next Typhoon hits (no crowd needed).

What about pictures? After all, pictures are worth a thousand words. Is it possible to combine artificial intelligence with human input to automatically identify pictures that show infrastructure damage? Thanks to recent break-throughs in computer vision, this is indeed possible. Take Metamind, for example, a new startup I just met with in Silicon Valley. Metamind is barely 6 months old but the team has already demonstrated that one can indeed automatically identify a whole host of features in pictures by using artificial intelligence and some initial human input. The key is human input since this is what trains the algorithms. The more human-generated training data you have, the better your algorithms.

My team and I at QCRI are collaborating with Metamind to create algorithms that can automatically detect infrastructure damage in pictures. The Silicon Valley start-up is convinced that we’ll be able to create a highly accurate algorithms if we have enough training data. This is where MicroMappers comes in. We’re already using MicroMappers to create training data for tweets and text messages (which is what AIDR uses to create algorithms). In addition, we’re already using MicroMappers to tag and map pictures of disaster damage. The missing link—in order to turn this tagged data into algorithms—is Metamind. I’m excited about the prospects, so stay tuned for updates as we plan to start teaching Metamind’s AI engine this month.

How about videos as a source of Big Data during disasters? I was just in Austin for SXSW 2015 and met up with the CEO of WireWax, a British company that uses—you guessed it—artificial intelligence and human input to automatically detect countless features in videos. Their platform has already been used to automatically find guns and Justin Bieber across millions of videos. Several other groups are also working on feature detection in videos. Colleagues at Carnegie Melon University (CMU), for example, are working on developing algorithms that can detect evidence of gross human rights violations in YouTube videos coming from Syria. They’re currently applying their algorithms on videos of disaster footage, which we recently shared with them, to determine whether infrastructure damage can be automatically detected.

What about satellite & aerial imagery? Well the team driving DigitalGlobe’s Tomnod platform have already been using AI powered by crowdsourcing to automatically identify features of interest in satellite (and now aerial) imagery. My team and I are working on similar solutions with MicroMappers, with the hope of creating real-time machine learning solutions for both satellite and aerial imagery. Unlike Tomnod, the MicroMappers platform is free and open source (and also filters social media, photographs, videos & mainstream news).

So there you have it. The future of humanitarian information systems will not be an App Store but an “Alg Store”, i.e, an Algorithm Store providing a growing menu of algorithms that have already been trained to automatically detect certain features in texts, imagery and videos that gets generated during disasters. These algorithms will also “talk to each other” and integrate other feeds (from real-time sensors, Internet of Things) thanks to data-fusion solutions that already exist and others that are in the works.

Now, the astute reader may have noted that I omitted audio/speech in my post. I’ll be writing about this in a future post since this one is already long enough.

7 Comments

Posted in Big Data, Crowdsourcing, Humanitarian Technologies, Satellite Imagery, Social Computing, Social Media

Tagged AI, Artificial, Intelligence, Learning, Machine

How to Become a Digital Sherlock Holmes and Support Relief Efforts

Posted on February 26, 2015 | 2 comments

Humanitarian organizations need both timely and accurate information when responding to disasters. Where is the most damage located? Who needs the most help? What other threats exist? Respectable news organizations also need timely and accurate information during crisis events to responsibly inform the public. Alas, both humanitarian & mainstream news organizations are often confronted with countless rumors and unconfirmed reports. Investigative journalists and others have thus developed a number of clever strategies to rapidly verify such reports—as detailed in the excellent Verification Handbook. There’s just one glitch: Journalists and humanitarians alike are increasingly overwhelmed by the “Big Data” generated during crises, particularly information posted on social media. They rarely have enough time or enough staff to verify the majority of unconfirmed reports. This is where Verily comes in, a new type of Detective Agency for a new type of detective: The Virtual Digital Detective.

The purpose of Verily is to rapidly crowdsource the verification of unconfirmed reports during major disasters. The way it works is simple. If a humanitarian or news organization has a verification request, they simply submit this request online at Verily. This request must be phrased in the form of a Yes-or-No question, such as: “Has the Brooklyn Bridge been destroyed by the Hurricane?”; “Is this Instagram picture really showing current flooding in Indonesia”?; “Is this new YouTube video of the Chile earthquake fake?”; “Is it true that the bush fires in South Australia are getting worse?” and so on.

Verily helps humanitarian & news organizations find answers to these questions by rapidly crowdsourcing the collection of clues that can help answer said questions. Verification questions are communicated widely across the world via Verily’s own email-list of Digital Detectives and also via social media. This new bread of Digital Detectives then scour the web for clues that can help answer the verification questions. Anyone can become a Digital Detective at Verily. Indeed, Verily provides a menu of mini-verification guides for new detectives. These guides were written by some of the best Digital Detectives on the planet, the authors of the Verification Handbook. Verily Detectives post the clues they find directly to Verily and briefly explain why these clues help answer the verification question. That’s all there is to it.

If you’re familiar with Reddit, you may be thinking “Hold on, doesn’t Reddit do this already?” In part yes, but Reddit is not necessarily designed to crowdsource critical thinking or to create skilled Digital Detectives. Recall this fiasco during the Boston Marathon Bombings which fueled disastrous “witch hunts”. Said disaster would not have happened on Verily because Verily is deliberately designed to focus on the process of careful detective work while providing new detectives with the skills they need to precisely avoid the kind of disaster that happened on Reddit. This is no way a criticism of Reddit! One single platform alone cannot be designed to solve every problem under the sun. Deliberate, intentional design is absolutely key.

In sum, our goal at Verily is to crowdsource Sherlock Holmes. Why do we think this will work? For several reasons. First, authors of the Verification Handbook have already demonstrated that individuals working alone can, and do, verify unconfirmed reports during crises. We believe that creating a community that can work together to verify rumors will be even more powerful given the Big Data challenge. Second, each one of us with a mobile phone is a human sensor, a potential digital witness. We believe that Verily can help crowdsource the search for eyewitnesses, or rather the search for digital content that these eyewitnesses post on the Web. Third, the Red Balloon Challenge was completed in a matter of hours. This Challenge focused on crowdsourcing the search for clues across an entire continent (3 million square miles). Disasters, in contrast, are far more narrow in terms of geographic coverage. In other words, the proverbial haystack is smaller and thus the needles easier to find. More on Verily here & here.

So there’s reason to be optimistic that Verily can succeed given the above and recent real-world deployments. Of course, Verily is is still very much in early phase and still experimental. But both humanitarian organizations and high-profile news organizations have expressed a strong interest in field-testing this new Digital Detective Agency. To find out more about Verily and to engage with experts in verification, please join us on Tuesday, March 3rd at 10:00am (New York time) for this Google Hangout with the Verily Team and our colleague Craig Silverman, the Co-Editor of the Verification Handbook. Click here for the Event Page and here to follow on YouTube. You can also join the conversations on Twitter and pose questions or comments using the hashtag #VerilyLive.

2 Comments

Posted in Crowdsourcing, Humanitarian Technologies, Information Forensics, Social Media

Tagged rumors, verification, Verily

Category Archives: Social Media

Listening to Humanitarian Needs

Facebook Safety Check

Facebook Location Maps

Moving Forward

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

List of Nepal Crisis Maps

Share this:

Share this:

Share this:

Share this:

Patrick Meier, PhD

Table of Contents