Category Archives: Crowdsourcing

Using Rayesna to Track the 2012 Egyptian Presidential Candidates on Twitter

My (future) colleague at the Qatar Foundation’s Computing Research Institute (QCRI) have just launched a new platform that Al Jazeera is using to track the 2012 Egyptian Presidential Candidates on Twitter. Called Rayesna, which  means “our president” in colloquial Egyptian Arabic, this fully automated platform uses cutting-edge Arabic computational linguistics processing developed by the Arabic Language Technology (ALT) group at QCRI.

“Through Rayesna, you can find out how many times a candidate is mentioned, which other candidate he is likely to appear with, and the most popular tweets for a candidate, with a special category for the most retweeted jokes about the candidates. The site also has a time-series to explore and compares the mentions of the candidate day-by-day. Caveats: 1. The site reflects only the people who choose to tweet, and this group may not be representative of general society; 2. Tweets often contain foul language and we do not perform any filtering.”

I look forward to collaborating with the ALT group and exploring how their platform might also be used in the context of humanitarian response in the Arab World and beyond. There may also be important synergies with the work of the UN Global Pulse, particularly vis-a-vis their use of Twitter for real-time analysis of vulnerable communities.

From Gunfire at Sea to Maps of War: Implications for Humanitarian Innovation

MIT Professor Eric von Hippel is the author of Democratizing Innovation, a book I should have read when it was first published seven years ago. The purpose of this blog post, however, is to share some thoughts on “Gunfire at Sea: A Case Study in Innovation” (PDF), which Eric recently instructed me to read. Authored by Elting Morison in 1968, this piece is definitely required reading for anyone engaged in disruptive innovation, particularly in the humanitarian space. Morison was one of the most distinguished historians of the last century and the founder of MIT’s Program in Science, Technology and Society (STS). The Boston Globe called him “an educator and industrial historian who believed that technology could only be harnessed to serve human beings when scientists and poets could meet with mutual understanding.”

Morison details in intriguing fashion the challenges of using light artillery at sea in the late 1,800’s to illustrate how new technologies and new forms of power collide and indeed, “bombard the fixed structure of our habits of mind and behavior.” The first major innovative disruption in naval gunfire technology is the result of one person’s acute observation. Admiral Sir Percy Scott happened to watched his men during target practice one day while the ship they were on was pitching and rolling acutely due to heavy weather. The resulting accuracy of the shots was dismal save for one man who was doing something slightly different to account for the swaying. Scott observed this positive deviance carefully and cobbled existing to technology to render the strategy easier to repeat and replicate. Within a year, his gun crews were remarkable accurate.

Note that Scott was not responsible for the invention of the basic instruments he cobbled together to scale the positive deviance he observed. Scott’s contribution, rather, was  a mashup of existing technology made possible thanks to mechanical ingenuity and a keen eye for behavioral processes. As for the personality of the innovator, Scott possessed “a savage indignation directed ordinarily at the inelastic intelligence of all constituted authority, especially the British Admiralty.” Chance also plays a role in this story. “Fortune (in this case, the unaware gun pointer) indeed favors the prepared mind, but even fortune and the prepared mind need a favorable environment before they can conspire to produce sudden change. No intelligence can proceed very far above the threshold of existing data or the binding combinations of existing data.”

Whilst stationed in China several years later, Admiral Scott crosses paths with William Sims, an American Junior Officer of similar temperament. Sims’s efforts to reform the naval service are perhaps best told in his own words: “I am perfectly willing that those holding views differing from mine should continue to live, but with every fibre of my being I loathe indirection and shiftiness, and where it occurs in high place, and is used to save face at the expense of the vital interests of our great service (in which silly people place such a child-like trust), I want that man’s blood and I will have it no matter what it costs me personally.” Sims built on Scott’s inventions and made further modifications, resulting in new records in accuracy. “These elements were brought into successful combination by minds not interested in the instruments for themselves but in what they could do with them.”

“Sure of the usefulness of his gunnery methods, Sims then turned to the task of educating the Navy at large.” And this is where the fun really begins. His first strategy was to relay in writing the results of his methods “with a mass of factual data.” Sims authored over a dozen detailed data-driven reports on innovations in naval gunfire strage which he sent from his China Station to the powers that be in Washington DC. At first, there was no response from DC. Sims thus decided to change his tone by using deliberately shocking language in subsequent reports. Writes Sims: “I therefore made up my mind I would give these later papers such a form that they would be dangerous documents to leave neglected in the files.” Sims also decided to share his reports with other officers in the fleet to force a response from the men in Washington.

The response, however, was not exactly what Sims had hoped. Washington’s opinion was that American technology was generally as good as the British, which implied that the trouble was with the men operating the technology, which thus meant that ship officers ought to conduct more training. What probably annoyed Sims most, however, was Washington’s comments vis-a-vis the new records in accuracy that Sims claimed to have achieved. Headquarters simply waived these off as impossible. So while the first reaction was dead silence, DC’s second strategy was to try and “meet Sims’s claims by logical, rational rebuttal.”

I agree with the author, Elting Morison, that this second stage reaction, “the apparent resort to reason,” is the “most entertaining and instructive in our investigation of the responses to innovation.” That said, the third stage, name-calling, can be just as entertaining for some, and Sims took the argumentum ad hominem as evidence that “he was being attacked by shifty, dishonest men who were the victims, as he said, of insufferable conceit and ignorance.” He thus took the extraordinary step of writing directly to the President of the United States, Theodore Roosevelt, to inform him of the remarkable achievements in accuracy that he and Admiral Scott had achieved. “Roosevelt, who always liked to respond to such appeals when he conveniently could, brought Sims back from China late in 1902 and installed him as Inspector of Target Practice […]. And when he left, after many spirited encounters […], he was universally acclaimed as ‘the man who taught us how to shoot.'”

What fascinates Morison in this story is the concerted resistance triggered by Sims’s innovation. Why so much resistance? Morison identifies three main sources: “honest disbelief in the dramatic but substantiated claims of the new process; protection of the existing devices and instruments with which they identified themselves; and maintenance of the existing society with which they were identified.” He argues that the latter explanation is the most important, i.e., resistance due to the “fixed structure of our habits of mind and behavior” and the fact that relatively small innovations in gunfire accuracy could quite conceivably unravel the entire fabric of naval doctrine. Indeed,

“From changes in gunnery flowed an extraordinary complex of changes: in shipboard routines, ship design, and fleet tactics. There was, too, a social change. In the days when gunnery was taken lightly, the gunnery officer was taken lightly. After 1903, he became one of the most significant and powerful members of a ship’s company, and this shift of emphasis nat- urally was shortly reflected in promotion lists. Each one of these changes provoked a dislocation in the naval society, and with man’s troubled foresight and natural indisposition to break up classic forms, the men in Washington withstood the Sims onslaught as long as they could. It is very significant that they withstood it until an agent from outside, outside and above, who was not clearly identified with the naval society, entered to force change.”

The resistance to change thus “springs from the normal human instinct to protect oneself, and more especially, one’s way of life.” Interestingly, the deadlock between those who sought change and those who sought to retain things as they were was broken only by an appeal to superior force, a force removed from and unidentified with the mores, conventions, devices of the society. This seems to me a very important point.”  The appeal to Roosevelt suggests perhaps that no organization “should or can undertake to reform itself. It must seek assistance from outside.”

I am absolutely intrigued by what these insights might imply vis-a-vis innovation (and resistance to innovation) in the humanitarian sector. Whether it be the result of combining existing technologies to produce open-source crisis mapping platforms or the use of new information management processes such as crowdsourcing, is concerted resistance to such innovation in the humanitarian space inevitable as well? Do we have a Roosevelt equivalent, i..e, an external and somewhat independent actor who might disrupt the resistance? I can definitely trace the same stages of resistance to innovations in humanitarian technology as those identified by Morison: (1) dead silence; (2) reasoned dismissal; and (3) name-calling. But as Morison himself is compelled to ask: “How then can we find the means to accept with less pain to ourselves and less damage to our social organization the dislocations in our society that are produced by innovation?”

This question, or rather Morison’s insights in tackling this question are profound and have important implications vis-a-vis innovation in the humanitarian space. Morison hones in on the imperative of “identification” in innovation:

“It cannot have escaped notice that some men identified themselves with their creations- sights, gun, gear, and so forth-and thus obtained a presumed satisfaction from the thing itself, a satisfaction that prevented them from thinking too closely on either the use or the defects of the thing; that others identified themselves with a settled way of life they had inherited or accepted with minor modification and thus found their satisfaction in attempting to maintain that way of life unchanged; and that still others identified themselves as rebellious spirits, men of the insurgent cast of mind, and thus obtained a satisfaction from the act of revolt itself.”

This purely personal identification is a powerful barrier to innovation. So can this identifying process be tampered in order to facilitate change that is ultima-tely in everyone’s interest? Morison recommends that we “spend some time and thought on the possibility of enlarging the sphere of our identifications from the part to the whole.” In addition, he suggests an emphasis on process rather than product. If we take this advice to heart, what specific changes should we seek to make in the humanitarian technology space? How do we enlarge the sphere of our identifications and in doing so focus on processes rather than products? There’s no doubt that these are major challenges in and of themselves, but ignoring them may very well mean that important innovations in life-saving technologies and processes will go un-adopted by large humanitarian organiza-tions for many years to come.

Predicting the Future of Global Geospatial Information Management

The United Nations Committee of Experts on Global Information Management (GGIM) recently organized a meeting of thought-leaders and visionaries in the geo-spatial world to identify the future of this space over the next 5-10 years. These experts came up with some 80+ individual predictions. I’ve included some of the more interesting ones below.

  • The use of Unmanned Aerial Vehicles (UAVs) as a tool for rapid geospatial data collection will increase.
  • 3D and even 4D geospatial information, incorporating time as the fourth dimension, will increase.
  • Technology will move faster than legal and governance structures.
  • The link between geospatial information and social media, plus other actor networks, will become more and more important.
  • Real-time info will enable more dynamic modeling & response to disasters.
  • Free and open source software will continue to grow as viable alternatives both in terms of software, and potentially in analysis and processing.
  • Geospatial computation will increasingly be non-human consumable in nature, with an increase in fully-automated decision systems.
  • Businesses and Governments will increasingly invest in tools and resources to manage Big Data. The technologies required for this will enable greater use of raw data feeds from sensors and other sources of data.
  • In ten years time it is likely that all smart phones will be able to film 360 degree 3D video at incredibly high resolution by today’s standards & wirelessly stream it in real time.
  • There will be a need for geospatial use governance in order to discern the real world from the virtual/modelled world in a 3D geospatial environ-ment.
  • Free and open access to data will become the norm and geospatial information will increasingly be seen as an essential public good.
  • Funding models to ensure full data coverage even in non-profitable areas will continue to be a challenge.
  • Rapid growth will lead to confusion and lack of clarity over data ownership, distribution rights, liabilities and other aspects.
  • In ten years, there will be a clear dividing line between winning and losing nations, dependent upon whether the appropriate legal and policy frameworks have been developed that enable a location-enabled society to flourish.
  • Some governments will use geospatial technology as a means to monitor or restrict the movements and personal interactions of their citizens. Individuals in these countries may be unwilling to use LBS or applications that require location for fear of this information being shared with authorities.
  • The deployment of sensors and the broader use of geospatial data within society will force public policy and law to move into a direction to protect the interests and rights of the people.
  • Spatial literacy will not be about learning GIS in schools but will be more centered on increasing spatial awareness and an understanding of the value of understanding place as context.
  • The role of National Mapping Agencies as an authoritative supplier of high quality data and of arbitrator of other geospatial data sources will continue to be crucial.
  • Monopolies held by National Mapping Agencies in some areas of specialized spatial data will be eroded completely.
  • More activities carried out by National Mapping Agencies will be outsourced and crowdsourced.
  • Crowdsourced data will push National Mapping Agencies towards niche markets.
  • National Mapping Agencies will be required to find new business models to provide simplified licenses and meet the demands for more free data from mapping agencies.
  • The integration of crowdsourced data with government data will increase over the next 5 to 10 years.
  • Crowdsourced content will decrease cost, improve accuracy and increase availability of rich geospatial information.
  •  There will be increased combining of imagery with crowdsourced data to create datasets that could not have been created affordably on their own.
  • Progress will be made on bridging the gap between authoritative data and crowdsourced data, moving towards true collaboration.
  • There will be an accelerated take-up of Volunteer Geographic Information over the next five years.
  • Within five years the level of detail on transport systems within OpenStreetMap will exceed virtually all other data sources & will be respected/used by major organisations & governments across the globe.
  • Community-based mapping will continue to grow.
  • There is unlikely to be a market for datasets like those currently sold to power navigation and location-based services solutions in 5 years, as they will have been superseded by crowdsourced datasets from OpenStreetMaps or other comparable initiatives.

Which trends have the experts missed? Do you think they’re completely off on any of the above? The full set of predictions on the future of global geospatial information management is available here as a PDF.

Building Egypt 2.0: When Institutions Fail, Crowdsourcing Surges

I recently presented at Where 2.0 and had the chance to catch Adel Youssef’s excellent talk on “How Location Based Services is Used to Build Egypt 2.0.” He shared some important gems on digital activism. For example, while Facebook allowed Egyptians to “like” a protest event or say they were headed to the streets, check-in’s were a more powerful way to recruit others because they let your friends know that you were actively in the location and actually protesting. In other words, activists were not checking into a place per se, but rather creating an event and checking into that to encourage people to participate in said event.

Adel also shared some interesting insights on how location-aware mobile tech-nologies are being used to build a new Egypt. “After the revolution, the police force just disappeared, there is no police; and there is no traffic control. But this drove more crowdsourced traffic control, crowdsourced police, crowdsourced services. And this has been happening in the last year alone. Crowdsourcing revolution. But not a revolution to overthrow a tyrant but a revolution to build a developed country. […] People going to clean the streets, planting trees, repainting the streets. And they are feeling ownership of their campaign.”

Adel shared several other crowdsourcing initiatives in his talk, from OneYad (matching volunteers) and Zabatak (monitoring corruption) to EntaFeen (check-in’s for good), Bey2Ollak and Wasalny (both addressing the problem of road traffic). I’m excited by all this innovation happening elsewhere than Silicon Valley and hope these platforms will go mainstream beyond the region in the near future. Indeed, I just signed up for the OneYad beta because I really think this kind of tool could be used in the West.

Adel: “We see a lot of crowdsourced networks built after the revolution because we need to build the country and we want to do this bottom-up, want to do it by the people, you want to empower the people.” The point, for Adel, is to go “from social networking to social working” and thus fill the gaps in services that institutions are failing to provide. This reminded me of Tunisian Ambassador Mohamed Salah Tekaya’s remarks last year: “During the Arab Spring, we have seen the power of Twitter and Facebook… Now we need to use the power of LinkedIn.”

Behind the Scenes: The Digital Operations Center of the American Red Cross

The Digital Operations Center at the American Red Cross is an important and exciting development. I recently sat down with Wendy Harman to learn more about the initiative and to exchange some lessons learned in this new world of digital  humanitarians. One common challenge in emergency response is scaling. The American Red Cross cannot be everywhere at the same time—and that includes being on social media. More than 4,000 tweets reference the Red Cross on an average day, a figure that skyrockets during disasters. And when crises strike, so does Big Data. The Digital Operations Center is one response to this scaling challenge.

Sponsored by Dell, the Center uses customized software produced by Radian 6 to monitor and analyze social media in real-time. The Center itself sits three people who have access to six customized screens that relate relevant information drawn from various social media channels. The first screen below depicts some of key topical areas that the Red Cross monitors, e.g., references to the American Red Cross, Storms in 2012, and Delivery Services.

Circle sizes in the first screen depict the volume of references related to that topic area. The color coding (red, green and beige) relates to sentiment analysis (beige being neutral). The dashboard with the “speed dials” right underneath the first screen provides more details on the sentiment analysis.

Lets take a closer look at the circles from the first screen. The dots “orbiting” the central icon relate to the categories of key words that the Radian 6 platform parses. You can click on these orbiting dots to “drill down” and view the individual key words that make up that specific category. This circles screen gets updated in near real-time and draws on data from Twitter, Facebook, YouTube, Flickr and blogs. (Note that the distance between the orbiting dots and the center does not represent anything).

An operations center would of course not be complete without a map, so the Red Cross uses two screens to visualize different data on two heat maps. The one below depicts references made on social media platforms vis-a-vis storms that have occurred during the past 3 days.

The screen below the map highlights the bio’s of 50 individual twitter users who have made references to the storms. All this data gets generated from the “Engagement Console” pictured below. The purpose of this web-based tool, which looks a lot like Tweetdeck, is to enable the Red Cross to customize the specific types of information they’re looking form, and to respond accordingly.

Lets look at the Consul more closely. In the Workflow section on the left, users decide what types of tags they’re looking for and can also filter by priority level. They can also specify the type of sentiment they’re looking, e.g., negative feelings vis-a-vis a particular issue. In addition, they can take certain actions in response to each information item. For example, they can reply to a tweet, a Facebook status update, or a blog post; and they can do this directly from the engagement consul. Based on the license that the Red Cross users, up to 25 of their team members can access the Consul and collaborate in real-time when processing the various tweets and Facebook updates.

The Consul also allows users to create customized timelines, charts and wordl graphics to better understand trends changing over time in the social media space. To fully leverage this social media monitoring platform, Wendy and team are also launching a digital volunteers program. The goal is for these volunteers to eventually become the prime users of the Radian platform and to filter the bulk of relevant information in the social media space. This would considerably lighten the load for existing staff. In other words, the volunteer program would help the American Red Cross scale in the social media world we live in.

Wendy plans to set up a dedicated 2-hour training for individuals who want to volunteer online in support of the Digital Operations Center. These trainings will be carried out via Webex and will also be available to existing Red Cross staff.


As  argued in this previous blog post, the launch of this Digital Operations Center is further evidence that the humanitarian space is ready for innovation and that some technology companies are starting to think about how their solutions might be applied for humanitarian purposes. Indeed, it was Dell that first approached the Red Cross with an expressed interest in contributing to the organization’s efforts in disaster response. The initiative also demonstrates that combining automated natural language processing solutions with a digital volunteer net-work seems to be a winning strategy, at least for now.

After listening to Wendy describe the various tools she and her colleagues use as part of the Operations Center, I began to wonder whether these types of tools will eventually become free and easy enough for one person to be her very own operations center. I suppose only time will tell. Until then, I look forward to following the Center’s progress and hope it inspires other emergency response organizations to adopt similar solutions.

Twitcident: Filtering Tweets in Real-Time for Crisis Response

The most recent newcomer to the “tweetsourcing” space comes to us from Delft University of Technology in the Netherlands. Twitcident is a web-based filtering system that extracts crisis information from Twitter in real-time to support emergency response efforts. Dutch emergency services have been testing the platform over the past 10 months and results “show the system to be far more useful than simple keyword searching of a twitter feed” (NewScientist).

Here’s how it works. First the dashboard, which shows current events-of-interest being monitored.

Lets click on “Texas”, which produces the following page. More than 22,000 tweets potentially relate to the actual fire of interest.

This is where the filtering comes in:

The number of relevant tweets is reduced with every applied filter.

Naturally, geo-location is also an optional filter.

Twitcident also allows for various visualization options, including timelines, word clouds and charts.

The system also allows the user to view the filtered tweets on a map. The pictures and videos shared via twitter are also aggregated and viewable on dedicated tabs.

The developers of the platform have not revealed how their algorithms work but will demo the tool at the World Wide Web 2012 conference in France next week. In the meantime, here’s a graphic that summarizes the platform workflow.

I look forward to following Twitcident’s developments. I’d be particularly interested in learning more about how Dutch emergency services have been using the tool and what features they think would improve the platform’s added value.

Does the Humanitarian Industry Have a Future in The Digital Age?

I recently had the distinct honor of being on the opening plenary of the 2012 Skoll World Forum in Oxford. The panel, “Innovation in Times of Flux: Opportunities on the Heels of Crisis” was moderated by Judith Rodin, CEO of the Rockefeller Foundation. I’ve spent the past six years creating linkages between the humanitarian space and technology community, so the conversations we began during the panel prompted me to think more deeply about innovation in the humanitarian industry. Clearly, humanitarian crises have catalyzed a number of important innovations in recent years. At the same time, however, these crises extend the cracks that ultimately reveal the inadequacies of existing organiza-tions, particularly those resistant to change; and “any organization that is not changing is a battle-field monument” (While 1992).

These cracks, or gaps, are increasingly filled by disaster-affected communities themselves thanks in part to the rapid commercialization of communication technology. Question is: will the multi-billion dollar humanitarian industry change rapidly enough to avoid being left in the dustbin of history?

Crises often reveal that “existing routines are inadequate or even counter-productive [since] response will necessarily operate beyond the boundary of planned and resourced capabilities” (Leonard and Howitt 2007). More formally, “the ‘symmetry-breaking’ effects of disasters undermine linearly designed and centralized administrative activities” (Corbacioglu 2006). This may explain why “increasing attention is now paid to the capacity of disaster-affected communities to ‘bounce back’ or to recover with little or no external assistance following a disaster” (Manyena 2006).

But disaster-affected populations have always self-organized in times of crisis. Indeed, first responders are by definition those very communities affected by disasters. So local communities—rather than humanitarian professionals—save the most lives following a disaster (Gilbert 1998). Many of the needs arising after a disaster can often be met and responded to locally. One doesn’t need 10 years of work experience with the UN in Darfur or a Masters degree to know basic first aid or to pull a neighbor out of the rubble, for example. In fact, estimates suggest that “no more than 10% of survival in emergencies can be attributed to external sources of relief aid” (Hilhorst 2004).

This figure may be higher today since disaster-affected communities now benefit from radically wider access to information and communication technologies (ICTs). After all, a “disaster is first of all seen as a crisis in communicating within a community—that is as a difficulty for someone to get informed and to inform other people” (Gilbert 1998). This communication challenge is far less acute today because disaster-affected communities are increasingly digital, and thus more and more the primary source of information communicated following a crisis. Of course, these communities were always sources of information but being a source in an analog world is fundamentally different than being a source of information in the digital age. The difference between “read-only” versus “read-write” comes to mind as an analogy. And so, while humanitarian organiza-tions typically faced a vacuum of information following sudden onset disasters—limited situational awareness that could only be filled by humanitarians on the ground or via established news organizations—one of the major challenges today is the Big Data produced by disaster-affected communities themselves.

Indeed, vacuums are not empty and local communities are not invisible. One could say that disaster-affected communities are joining the quantified self (QS) movement given that they are increasingly quantifying themselves. If inform-ation is power, then the shift of information sourcing and sharing from the select few—the humanitarian professionals—to the masses must also engender a shift in power. Indeed, humanitarians rarely have access to exclusive information any longer. And even though affected populations are increasingly digital, some groups believe that humanitarian organizations have largely failed at commu–nicating with disaster-affected communities. (Naturally, there are important and noteworthy exceptions).

So “Will Twitter Put the UN Out of Business?” (Reuters), or will humanitarian organizations cope with these radical changes by changing themselves and reshaping their role as institutions before it’s too late? Indeed, “a business that doesn’t communicate with its customers won’t stay in business very long—it’ll soon lose track of what its clients want, and clients won’t know what products or services are on offer,” whilst other actors fill the gaps (Reuters). “In the multi-billion dollar humanitarian aid industry, relief agencies are businesses and their beneficiaries are customers. Yet many agencies have muddled along for decades with scarcely a nod towards communicating with the folks they’re supposed to be serving” (Reuters).

The music and news industries were muddling along as well for decades. Today, however, they are facing tremendous pressures and are undergoing radical structural changes—none of them by choice. Of course, it would be different if affected communities were paying for humanitarian services but how much longer do humanitarian organizations have until they feel similar pressures?

Whether humanitarian organizations like it or not, disaster affected communities will increasingly communicate their needs publicly and many will expect a response from the humanitarian industry. This survey carried out by the American Red Cross two years ago already revealed that during a crisis the majority of the public expect a response to needs they communicate via social media. Moreover, they expect this response to materialize within an hour. Humanitarian organizations simply don’t have the capacity to deal with this surge in requests for help, nor are they organizationally structured to do so. But the fact of the matter is that humanitarian organizations have never been capable of dealing with this volume of requests in the first place. So “What Good is Crowd-sourcing When Everyone Needs Help?” (Reuters). Perhaps “crowdsourcing” is finally revealing all the cracks in the system, which may not be a bad thing. Surely by now it is no longer a surprise that many people may be in need of help after a disaster, hence the importance of disaster risk reduction and preparedness.

Naturally, humanitarian organizations could very well chose to continue ignoring calls for help and decide that communicating with disaster affected communities is simply not tenable. In the analog world of the past, the humanitarian industry was protected by the fact that their “clients” did not have a voice because they could not speak out digitally. So the cracks didn’t show. Today, “many traditional humanitarian players see crowdsourcing as an unwelcome distraction at a time when they are already overwhelmed. They worry that the noise-to-signal ration is just too high” (Reuters). I think there’s an important disconnect here worth emphasizing. Crowdsourced information is simply user-generated content. If humanitarians are to ignore user-generated content, then they can forget about two-way communications with disaster-affected communities and drop all the rhetoric. On the other hand, “if aid agencies are to invest time and resources in handling torrents of crowdsourced information in disaster zones, they should be confident it’s worth their while” (Reuters).

This last comment is … rather problematic for several reasons (how’s that for being diplomatic?). First of all, this kind of statement continues to propel the myth that we the West are the rescuers and aid does not start until we arrive (Barrs 2006). Unfortunately, we rarely arrive: how many “neglected crises” and so-called “forgotten emergencies” have we failed to intervene in? This kind of mindset may explain why humanitarian interventions often have the “propensity to follow a paternalistic mode that can lead to a skewing of activities towards supply rather than demand” and towards informing at the expense of listening (Manyena 2006).

Secondly, the assumption that crowdsourced data would be for the exclusive purpose of the humanitarian cavalry is somewhat arrogant and ignores the reality that local communities are by definition the first responders in a crisis. Disaster-affected communities (and Diasporas) are already collecting (and yes crowdsourcing) information to create their own crisis maps in times of need as a forthcoming report shows. And they’ll keep doing this whether or not humanita-rian organizations approve or leverage that information. As my colleague Tim McNamara has noted “Crisis mapping is not simply a technological shift, it is also a process of rapid decentralization of power. With extremely low barriers to entry, many new entrants are appearing in the fields of emergency and disaster response. They are ignoring the traditional hierarchies, because the new entrants perceive that there is something that they can do which benefits others.”

Thirdly, humanitarian organizations are far more open to using free and open source software than they were just two years ago. So the resources required to monitor and map crowdsourced information need not break the bank. Indeed, the Syria Crisis Map uses a free and open source data-mining platform called HealthMap, which has been monitoring some 2,000 English-based sources on a daily basis for months. The technology powering the map itself, Ushahidi, is also free and open source. Moreover, the team behind the project is comprised of just a handful of volunteers doing this in their own free time (for almost an entire year now). And as a result of this initiative, I am collaborating with a colleague from UNDP to pilot HealthMap’s data mining feature for conflict monitoring and peacebuilding purposes.

Fourth, other than UN Global Pulse, humanitarian agencies are not investing time and resources to manage Big (Crisis) Data. Why? Because they have neither the time nor the know-how. To this end, they are starting to “outsource” and indeed “crowdsource” these tasks—just as private sector businesses have been doing for years in order to extend their reach. Anyone actually familiar with this space and developments since Haiti already knows this. The CrisisMappers Network, Standby Volunteer Task Force (SBTF), Humanitarian OpenStreetMap (HOT) and Crisis Commons (CC) are four volunteer/technical networks that have already collaborated actively with a number of humanitarian organizations since Haiti to provide the “surge capacity” requested by the latter; this includes UN OCHA in Libya and Colombia, UNHCR in Somalia and WHO in Libya, to name a few. In fact, these groups even have their own acronym: Volunteer & Technical Communities (V&TCs).

As the former head of OCHA’s Information Services Section (ISS) noted after the SBTF launched the Libya Crisis Map, “Your efforts at tackling a difficult problem have definitely reduced the information overload; sorting through the multitude of signals on the crisis is not easy task” (March 8, 2011). Furthermore, the crowdsourced social media information mapped on the Libya Crisis Map was integrated into official UN OCHA information products. I dare say activating the SBTF was worth OCHA’s while. And it cost the UN a grand total of $0 to benefit from this support.

Credit: Chris Bow

The rapid rise of V&TC’s has catalyzed the launch of the Digital Humanitarian Network (DHN), formerly called the Humanitarian Standby Task Force (H-SBTF). Digital Humanitarians is a network-of-network catalyzed by the UN and comprising some of the most active members of the volunteer & technical co-mmunity. The purpose of the Digital Humanitarian platform (powered by Ning) is to provide a dedicated interface for traditional humanitarian organizations to outsource and crowdsource important information management tasks during and in-between crises. OCHA has also launched the Communities of Interest (COIs) platform to further leverage volunteer engagement in other areas of humanitarian response.

These are not isolated efforts. During the massive Russian fires of 2010, volunteers launched their own citizen-based disaster response agency that was seen by many as more visible and effective than the Kremlin’s response. Back in Egypt, volunteers used IntaFeen.com to crowdsource and coordinate their own humanitarian convoys to Libya, for example. The company LinkedIn has also taken innovative steps to enable the matching of volunteers with various needs. They recently added a “Volunteer and Causes” field to its member profile page, which is now available to 150 million LinkedIn users worldwide. Sparked.com is yet another group engaged in matching volunteers with needs. The company is the world’s first micro-volunteering network, sending challenges to registered volunteers that are targeted to their skill set and the causes that they are most passionate about.

It is not farfetched to envisage how these technologies could be repurposed or simply applied to facilitate and streamline volunteer management following a disaster. Indeed, researchers at the University of Queensland in Australia have already developed a new smart phone app to help mobilize and coordinate volunteer efforts during and following major disasters. The app not only provides information on preparedness but also gives real-time updates on volunteering opportunities by local area. For example, volunteers can register for a variety of tasks including community response to extreme weather events.

Meanwhile, the American Red Cross just launched a Digital Operations Center in partnership with Dell Labs, which allows them to leverage digital volunteers and Dell’s social media monitoring platforms to reduce the noise-to-signal ratio. This is a novel “social media-based operation devoted to humanitarian relief, demonstrating the growing importance of social media in emergency situations.” As part of this center, the Red Cross also “announced a Digital Volunteer program to help respond to question from and provide information to the public during disasters.”

While important challenges do exist, there are many positive externalities to leveraging digital volunteers. As deputy high commissioner of UNHCR noted about this UNHCR-volunteer project in Somalia, these types of projects create more citizen-engagement and raises awareness of humanitarian organizations and projects. This in part explains why UNHCR wants more, not less, engage-ment with digital volunteers. Indeed, these volunteers also develop important skills that will be increasingly sought after by humanitarian organizations recruit-ing for junior full-time positions. Humanitarian organizations are likely to be come smarter and more up to speed on humanitarian technologies and digital humanitarian skills as a result. This change should be embraced.

So given the rise of “self-quantified” disaster-affected communities and digitally empowered volunteer communities, is there a future for traditional humani-tarian organizations? Of course, anyone who suggests otherwise is seriously misguided and out of touch with innovation in the humanitarian space. Twitter will not put the UN out of business. Humanitarian organizations will continue to play some very important roles, especially those relating to logistics and coor-dination. These organizations will continue outsourcing some roles but will also take on some new roles. The issue here is simply one of comparative advantage. Humanitarian organizations used to have a comparative advantage in some areas, but this has shifted for all the reasons described above. So outsourcing in some cases makes perfect sense.

Interestingly, organizations like UN OCHA are also changing some of their own internal information management processes as a result of their collaboration with volunteer networks like the SBTF, which they expect will lead to a number of efficiency gains. Furthermore, OCHA is behind the Digital Humanitarians initiative and has also been developing a check-in app for humanitarian pro-fessionals to use in disaster response—clear signs of innovation and change. Meanwhile, the UK’s Department for International Development (DfID) has just launched a $75+ million fund to leverage new technologies in support of humani-tarian response; this includes mobile phones, satellite imagery, Twitter as well as other social media technologies, digital mapping and gaming technologies. Given that crisis mapping integrates these new technologies and has been at the cutting edge of innovation in the humanitarian space, I’ve invited DfID to participate in this year’s International Conference on Crisis Mapping (ICCM 2012).

In conclusion, and as argued two years ago, the humanitarian industry is shifting towards a more multi-polar system. The rise of new actors, from digitally empowered disaster-affected communities to digital volunteer networks, has been driven by the rapid commercialization of communication technology—particularly the mobile phone and social networking platforms. These trends are unlikely to change soon and crises will continue to spur innovations in this space. This does not mean that traditional humanitarian organizations are becoming obsolete. Their roles are simply changing and this change is proof that they are not battlefield monuments. Of course, only time will tell whether they change fast enough.

Crowdsourcing Humanitarian Convoys in Libya

Many activists in Egypt donated food and medical supplies to support the Libyan revolution in early 2011. As a result, volunteers set up and coordinated humanitarian convoys from major Egyptian cities to Tripoli. But these convoys faced two major problems. First, volunteers needed to know where the convoys were in order to communicate this to Libyan revolutionists so they could wait for the fleet at the border and escort them to Tripoli. Second, because these volunteers were headed into a war zone, their friends and family wanted to keep track of them to make sure they were safe. The solution? IntaFeen.com.

Inta feen? means “where are you?” in Arabic and IntaFeen.com is a mobile check-in service like Foursquare but localized for the Arab World. Convoy drivers used IntaFeen to check-in at different stops along the way to Tripoli to provide regular updates on the situation. This is how volunteers back in Egypt who coordinated the convoy kept track of their progress and communicated updates in real-time to their Libyan counterparts. Volunteers who went along with the convoys also used IntaFeen and their check-in’s would also get posted on Twitter and Facebook, allowing families and friends in Egypt to track their whereabouts.

Al Amain Road is a highway between Alexandria and Tripoli. These tweets and check-in’s acted as a DIY fleet management system for volunteers and activists.

The use of IntaFeen combined with Facebook and Twitter also created an interesting side-effect in terms of social media marketing to promote activism. The sharing of these updates within and across various social networks galvanized more Egyptians to volunteer their time and resulted in more convoys.

I wonder whether these activists knew about another crowdsourced volunteer project taking place at exactly the same time in support of the UN’s humanitarian relief operations: Libya Crisis Map. Much of the content added to the map was sourced from social media. Could the #LibyaConvoy project have benefited from the real-time situational awareness provided by the Libya Crisis Map?

Will we see more convergence between volunteer-run crisis maps and volunteer-run humanitarian response in the near future?

Big thanks to Adel Youssef from IntaFeen.com who spoke about this fascinating project (and Ushahidi) at Where 2.0 this week. More information on #Libya Convoy is available here. See also my earlier blog posts on the use of check-in’s for activism and disaster response.

Crisis Mapping Syria: Automated Data Mining and Crowdsourced Human Intelligence

The Syria Tracker Crisis Map is without doubt one of the most impressive crisis mapping projects yet. Launched just a few weeks after the protests began one year ago, the crisis map is spearheaded by a just handful of US-based Syrian activists have meticulously and systematically documented 1,529 reports of human rights violations including a total of 11,147 killings. As recently reported in this NewScientist article, “Mapping the Human Cost of Syria’s Uprising,” the crisis map “could be the most accurate estimate yet of the death toll in Syria’s uprising […].” Their approach? “A combination of automated data mining and crowdsourced human intelligence,” which “could provide a powerful means to assess the human cost of wars and disasters.”

On the data-mining side, Syria Tracker has repurposed the HealthMap platform, which mines thousands of online sources for the purposes of disease detection and then maps the results, “giving public-health officials an easy way to monitor local disease conditions.” The customized version of this platform for Syria Tracker (ST), known as HealthMap Crisis, mines English information sources for evidence of human rights violations, such as killings, torture and detainment. As the ST Team notes, their data mining platform “draws from a broad range of sources to reduce reporting biases.” Between June 2011 and January 2012, for example, the platform collected over 43,o00 news articles and blog posts from almost 2,000 English-based sources from around the world (including some pro-regime sources).

Syria Tracker combines the results of this sophisticated data mining approach with crowdsourced human intelligence, i.e., field-based eye-witness reports shared via webform, email, Twitter, Facebook, YouTube and voicemail. This naturally presents several important security issues, which explains why the main ST website includes an instructions page detailing security precautions that need to be taken while sub-mitting reports from within Syria. They also link to this practical guide on how to protect your identity and security online and when using mobile phones. The guide is available in both English and Arabic.

Eye-witness reports are subsequently translated, geo-referenced, coded and verified by a group of volunteers who triangulate the information with other sources such as those provided by the HealthMap Crisis platform. They also filter the reports and remove dupli-cates. Reports that have a low con-fidence level vis-a-vis veracity are also removed. Volunteers use a dig-up or vote-up/vote-down feature to “score” the veracity of eye-witness reports. Using this approach, the ST Team and their volunteers have been able to verify almost 90% of the documented killings mapped on their platform thanks to video and/or photographic evidence. They have also been able to associate specific names to about 88% of those reported killed by Syrian forces since the uprising began.

Depending on the levels of violence in Syria, the turn-around time for a report to be mapped on Syria Tracker is between 1-3 days. The team also produces weekly situation reports based on the data they’ve collected along with detailed graphical analysis. KML files that can be uploaded and viewed using Google Earth are also made available on a regular basis. These provide “a more precisely geo-located tally of deaths per location.”

In sum, Syria Tracker is very much breaking new ground vis-a-vis crisis mapping. They’re combining automated data mining technology with crowdsourced eye-witness reports from Syria. In addition, they’ve been doing this for a year, which makes the project the longest running crisis maps I’ve seen in a hostile environ-ment. Moreover, they’ve been able to sustain these import efforts with just a small team of volunteers. As for the veracity of the collected information, I know of no other public effort that has taken such a meticulous and rigorous approach to documenting the killings in Syria in near real-time. On February 24th, Al-Jazeera posted the following estimates:

Syrian Revolution Coordination Union: 9,073 deaths
Local Coordination Committees: 8,551 deaths
Syrian Observatory for Human Rights: 5,581 deaths

At the time, Syria Tracker had a total of 7,901 documented killings associated with specific names, dates and locations. While some duplicate reports may remain, the team argues that “missing records are a much bigger source of error.” Indeed, They believe that “the higher estimates are more likely, even if one chooses to disregard those reports that came in on some of the most violent days where names were not always recorded.”

The Syria Crisis Map itself has been viewed by visitors from 136 countries around the world and 2,018 cities—with the top 3 cities being Damascus, Washington DC and, interestingly, Riyadh, Saudia Arabia. The witnessing has thus been truly global and collective. When the Syrian regime falls, “the data may help sub-sequent governments hold him and other senior leaders to account,” writes the New Scientist. This was one of the principle motivations behind the launch of the Ushahidi platform in Kenya over four years ago. Syria Tracker is powered by Ushahidi’s cloud-based platform, Crowdmap. Finally, we know for a fact that the International Criminal Court (ICC) and Amnesty International (AI) closely followed the Libya Crisis Map last year.

Communicating with Disaster Affected Communities (CDAC)

Communication is Aid: Curated tweets and commentary from the CDAC Network’s Media and Technology Fair, London 2012. My commentary in blue. This is the first time I’ve used Storify to curate content. (I bumped into the co-founder of the platform at SXSW which reminded me I really needed to get in on the action).

  1. Sha

    re
    “After the Japan earthquake, >20% of ALL web queries issued were on tsunamis.” @spangledrongo #commisaid

    Thu, Mar 22 2012 08:53:43
  2. Would be great to see how this type of search data compares to data from Tweets. Take this analysis of tweets following the earthquake in Chile, for example.
  3. Share
    RT @UNFPA: Professional systems are being replaced by consumer tools says @Google Crisis Response #commisaid

    Thu, Mar 22 2012 08:48:58
  4. And as a result, crisis-affected communities are increasingly becoming digital as I note in this blog post.
  5. Share
    Closed systems closed data will be left behind and unused:crisis response is social and collaboration is empowering @CDACNetwork #commisaid

    Thu, Mar 22 2012 08:49:58
  6. Share
    RT @catherinedem: Crisis response is #social – online social collaboration spikes during and after disaster @spangledrongo #commisaid

    Thu, Mar 22 2012 08:54:03
  7. Share
    “It is essential to have authoritative content” Nigel Snoad at @CDACNetwork’s Media & Tech Fair #commisaid

    Thu, Mar 22 2012 08:58:28
  8. Does this mean that all user-generated content should be ignored because said content does not necessarily come from a known and authoritative source? Who decides what is authoritative?
  9. Share
    we don’t empower communities by giving them info,they empower themselves by giving us info that we can act on-@komunikasikan #commisaid

    Thu, Mar 22 2012 11:31:02
  10. What if this information is not authoritative because it does not come from official sources?
  11. Share
    RT @ushahidi: “In a crisis, the mobile internet stays most resilient, even more than SMS.” #commisaid Nigel Snoad

    Thu, Mar 22 2012 09:02:53
  12. Share
    RT @jqg: Empower local communities to generate their own tools and figure out their own solutions #commisaid

    Thu, Mar 22 2012 09:10:43
  13. See this blog post on Democratizing ICT for Development Using DIY Innovation and Open Data.
  14. Share
    Through #Mission4636 SMS system, radio presenter @carelpedre was able to communicate directly with affected people in Haiti #commisaid

    Thu, Mar 22 2012 12:20:14
  15. This is rather interesting, I hadn’t realized that radio stations in Haiti actively used the information from the Ushahidi Haiti 4636 project.
  16. Share
    Fascinating talk with @carelpedre- many of #Haiti ‘s pre-earthquake twitter users came from @juno7’s lottery push notifications #Commisaid

    Thu, Mar 22 2012 11:11:43
  17. Share
    More about @Carelpedre using 4636 project after #Haiti earthquake bit.ly/plgzXJ #commisaid

    Thu, Mar 22 2012 12:17:52
  18. Share
    Internews: Innovation comes within, info comes from within, people will find ways to communicate no matter what #commisaid @CDACNetwork

    Thu, Mar 22 2012 09:26:25
  19. So best of luck to those who wish to regulate this space! As my colleague Tim McNamara has noted “Crisis mapping is not simply a technological shift, it is also a process of rapid decentralisation of power. With extremely low barriers to entry, many new entrants are appearing in the fields of emergency and disaster response. They are ignoring the traditional hierarchies, because the new entrants perceive that there is something that they can do which benefits others.”
  20. Share
    @souktel: “be simple, be creative and learn from the community around you” #commisaid

    Thu, Mar 22 2012 10:57:09
  21. Share
    Tools shouldn’t own data. RT @whiteafrican: “It’s not about the platform being open, it’s about the data being open”- @jcrowley #commisaid

    Thu, Mar 22 2012 11:56:08
  22. Share
    Humanitarians have to get their heads around media and tech or risk being left behind #m4d #media #tech #commisaid

    Thu, Mar 22 2012 11:56:58
  23. Share
    Cutting edge is to get the #crowd & the #algorithm to filter each other in filtering massive overload of information in a crisis #commisaid

    Thu, Mar 22 2012 12:01:17
  24. As Robert Kirkpatrick likes to say, “Use the hunch of the expert, machine algorithms and the wisdom of the crowd.”
  25. Share
    “How long until the disaster affected communities start analyzing the aid agencies?” #commisaid

    Thu, Mar 22 2012 12:14:50
  26. Yes! Sousveillance meets analysis of big data on the humanitarian sector.
  27. Share
    Why is it such a big deal, for the humanitarian industry to get feedback from a community? Companies have done it for decades. #commisaid

    Thu, Mar 22 2012 12:17:31
  28. Some of my thoughts on what the humanitarian community can learn from the private sector vis-a-vis customer support.
  29. Share
    People who are not traditional humanitarian actors are taking on humanitarian roles, driven by the democratisation of technology #commisaid

    Thu, Mar 22 2012 13:00:12
  30. Indeed, not only are disaster-affected communities increasingly digital, so are global volunteer networks like the Standby Volunteer Task Force (SBTF).
  31. Share
    Technology is shifting the power balance – it’s helping local communities to organise their own responses to disasters #commisaid

    Thu, Mar 22 2012 13:11:18
  32. Indeed, as a result of these mobile technologies, affected populations are increasingly able to source, share and generate a vast amount of information, which is completely transforming disaster response. More on this here.
  33. Share
    Like Paul Currion’s analogy – humanitarian sector risks obsolescence in the same way the record industry did. Watch out? #commisaid

    Thu, Mar 22 2012 13:17:36
  34. Share


    Thu, Mar 22 2012 14:57:13
  35. One of my favorite books, The Starfish and the Spider: The Unstoppable Power of Leaderless Organizations, has an excellent case study on the music industry. The above picture is taken from that chapter and charts the history of the industry from the perspective of hierarchies vs networks. I’ve argued a couple years ago that the same dynamic is taking place within humanitarian response. See this blog post on Disaster Relief 2.0: Toward a Multipolar System.
  36. Share
    A thousand flowers can bloom beautifully IF common data standards allow sharing. Right now no natural selection improving quality #commisaid

    Thu, Mar 22 2012 13:43:19
  37. Share
    @GSMADisasterRes :free phone numbers &short codes r not silver bullets 4 meaningful communication w/disaster affected communities #commisaid

    Thu, Mar 22 2012 13:58:44
  38. Share
    Scale horizontally, not vertically. The end of command and control?! #commisaid

    Thu, Mar 22 2012 14:07:54
  39. Share
    RT @reeniac: what now? communities have the solution, we need to listen. start by including them in the discussions – Dr Jamilah #commisaid

    Thu, Mar 22 2012 14:14:55