Using CrowdFlower to Microtask Disaster Response

Cross-posted from CrowdFlower blog

A devastating earthquake struck Port-au-Prince on January 12, 2010. Two weeks later, on January 27th, a CrowdFlower was used to translate text messages from Haitian Creole to English. Tens of thousands of messages were sent by affected Haitians over the course of several months. All of these were heroically translated by hundreds of dedicated Creole-speaking volunteers based in dozens of countries across the globe. While Ushahidi took the lead by developing the initial translation platform used just days after the earthquake, the translation efforts were eventually rerouted to CrowdFlower. Why? Three simple reasons:

  1. CrowdFlower is one of the leading and most highly robust micro-tasking platforms there is;
  2. CrowdFlower’s leadership is highly committed to supporting digital humanitarian response efforts;
  3. Haitians in Haiti could now be paid for their translation work.

While the CrowdFlower project was launched 15 days after the earthquake, i.e., following the completion of search and rescue operations, every single digital humanitarian effort in Haiti was reactive. The key takeaway here was the proof of concept–namely that large-scale micro-tasking could play an important role in humanitarian information management. This was confirmed months later when devastating floods inundated much of Pakistan. CrowdFlower was once again used to translate incoming messages from the disaster affected population. While still reactive, this second use of CrowdFlower demonstrated replicability.

The most recent and perhaps most powerful use of CrowdFlower for disaster response occurred right after Typhoon Pablo devastated the Philippines in early December 2012. The UN Office for the Coordination of Humanitarian Affairs (OCHA) activated the Digital Humanitarian Network (DHN) to rapidly deliver a detailed dataset of geo-tagged pictures and video footage (posted on Twitter) depicting the damage caused by the Typhoon. The UN needed this dataset within 12 hours, which required that 20,000 tweets to be analyzed as quickly as possible. The Standby Volunteer Task Force (SBTF), a member of Digital Huma-nitarians, immediately used CrowdFlower to identify all tweets with links to pictures & video footage. SBTF volunteers subsequently analyzed those pictures and videos for damage and geographic information using other means.

This was the most rapid use of CrowdFlower following a disaster. In fact, this use of CrowdFlower was pioneering in many respects. This was the first time that a member of the Digital Humanitarian Network made use of CrowdFlower (and thus micro-tasking) for disaster response. It was also the first time that Crowd-Flower’s existing workforce was used for disaster response. In addition, this was the first time that data processed by CrowdFlower contributed to an official crisis map produced by the UN for disaster response (see above).

These three use-cases, Haiti, Pakistan and the Philippines, clearly demonstrate the added value of micro-tasking (and hence CrowdFlower) for disaster response. If CrowdFlower had not been available in Haiti, the alternative would have been to pay a handful of professional translators. The total price could have come to some $10,000 for 50,000 text messages (at 0.20 cents per word). Thanks to CrowdFlower, Haitians in Haiti were given the chance to make some of that money by translating the text messages themselves. Income generation programs are absolutely critical to rapid recovery following major disasters. In Pakistan, the use of CrowdFlower enabled Pakistani students and the Diaspora to volunteer their time and thus accelerate the translation work for free. Following Typhoon Pablo, paid CrowdFlower workers from the Philippines, India and Australia categorized several thousand tweets in just a couple hours while the volunteers from the Standby Volunteer Task Force geo-tagged the results. Had CrowdFlower not been available then, it is highly, highly unlikely that the mission would have succeeded given the very short turn-around required by the UN.

While impressive, the above use-cases were also reactive. We need to be a lot more pro-active, which is why I’m excited to be collaborating with CrowdFlower colleagues to customize a standby platform for use by the Digital Humanitarian Network. Having a platform ready-to-go within minutes is key. And while digital volunteers will be able to use this standby platform, I strongly believe that paid CrowdFlower workers also have a key role to play in the digital huma-nitarian ecosystem. Indeed, CrowdFlower’s large, multinational and multi-lingual global workforce is simply unparalleled and has the distinct advantage of being very well versed in the CrowdFlower platform.

In sum, it is high time that the digital humanitarian space move from crowd-sourcing to micro-tasking. It has been three years since the tragic earthquake in Haiti but we have yet to adopt micro-tasking more widely. CrowdFlower should thus play a key role in promoting and enabling this important shift. Their con-tinued important leadership in digital humanitarian response should also serve as a model for other private sector companies in the US and across the globe.

bio

5 responses to “Using CrowdFlower to Microtask Disaster Response

  1. Refugee Archives at UEL

    Reblogged this on Refugee Archives Blog.

  2. Patrick, as someone who admires, supports, and respects the great majority of your thoughts, I hope you will understand my own feelings about this particular piece. This is one of the few times I have read something of yours and felt a very disconcerting unease. For myself, this piece wanders into that foggy realm where research, opinion, objectivity, vested interests, and marketing all become difficult to distinguish. Perhaps I am old-fashioned, though not so naive to know that there is nothing new in the symbiosis of government interests, business interests, and scientific interests.

    I think scientists have long recognized that these relationships are not without significant peril, particularly as they are often subtle and come in many forms. In the digital age, however, I fear we may have lowered our defenses, or maybe it is that the defenses are harder to maintain.

    It seems we are in a world today where companies write their own Wikipedia entries without acknowledging it; and where blogs become re-blogged and accepted too easily as fact, sometimes then becoming PR materials. I have even seen blogs sometimes cited in the literature like the contents are the equal of research findings….or in some cases, the material contained in a company blog being cited in published papers and used to support a point, when the writer of the company blog and writer of the paper are the same…which to me seems to be a sort of academic/intellectual version of money laundering. Perhaps worst of all, not long ago I came across an article published in a journal where a very long entry in the footnotes was pulled verbatim from a company website without citation (maybe some do not know that a few of us were taught long ago to search out and read authors’ cited references, even when they neglect to cite them). .

    In other words, it seems we are in a world today where some damn shoddy scholarship goes on. But damn shoddy scholarship is nothing new. The ability of shoddy scholarship to gain wide audience is new, and not many people seem to stop and say anymore “Whoa, wait a minute…” (unless of course they don’t like the researchers or their conclusions, regardless of whether their methods are sound). It does not help that we are in a rather small field, where research of any kind can be hard to find, good, bad, or mediocre.

    You are an admirable scholar Patrick. Please do not take my remarks to mean anything else. I believe we need more fine scholars working in the field. I hope in your boundless passion, exuberance, collaboration with CrowdFlower, and…yes I will say it…your ego (we all have one, some just smaller or larger than others) you do not, however, allow your own skeptical eye to become distracted from deeper questioning of what is fact and what is assertion in need of further exploration and validation.

    • Thanks Joseph, I really appreciate your taking the time to share your unease with me. There’s more to this post that may meet the eye. It was a calculated strategy, a means to an end rather than an end in itself. Happy to explain further offline as there are some delicate issues at play. Thanks again for writing in and for reading my blog. I hope you’ll find future posts less disconcerting.

      My very best,
      Patrick

  3. Whoa, wait a minute… CrowdFlower.com itself doesn’t seem to focus on humanitarian uses at all. In fact a quick click around takes me to this page, suggesting that contributing to CrowdFlower micro tasks is all about stupid get rich quick schemes, earning money by filling in surveys and viewing ads. Frankly I’m annoyed that I’m having my attention drawn to this via a crisismapping mailing list post.

    • As per the post (assuming you actually read it), CrowdFlower has been used on several occasions to support humanitarian response by processing information used to populate live crisis maps. But not to worry, I for one will no longer be making use of CrowdFlower due to some seriously unprofessional behavior on their part.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s