While spam filters provide additional layers of security to websites, they can also be used to process all kinds of information. Perhaps most famously, for example, the reCAPTCHA spam filter was used to transcribe the New York Times’ entire paper-based archives. See my previous blog post to learn how this was done and how spam filters can also be used to process information for disaster response. Given the positive response I received from humanitarian colleagues who read the blog post, I teamed up with my colleagues at QCRI to create the first ever spam filter for disaster response.
During international disasters, the humanitarian community (often lead by the UN’s Office for the Coordination of Humanitarian Affairs, OCHA) needs to carry out rapid damage assessments. Recently, these assessments have included the analysis of pictures shared on social media following a disaster. For example, OCHA activated the Digital Humanitarian Network (DHN) to collect and quickly tag pictures that capture evidence of damage in response to Typhoon Pablo in the Philippines (as described here and TEDx talk above). Some of these pictures, which were found on Twitter, were also geo-referenced by DHN volunteers. This enabled OCHA to create (over night) the unique damage assessment map below.
OCHA intends to activate the DHN again in future disasters to replicate this type of rapid damage assessment operation. This is where spam filters come in. The DHN often needs support to quickly tag these pictures (which may number in the tens of thousands). Adding a spam filter that requires email users to tag which image captures disaster damage not only helps OCHA and other organizations carry out a rapid damage assessment, but also increases the security of email systems at the same time. And it only takes 3 seconds to use the spam filter.
My team and I at QCRI have thus developed a spam filter plugin that can be easily added to email login pages like OCHA’s as shown above. When the Digital Humanitarian Network requires additional hands on deck to tag pictures during disasters, this plugin can simply be switched on. My team at QCRI can easily push the images to the plugin and pull data on which images have been tagged as showing disaster damage. The process for the end user couldn’t be simpler. Enter your username and password as normal and then simply select the picture below that shows disaster damage. If there are none, then simply click on “None” and then “Login”. The spam filter uses a predictive algorithm and an existing data-base of pictures as a control mechanism to ensure that the filter cannot be gamed. On that note, feel free to test the plugin here. We’d love your feedback as we continue testing.
The desired outcome? Each potential disaster picture is displayed to 3 different email account users. Only if each of the 3 users tag the same picture as capturing disaster damage does that picture get automatically forwarded to members of the Digital Humanitarian Network. To tag more pictures after logging in, users are invited to do so via MicroMappers, which launches this September in partnership with OCHA. MicroMappers enables members of the public to participate in digital disaster response efforts with a simple click of the mouse.
I would ideally like to see an innovative and forward-thinking organization like OCHA pilot the plugin for a two week feasibility test. If the results are positive and promising, then I hope OCHA and other UN agencies engaged in disaster response adopt the plugin more broadly. As mentioned in my previous blog post, the UN employs well over 40,000 people around the world. Even if “only” 10% login in one day, that’s still 4,000 images effortlessly tagged for use by OCHA and others during their disaster relief operations. Again, this plugin would only be used in response to major disasters when the most help is needed. We’ll be making the code for this plugin freely available and open source.
Please do get in touch if you’d like to invite your organization to participate in this innovative humanitarian technology project. You can support disaster response efforts around the world by simply logging into your email account, web portal, or Intranet!
iRevolutions 
As admirable as this effort is, it currently seems that a spambot has a 20% chance of passing the test. This means that web developers cannot yet use the plugin as an effective spam protection mechanism, as it can easily be brute-forced.
Have you considered using tasks that are not of multiple choice type? Perhaps it would work better to ask the human to mark the position of an object in an image, or to type in a few words that are descriptive? Even showing 10 images where each image has a 50% chance of being disaster related gives 1024 possible answers, instead of only 5.
I am not assuming you are stupid, I am just basing my comment on the screenshots included in the post where it says the user should select one image out of five. Perhaps I misunderstood.
Ah sorry, perhaps I misunderstood your initial observation. Note that the user is asked to select one of 6 options (5 pictures or none) and indeed we could increase the number of pictures to 10 with a 50% chance for each. Thanks for reading, and thanks for your constructive suggestions.
You’re right, the plugin is more focused on microtasking triangulation than sophisticated spam filtering. I can explain (offline) why this was prioritized as such. But since we are publicly advertising this as a spam filter, we’ll make a few changes to make the filter less easy to brute-force. We’ll stay truer to the reCAPTCHA approach. When the new settings are in place, the chances of a spambot passing the test will be between 0.8% to 5% depending on the UN’s preference. We could even go down to 0.003% chance, but I think that would be overkill. Thanks again, Jakob.
Reblogged this on Dead Wife.
Pingback: Weekly Roundup of Awesome Links: Week of July 29th 2013 - Melissa Agnes Crisis Management - Online Crisis Management Firm
Very interesting approach, since we haven’t reviewed or tested this with the DHN yet, I’m interested in learning more about how that will work. Thanks for sharing it Patrick!
Thanks for reading Cat! We’ve just launched a revised version that truly acts as a spam filter:
http://recaptcha.qcri.org
MicroMappers is a better fit for the DHN’s work and indeed is designed for the DHN whereas the spam filter is designed for email login pages like UN OCHA’s. But both do use microtasking.
We’re currently re-writing Crisis Cleanup (crisiscleanup.org, an open source disaster relief coordination platform based on bounded crowdsourcing principles, now on our 29th disaster in 4 countries). I’d really like to implement Disaster ReCaptcha as a way to support the community. Is this still a live project?
Thanks for your note and interest, Aaron. Alas, the project is no longer live. But you may want to look into MicroMappers.org. Also, is Crisis Cleanup connected with the Digital Humanitarian Network already?
Just getting into DHN. I never really meant for Crisis Cleanup to be a “thing,” until, three years after Sandy, I get a call every time there is a major disaster. Last I calculated, the improved volunteer efficiencies of using the site have resulted in ~$50M in value for survivors, but I only advertise $26M. It will always be free to relief agencies.
Crisis Cleanup is a labor of love. We support primarily the VOAD movement in the US– Voluntary Organizations Active in Disaster. Crisis Cleanup does not interact directly with survivors or the public. If you’ve got a second, here is an introductory video: https://www.youtube.com/watch?v=yCxI5YHyX5k
And a walk-through: https://www.youtube.com/watch?v=tpMOgDr_KGI
If you have an opportunity, I’d love to share some of the lessons we’ve learned, and pick your brain.
As an aside, I read your book as I was researching one I’m writing. I can relate to your narrative in Digital Humanitarians on a visceral level. Thank you for your leadership.