Monthly Archives: October 2013

Mining Mainstream Media for Emergency Management 2.0

Posted on October 29, 2013 | 4 comments

There is so much attention (and hype) around the use of social media for emergency management (SMEM) that we often forget about mainstream media when it comes to next generation humanitarian technologies. The news media across the globe has become increasingly digital in recent years—and thus analyzable in real-time. Twitter added little value during the recent Pakistan Earthquake, for example. Instead, it was the Pakistani mainstream media that provided the immediate situational awareness necessary for a preliminary damage and needs assessment. This means that our humanitarian technologies need to ingest both social media and mainstream media feeds.

Now, this is hardly revolutionary. I used to work for a data mining company ten years ago that focused on analyzing Reuters Newswires in real-time using natural language processing (NLP). This was for a conflict early warning system we were developing. The added value of monitoring mainstream media for crisis mapping purposes has also been demonstrated repeatedly in recent years. In this study from 2008, I showed that a crisis map of Kenya was more complete when sources included mainstream media as well as user-generated content.

So why revisit mainstream media now? Simple: GDELT. The Global Data Event, Language and Tone dataset that my colleague Kalev Leetaru launched earlier this year. GDELT is the single largest public and global event-data catalog ever developed. Digital Humanitarians need no longer monitor mainstream media manually. We can simply develop a dedicated interface on top of GDELT to automatically extract situational awareness information for disaster response purposes. We’re already doing this with Twitter, so why not extend the approach to global digital mainstream media as well?

GDELT data is drawn from a “cross-section of all major international, national, regional, local, and hyper-local news sources, both print and broadcast, from nearly every corner of the globe, in both English and vernacular.” All identified events are automatically coded using the CAMEO coding framework (although Kalev has since added several dozen additional event-types). In short, GDELT codes all events by the actors involved, the type of event, location, time and other meta-data attributes. For example, actors include “Refugees,” “United Nations,” and “NGO”. Event-types include variables such as “Affect” which captures everything from refugees to displaced persons, evacuations, etc. Humanitarian crises, aid, disasters, disaster relief, etc. are also included as an event-type. The “Provision of Humanitarian Aid” is another event-type, for example. GDELT data is currently updated every 24 hours, and Kalev has plans to provide hourly updates in the near future and ultimately 30-minute updates.

If this isn’t impressive enough, Kalev and colleagues have just launched the GDELT Global Knowledge Graph (GKG). “To sum up the GKG in a single sentence, it connects every person, organization, location, count, theme, news source, and event across the planet into a single massive network that captures what’s happening around the world, what its context is and who’s involved, and how the world is feeling about it, every single day.” The figure above (click to enlarge) is based on a subset of a single day of the GDELT Knowledge Graph, showing “how the cities of the world are connected to each other in a single day’s worth of news. A customized version of the GKG could perhaps prove useful for UN OCHA’s “Who Does What, Where” (3Ws) directory in the future.

I’ve had long conversations with Kalev this month about leveraging GDELT for disaster response and he is very supportive of the idea. My hope is that we’ll be able to add a GDELT feed to MicroMappers next year. I’m also wondering whether we could eventually create a version of the AIDR platform that ingests GDELT data instead (or in addition to) Twitter. There is real potential here, which is why I’m excited that my colleagues at OCHA are exploring GDELT for humanitarian response. I’ll be meeting with them this week and next to explore ways to collaborate on making the most of GDELT for humanitarian response.

Note: Mainstream media obviously includes television and radio as well. Some colleagues of mine in Austria are looking at triangulating television broadcasts with text-based media and social media for a European project.

4 Comments

Posted in Big Data, Humanitarian Technologies

Tagged Emergency, GDELT, Humanitarian, Mainstream, media, Response

Automatically Identifying Eyewitness Reporters on Twitter During Disasters

Posted on October 28, 2013 | 3 comments

My colleague Kate Starbird recently shared a very neat study entitled “Learning from the Crowd: Collaborative Filtering Techniques for Identifying On-the-Ground Twitterers during Mass Disruptions” (PDF). As she and her co-authors rightly argue, “most Twitter activity during mass disruption events is generated by the remote crowd.” So can we use advanced computing to rapidly identify Twitter users who are reporting from ground zero? The answer is yes.

An important indicator of whether or not a Twitter user is reporting from the scene of a crisis is the number of times they are retweeted. During the Egyptian revolution in early 2011, “nearly 30% of highly retweeted Twitter users were physically present at those protest events.” Kate et al. drew on this insight to study tweets posted during the Occupy Wall Street (OWS) protests in September 2011. The authors manually analyzed a sample of more than 2,300 Twitter users to determine which were tweeting from the protests. They found that 4.5% of Twitter users in their sample were actually onsite. Using this dataset as training data, Kate et al. were able to develop a classifier that can automatically identify Twitter users reporting from the protests with an accuracy of just shy of 70%. I expect that more training data could very well help increase this accuracy score.

In any event, “the information resulting from this or any filtering technique must be further combined with human judgment to assess its accuracy.” As the authors rightly note, “this ‘limitation’ fits well within an information space that is witnessing the rise of digital volunteer communities who monitor multiple data sources, including social media, looking to identify and amplify new information coming from the ground.” To be sure, “For volunteers like these, the use of techniques that increase the signal to noise ratio in the data has the potential to drastically reduce the amount of work they must do. The model that we have outlined does not result in perfect classification, but it does increase this signal-to-noise ratio substantially—tripling it in fact.”

I really hope that someone will leverage Kate’s important work to develop a standalone platform that automatically generates a list of Twitter users who are reporting from disaster-affected areas. This would be a very worthwhile contribution to the ecosystem of next-generation humanitarian technologies. In the meantime, perhaps QCRI’s Artificial Intelligence for Disaster Response (AIDR) platform will help digital humanitarians automatically identify tweets posted by eyewitnesses. I’m optimistic since we were able to create a machine learning classifier with an accuracy of 80%-90% for eyewitness tweets. More on this in our recent study.

One question that remains is how to automatically identify tweets like the one above? This person is not an eyewitness but was likely on the phone with her family who are closer to the action. How do we develop a classifier to catch these “second-hand” eyewitness reports?

3 Comments

Posted in Big Data, Crowdsourcing, Humanitarian Technologies, Social Computing, Social Media

Tagged Disaster, Eyewitness, Local, Response, Starbird, Twitter

Why Anonymity is Important for Truth and Trustworthiness Online

Posted on October 22, 2013 | 7 comments

Philosophy Professor, Karen Frost-Arnold, has just published a highly lucid analysis of the dangers that come with Internet accountability (PDF). While the anonymity provided by social media can facilitate the spread of lies, Karen rightly argues that preventing anonymity can undermine online communities by stifling communication and spreading ignorance, thus leading to a larger volume of untrustworthy information. Her insights are instructive for those interested in information forensics and digital humanitarian action.

To make her case, Karen distinguishes between error-avoidance and truth-attainment. The former seeks to avoid false beliefs while the latter seeks to attain true belief. Take mainstream and social media, for example. Some argue that the “value of traditional media surpasses that of the blogosphere […] because the traditional media are superior at filtering out false claims” since professional journalists “reduce the number of errors that might otherwise be reported and believed.” Others counter this assertion: “People who confine themselves to a filtered medium may well avoid believing falsehoods (if the filters are working well), but inevitably they will also miss out on valuable knowledge,” including many true beliefs.

Karen argues that Internet anonymity is at odds with both error-avoiding purists and truth-seeking purists. For example, “some experimental evidence indicates that anonymity in computer-mediated discussion increases the quantity and novelty of ideas shared.” In addition, anonymity provides a measure of safety. This is particularly important for digital activists and others who are vulnerable and oppressed. Without this anonymity, important knowledge may not be shared. To this end, “Removal of anonymity could deprive the community of true beliefs spread by reports from socially threatened groups. Without online anonymity, activists, citizen journalists, and members of many socially stigmatized groups are much less likely to take the risk of sharing what they know with others.”

This leads to decreased participation, which in turn undermines the diversity of online communities and their ability to detect errors. To be sure, “anonymity can enhance error-detection by enabling increased transformative criticism to weed out error and bias.” In fact, “anonymity enables such groups to share criticisms of false beliefs. These criticisms can lead community members to reject or suspend judgment on false claims.” In other words, “Blogging and tweeting are not simply means of disseminating knowledge claims; they are also means of challenging, criticizing & uncovering errors in others’ knowledge claims.” As Karen rightly notes, “The error-uncovering efficacy of such criticism is enhanced by the anonymity that facilitates participation by diverse groups who would otherwise, for fear of sanction, not join the discussion. Removing anonymity risks silencing their valuable criticisms.” In sum, “anonymity facilitates error detection as well as truth attainment.”

Karen thus argues for internet norms of civility instead of barring Internet anonymity. She also outlines the many costs of enforcing the use of real-world identities online. Detecting false identities is both time and resource intensive. I experienced this first-hand during the Libya Crisis Map operation. Investigating online identities diverts time and resources away obtaining other valuable truths and detecting other important errors. Moreover, this type of investigative accountability “can have a dampening effect on internet speech as those who desire anonymity avoid making surprising claims that might raise the suspicions of potential investigators.” This curtails the sharing of valuable truths.

“To prevent the problem of disproportionate investigation of marginalized and minority users,” Karen writes that online communities “need mechanisms for checking the biases of potential investigators.” To this end, “if the question of whether some internet speech merits investigation is debated within a community, then as the diversity of that community increases, the likelihood increases that biased reasons for suspicion will be challenged.”

Karen also turns to recent research in behavioral and experimental economics, sociology and psychology for potential solutions. For example, “People appear less likely to lie when the lie only gives them a small benefit but does the recipient a great harm.” Making this possible harm more visible to would-be perpetrators may dissuade dishonest actions. Research also shows that “when people are asked to reflect on their own moral values or read a code of ethics before being tempted with an opportunity for profitable deception, they are less likely to be dishonest, even when there is no risk of dishonesty being detected.” This is precisely the rational behind my piece on crowdsourcing honesty.

See also:

Crowdsourcing Critical Thinking to Verify Social Media [link]
Truth in the Age of Social Media: A Big Data Challenge [link]

7 Comments

Posted in Crowdsourcing, Social Media

Tagged Anonymity, trustworthiness, Truth

Analyzing Fake Content on Twitter During Boston Marathon Bombings

Posted on October 20, 2013 | 6 comments

As iRevolution readers already know, the application of Information Forensics to social media is one of my primary areas of interest. So I’m always on the lookout for new and related studies, such as this one (PDF), which was just published by colleagues of mine in India. The study by Aditi Gupta et al. analyzes fake content shared on Twitter during the Boston Marathon Bombings earlier this year.

Gupta et al. collected close to 8 million unique tweets posted by 3.7 million unique users between April 15-19th, 2013. The table below provides more details. The authors found that rumors and fake content comprised 29% of the content that went viral on Twitter, while 51% of the content constituted generic opinions and comments. The remaining 20% relayed true information. Interestingly, approximately 75% of fake tweets were propagated via mobile phone devices compared to true tweets which comprised 64% of tweets posted via mobiles.

The authors also found that many users with high social reputation and verified accounts were responsible for spreading the bulk of the fake content posted to Twitter. Indeed, the study shows that fake content did not travel rapidly during the first hour after the bombing. Rumors and fake information only goes viral after Twitter users with large numbers of followers start propagating the fake content. To this end, “determining whether some information is true or fake, based on only factors based on high number of followers and veriﬁed accounts is not possible in the initial hours.”

Gupta et al. also identified close to 32,000 new Twitter accounts created between April 15-19 that also posted at least one tweet about the bombings. About 20% (6,073 accounts) of these new accounts were subsequently suspended by Twitter. The authors found that 98.7% of these suspended accounts did not include the word Boston in their names and usernames. They also note that some of these deleted accounts were “quite influential” during the Boston tragedy. The figure below depicts the number of suspended Twitter accounts created in the hours and days following the blast.

The authors also carried out some basic social network analysis of the suspended Twitter accounts. First, they removed from the analysis all suspended accounts that did not interact with each other, which left just 69 accounts. Next, they analyzed the network typology of these 69 accounts, which produced four distinct graph structures: Single Link, Closed Community, Star Typology and Self-Loops. These are displayed in the figure below (click to enlarge).

The two most interesting graphs are the Closed Community and Star Typology graphs—the second and third graphs in the figure above.

Closed Community: Users that retweet and mention each other, forming a closed community as indicated by the high closeness centrality values produced by the social network analysis. “All these nodes have similar usernames too, all usernames have the same preﬁx and only numbers in the sufﬁxes are different. This indicates that either these proﬁles were created by same or similar minded people for posting common propaganda posts.” Gupta et al. analyzed the content posted by these users and found that all were “tweeting the same propaganda and hate ﬁlled tweet.”

Star Typology: Easily mistakable for the authentic “BostonMarathon” Twitter account, the fake account “BostonMarathons” created plenty of confusion. Many users propagated the fake content posted by the BostonMarathons account. As the authors note, “Impersonation or creating fake profiles is a crime that results in identity theft and is punishable by law in many countries.”

The automatic detection of these network structures on Twitter may enable us to detect and counter fake content in the future. In the meantime, my colleagues and I at QCRI are collaborating with Aditi Gupta et al. to develop a “Credibility Plugin” for Twitter based on this analysis and earlier peer-reviewed research carried out by my colleague ChaTo. Stay tuned for updates.

See also:

Boston Bombings: Analyzing First 1,000 Seconds on Twitter [link]
Taking the Pulse of the Boston Bombings on Twitter [link]
Predicting the Credibility of Disaster Tweets Automatically [link]
Auto-Ranking Credibility of Tweets During Major Events [link]
Auto-Identifying Fake Images on Twitter During Disasters [link]
How to Verify Crowdsourced Information from Social Media [link]
Crowdsourcing Critical Thinking to Verify Social Media [link]

6 Comments

Posted in Big Data, Crowdsourcing, Information Forensics, Social Computing, Social Media

Tagged #BostonMarathon, Boston, Fake, Marathon, rumors, Tweets

World Disaster Report: Next Generation Humanitarian Technology

Posted on October 17, 2013 | 3 comments

This year’s World Disaster Report was just released this morning. I had the honor of authoring Chapter 3 on “Strengthening Humanitarian Information: The Role of Technology.” The chapter focuses on the rise of “Digital Humanitarians” and explains how “Next Generation Humanitarian Technology” is used to manage Big (Crisis) Data. The chapter complements the groundbreaking report “Humanitarianism in the Network Age” published by UN OCHA earlier this year.

The key topics addressed in the chapter include:

Big (Crisis) Data
Self-Organized Disaster Response
Crowdsourcing & Bounded Crowdsourcing
Verifying Crowdsourced Information
Volunteer & Technical Communities
Digital Humanitarians
Libya Crisis Map
Typhoon Pablo Crisis Map
Syria Crisis Map
Microtasking for Disaster Response
MicroMappers
Machine Learning for Disaster Response
Artificial Intelligence for Disaster Response (AIDR)
American Red Cross Digital Operations Center
Data Protection and Security
Policymaking for Humanitarian Technology

I’m particularly interested in getting feedback on this chapter, so feel free to pose any comments or questions you may have in the comments section below.

See also:

What is Big (Crisis) Data? [link]
Humanitarianism in the Network Age [link]
Predicting Credibility of Disaster Tweets [link]
Crowdsourced Verification for Disaster Response [link]
MicroMappers: Microtasking for Disaster Response [link]
AIDR: Artificial Intelligence for Disaster Response [link]
Research Agenda for Next Generation Humanitarian Tech [link]

3 Comments

Posted in Big Data, Crisis Mapping, Crowdsourcing, Humanitarian Technologies, Information Forensics, Social Computing, Social Media

Tagged Disaster, Report, WDR2013, World

Humanitarian Crisis Computing 101

Posted on October 15, 2013 | Leave a comment

Disaster-affected communities are increasingly becoming “digital” communities. That is, they increasingly use mobile technology & social media to communicate during crises. I often refer to this user-generated content as Big (Crisis) Data. Humanitarian crisis computing seeks to rapidly identify informative, actionable and credible content in this growing stack of real-time information. The challenge is akin to finding the proverbial needle in the haystack since the vast majority of reports posted on social media is often not relevant for humanitarian response. This is largely a result of the demand versus supply problem described here.

In any event, the few “needles” of information that are relevant, can relay information that is vital and indeed-life saving for relief efforts—both traditional top-down efforts and more bottom-up grassroots efforts. When disaster strikes, we increasingly see social media traffic explode. We know there are important “pins” of relevant information hidden in this growing stack of information but how do we find them in real-time?

Humanitarian organizations are ill-equipped to managing the deluge of Big Crisis Data. They tend to sift through the stack of information manually, which means they aren’t able to process more than a small volume of information. This is represented by the dotted green line in the picture below. Big Data is often described as filter failure. Our manual filters cannot manage the large volume, velocity and variety of information posted on social media during disasters. So all the information above the dotted line, Big Data, is completely ignored.

This is where Advanced Computing comes in. Advanced Computing uses Human and Machine Computing to manage Big Data and reduce filter failure, thus allowing humanitarian organizations to process a larger volume, velocity and variety of crisis information in less time. In other words, Advanced Computing helps us push the dotted green line up the information stack.

In the early days of digital humanitarian response, we used crowdsourcing to search through the haystack of user-generated content posted during disasters. Note that said content can also include text messages (SMS), like in Haiti. Crowd-sourcing crisis information is not as much fun as the picture below would suggest, however. In fact, crowdsourcing crisis information was (and can still be) quite a mess and a big pain in the haystack. Needless to say, crowdsourcing is not the best filter to make sense of Big Crisis Data.

Recently, digital humanitarians have turned to microtasking crisis information as described here and here. The UK Guardian and Wired have also written about this novel shift from crowdsourcing to microtasking.

Microtasking basically turns a haystack into little blocks of stacks. Each micro-stack is then processed by one ore more digital humanitarian volunteers. Unlike crowdsourcing, a microtasking approach to filtering crisis information is highly scalable, which is why we recently launched MicroMappers.

The smaller the micro-stack, the easier the tasks and the faster that they can be carried out by a greater number of volunteers. For example, instead of having 10 people classify 10,000 tweets based on the Cluster System, microtasking makes it very easy for 1,000 people to classify 10 tweets each. The former would take hours while the latter mere minutes. In response to the recent earthquake in Pakistan, some 100 volunteers used MicroMappers to classify 30,000+ tweets in about 30 hours, for example.

Machine Computing, in contrast, uses natural language processing (NLP) and machine learning (ML) to “quantify” the haystack of user-generated content posted on social media during disasters. This enable us to automatically identify relevant “needles” of information.

An example of a Machine Learning approach to crisis computing is the Artificial Intelligence for Disaster Response (AIDR) platform. Using AIDR, users can teach the platform to automatically identify relevant information from Twitter during disasters. For example, AIDR can be used to automatically identify individual tweets that relay urgent needs from a haystack of millions of tweets.

The pictures above are taken from the slide deck I put together for a keynote address I recently gave at the Canadian Ministry of Foreign Affairs.

Hashtag Analysis of #Westgate Crisis Tweets

Posted on October 9, 2013 | Leave a comment

In July 2013, my team and I at QCRI launched this dashboard to analyze hashtags used by Twitter users during crises. Our first case study, which is available here, focused on Hurricane Sandy. Since then, both the UN and Greenpeace have also made use of the dashboard to analyze crisis tweets.

We just uploaded 700,000+ Westgate related tweets to the dashboard. The results are available here and also displayed above. The dashboard is still under development, so we very much welcome feedback on how to improve it for future analysis. You can upload your own tweets to the dashboard if you’d like to test drive the platform.

See also: Forensics Analysis of #Westgate Tweets (Link)

Forensics Analysis of #Westgate Tweets (Updated)

Posted on October 3, 2013 | 16 comments

Update 1: Our original Twitter collection of Westgate-related tweets included the following hashtags: #Kenya, #Nairobi #WestgateAttack, #WestagateMall, #WestgatemallAttack, #Westgateshootout & #Westgate. While we overlooked #Westlands and Westlands, we have just fixed the oversight. This explains why the original results below differed from the iHub’s analysis which was based on tweets with the keywords Westgate and Westlands.

Update 2: The list below of first tweets to report the attack has been updated to include tweets referring to Westlands. These are denoted by an asterisk (*).

I’m carrying out some preliminary “information forensics” research on the 740,000+ tweets posted during the Westgate attack. More specifically, I’m looking for any clues in the hours leading up to the attack that may reveal something out of the ordinary prior to the siege. Other questions I’m hoping to answer: Were any tweets posted during the crisis actionable? Did they add situational awareness? What kind of multimedia content was shared? Which tweets were posted by eyewitnesses? Were any tweets posted by the attackers or their supporters? If so, did these carry tactical information?

If you have additional suggestions on what else to search for, please feel free to post them in the comments section below, thank you very much. I’ll be working with QCRI research assistants over the next few weeks to dive deeper into the first 24 hours of the attack as reported on Twitter. This research would not be possible where it not for my colleagues at GNIP who very kindly granted me access their platform to download all the tweets. I’ve just reviewed the first hour of tweets (which proved to be highly emotional, as expected). Below are the very first tweets posted about the attack.

[12:38:20 local time]*
gun shots in westlands? wtf??

[12:41:49]*
Weird gunshot like sounds in westlands : (

[12:42:35]
Explosions and gunfight ongoing in #nairobi

[12:42:38]
Something really bad goin on at #Westgate. Gunshots!!!! Everyone’s fled.

[12:43:17] *
Somewhere behind Westlands? What’s up RT @[username]: Explosions and gunfight ongoing in #nairobi

[12:44:03]
Are these gunshots at #Westgate? Just heard shooting from the road behind sarit, sounded like it was coming from westgate

[12:44:37]*
@[username] shoot out at westgate westlands mall. going on for the last 10 min

[12:44:38]
Heavily armed thugs have taken over #WestGate shopping mall. Al occupants and shoppers are on the floor. Few gunshots heard…more to follow

[12:44:51]*
did anyone else in westlands hear that? #KOT #Nairobi

[12:45:04]
Seems like explosions and small arms fire are coming from Westlands or Gigiri #nairobi

[12:46:12]
Gun fight #westgate… @ntvkenya @KTNKenya @citizentvkenya any news…

[12:46:44]*
Several explosions followed by 10 minutes of running gunfight in Nairobi westlands

[12:46:59]
Small arms fire is continuing to be exchanged intermittently. #nairobi

[12:46:59]
Something’s going on around #Westgate #UkayCentre area. Keep away if you can

[12:47:54]
Gunshots and explosions heard around #Westgate anybody nearby? #Westlands

[12:48:33]*
@KenyaRedCross explosions and gunshots heard near Westgate Mall in Westlands. Fierce shoot out..casualties probable

[12:48:36]
Shoot to kill order #westgate

—

See also:

We Are Kenya: Global Map of #Westgate Tweets [Link]
Did Terrorists Use Twitter to Increase Situational Awareness? [Link]
Analyzing Tweets Posted During Mumbai Terrorist Attacks [Link]
Web 2.0 Tracks Attacks on Mumbai [Link]

16 Comments

Posted in Crowdsourcing, Information Forensics, Social Media

Tagged Attack, kenya, Nairobi, Twitter, Westgate

We Are Kenya: Global Map of #Westgate Tweets

Posted on October 3, 2013 | 5 comments

I spent over an hour trying to write this first paragraph last week and still don’t know where to start. I grew up in Nairobi, my parents lived in Kenya for more than 15 years, their house was 5 minutes from Westgate, my brother’s partner is Kenyan and I previously worked for Ushahidi, a Kenyan not-for-profit group. Witnessing the tragedy online as it unfolded in real-time, graphic pictures and all, was traumatic; I did not know the fate of several friends right away. This raw anxiety brought back memories from the devastating Haiti Earthquake of 2010; it took 12 long hours until I got word that my wife and friends had just made it out of a crumbling building.

What to do with this most recent experience and the pain that lingers? Amongst the graphic Westgate horror unfolding via Twitter, I also witnessed the outpouring of love, support and care; the offers of help from Kenyans and Somalis alike; collective grieving, disbelief and deep sadness; the will to remain strong, to overcome, to be united in support of the victims, their families and friends. So I reached out to several friends in Nairobi to ask them if aggregating and surfacing these tweets publicly could serve as a positive testament. They all said yes.

I therefore contacted colleagues at GNIP who kindly let me use their platform to collect more than 740,000 tweets related to the tragedy, starting from several hours before the horror began until the end of the siege. I then reached out to friends Claudia Perlich (data scientist) and Jer Throp (data artist) for their help on this personal project. They both kindly agreed to lend their expertise. Claudia quickly put together the map above based on the location of Twitter users responding to the events in Nairobi (click map to enlarge). The graph below depicts where Twitter users covering the Westgate tragedy were tweeting from during the first 35 hours or so.

We also did some preliminary content analysis of some keywords. The graph below displays the frequency of the terms “We Are One,” “Blood Appeal / Blood Donations,” and “Pray / Prayers” during the four day siege (click to enlarge).

Jer suggested (thankfully) a more compelling and elegant data visualization approach, which we are exploring this week. So we hope to share some initial visuals in the coming days. If you have any specific suggestions on other ways to analyze and visualize the data, please do share them in the comments section below, thank you.

See also: Forensics Analysis of #Westgate Tweets [Link]

5 Comments

Posted in Crowdsourcing, Social Media

Tagged Attack, Nairobi, Twitter, Westgate

AIDR: Artificial Intelligence for Disaster Response

Posted on October 1, 2013 | 22 comments

Social media platforms are increasingly used to communicate crisis information when major disasters strike. Hence the rise of Big (Crisis) Data. Humanitarian organizations, digital humanitarians and disaster-affected communities know that some of this user-generated content can increase situational awareness. The challenge is to identify relevant and actionable content in near real-time to triangulate with other sources and make more informed decisions on the spot. Finding potentially life-saving information in this growing stack of Big Crisis Data, however, is like looking for the proverbial needle in a giant haystack. This is why my team and I at QCRI are developing AIDR.

The free and open source Artificial Intelligence for Disaster Response platform leverages machine learning to automatically identify informative content on Twitter during disasters. Unlike the vast majority of related platforms out there, we go beyond simple keyword search to filter for informative content. Why? Because recent research shows that keyword searches can miss over 50% of relevant content posted on Twitter. This is very far from optimal for emergency response. Furthermore, tweets captured via keyword search may not be relevant since words can have multiple meanings depending on context. Finally, keywords are restricted to one language only. Machine learning overcomes all these limitations, which is why we’re developing AIDR.

So how does AIDR work? There are three components of AIDR: the Collector, Trainer and Tagger. The Collector simply allows you to collect and save a collection of tweets posted during a disaster. You can download these tweets for analysis at any time and also use them to create an automated filter using machine learning, which is where the Trainer and Tagger come in. The Trainer allows one or more users to train the AIDR platform to automatically tag tweets of interest in a given collection of tweets. Tweets of interest could include those that refer to “Needs”, “Infrastructure Damage” or “Rumors” for example.

A user creates a Trainer for tweets-of-interest by: 1) Creating a name for their Trainer, e.g., “My Trainer”; 2) Identifying topics of interest such as “Needs”, “Infrastructure Damage”, “Rumors” etc. (as many topics as the user wants); and 3) Classifying tweets by topic of interest. This last step simply involves reading collected tweets and classifying them as “Needs”, “Infrastructure Damage”, “Rumor” or “Other,” for example. Any number of users can participate in classifying these tweets. That is, once a user creates a Trainer, she can classify the tweets herself, or invite her organization to help her classify, or ask the crowd to help classify the tweets, or all of the above. She simply shares a link to her training page with whoever she likes. If she choses to crowdsource the classification of tweets, AIDR includes a built-in quality control mechanism to ensure that the crowdsourced classification is accurate.

As noted here, we tested AIDR in response to the Pakistan Earthquake last week. We quickly hacked together the user interface displayed below, so functionality rather than design was our immediate priority. In any event, digital humanitarian volunteers from the Standby Volunteer Task Force (SBTF) tagged over 1,000 tweets based on the different topics (labels) listed below. As far as we know, this was the first time that a machine learning classifier was crowdsourced in the context of a humanitarian disaster. Click here for more on this early test.

The Tagger component of AIDR analyzes the human-classified tweets from the Trainer to automatically tag new tweets coming in from the Collector. This is where the machine learning kicks in. The Tagger uses the classified tweets to learn what kinds of tweets the user is interested in. When enough tweets have been classified (20 minimum), the Tagger automatically begins to tag new tweets by topic of interest. How many classified tweets is “enough”? This will vary but the more tweets a user classifies, the more accurate the Tagger will be. Note that each automatically tagged tweet includes an accuracy score—i.e., the probability that the tweet was correctly tagged by the automatic Tagger.

The Tagger thus displays a list of automatically tagged tweets updated in real-time. The user can filter this list by topic and/or accuracy score—display all tweets tagged as “Needs” with an accuracy of 90% or more, for example. She can also download the tagged tweets for further analysis. In addition, she can share the data link of her Tagger with developers so the latter can import the tagged tweets directly into to their own platforms, e.g., MicroMappers, Ushahidi, CrisisTracker, etc. (Note that AIDR already powers CrisisTracker by automating the classification of tweets). In addition, the user can share a display link with individuals who wish to embed the live feed into their websites, blogs, etc.

In sum, AIDR is an artificial intelligence engine developed to power consumer applications like MicroMappers. Any number of other tools can also be added to the AIDR platform, like the Credibility Plugin for Twitter that we’re collaborating on with partners in India. Added to AIDR, this plugin will score individual tweets based on the probability that they convey credible information. To this end, we hope AIDR will become a key node in the nascent ecosystem of next-generation humanitarian technologies. We plan to launch a beta version of AIDR at the 2013 CrisisMappers Conference (ICCM 2013) in Nairobi, Kenya this November.

In the meantime, we welcome any feedback you may have on the above. And if you want to help as an alpha tester, please get in touch so I can point you to the Collector tool, which you can start using right away. The other AIDR tools will be open to the same group of alpha tester in the coming weeks. For more on AIDR, see also this article in Wired.

The AIDR project is a joint collaboration with the United Nations Office for the Coordination of Humanitarian Affairs (OCHA). Other organizations that have expressed an interest in AIDR include the International Committee of the Red Cross (ICRC), American Red Cross (ARC), Federal Emergency Management Agency (FEMA), New York City’s Office for Emergency Management and their counterpart in the City of San Francisco.

Note: In the future, AIDR could also be adapted to take in Facebook status updates and text messages (SMS).

22 Comments

Posted in Big Data, Crowdsourcing, Humanitarian Technologies, Social Computing, Social Media

Tagged AIDR, Disaster, Response, Twitter

Monthly Archives: October 2013

Mining Mainstream Media for Emergency Management 2.0

Patrick Meier, PhD

Table of Contents

Monthly Archives: October 2013

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Patrick Meier, PhD

Table of Contents