A research team at Yahoo recently completed an empirical study (PDF) on the behavior of Twitter users after the 8.8 magnitude earthquake in Chile. The study was based on 4,727,524 indexed tweets, about 20% of which were replies to other tweets. What is particularly interesting about this study is that the team also analyzed the spread of false rumors and confirmed news that were disseminated on Twitter.
The authors “manually selected some relevant cases of valid news items, which were confirmed at some point by reliable sources.” In addition, they “manually selected important cases of baseless rumors which emerged during the crisis (confirmed to be false at some point).” Their goal was to determine whether users interacted differently when faced with valid news vs false rumors.
The study shows that about 95% of tweets related to confirmed reports validated that information. In contrast only 0.03% of tweets denied the validity of these true cases. Interestingly, the results also show that “the number of tweets that deny information becomes much larger when the information corresponds to a false rumor.” In fact, about 50% of tweets will deny the validity of false reports. The table below lists the full results.
The authors conclude that “the propagation of tweets that correspond to rumors differs from tweets that spread news because rumors tend to be questioned more than news by the Twitter community. Notice that this fact suggests that the Twitter community works like a collaborative filter of information. This result suggests also a very promising research line: it could posible to detect rumors by using aggregate analysis on tweets.”
I think these findings are particularly important for projects like *Swift River, which try to validate crowdsourced crisis information in real-time. I would also be interested to see a similar study on tweets around the Haitian earthquake to explore whether this “collaborative filter” dynamic is an emergent phenomena in this complex systems or simply an artifact of something else.
Interested in learning more about “information forensics”? See this link and the articles below:
- Predicting the Credibility of Disaster Tweets Automatically
- Automatically Ranking Credibility of Tweets During Major Events
- Six Degrees of Separation: Implications for Verifying Social Media
- How to Verify Crowdsourced Information from Social Media
- Truth in Age of Social Media: Social Computing & Big Data Challenge
- Truthiness as Probability: Moving Beyond the True or False Dichotomy when Verifying Social Media
- How to Verify and Counter Rumors in Social Media
- Crowdsourcing for Human Rights Monitoring: Challenges and Opportunities for Information Collection & Verification
- Rapidly Verifying the Credibility of Sources on Twitter
- Accelerating the Verification of Social Media Content
iRevolutions 
This report suggests that “Truthiness” has some natural buoyancy, that the Truth “floats to the top” on Twitter — and I am really happy to hear this. It’s a fantastic thing that type of noise is dampened by the crowd. This confirms my anecdotal observations. I remember in the Mumbai attacks in 2008 as the first time that Twitter got serious about policing rumors. It would be great to see these results for that data set as well.
As we know, an organic rumor could be extremely dangerous, leading for instance to the death of first responders, the expulsion or aid workers, or the neglect of victims. I think about an instance where a rumor, “the aid worker poisoned the wells” would cause someone to get murdered, or provoke a riot.
So I am cheered to see that Truth seems to “float more” on Twitter.
But an intentional rumor, propagated by one or more bad actors, would be a much more dangerous case, and this is not covered in these data sets. The consequences of a rumor, even if it is not widely entertain as truth, could be truly catastrophic under the wrong circumstances. The rumors from Chile were criticized on Twitter (hooray!) but they are also serving as a blueprint for a first generation of network-mediated crisis exploitation (oh no!) Note that we have (only since January 2010) really begun to see the emergence of sophisticated scam artists in ad hoc digital crisis networks. So I think we have many worse situations to come than merely dealing with
I am warily watching to see what happens in the next few years. Our challenge socially, I think, is to be more engaged as critics of the news, and to invest now in infrastructure (filters, algorithms) and training (eg. via local emergency preparation groups). Regarding infrastructure, the authors of the paper conclude with a great recommendation for an infrastructural improvement in our ability to critique in this sense — a Twitter client could (and should!) implement this today:
“As an application, given that detecting when a tweet is asking for information … could for instance warn people that many other users are questioning the information item they are reading. This would provide signals for users to determine how much to trust a certain piece of information.”
Swift River is directly addressing this from an external perspective, but something native to Twitter could help prevent rumors, not just spot them.
And still we need better information literacy — more than any technological solution.
My fear is that even a single high profile failure will damage “the reputation of the crowd” in a way that will have a strong chilling effect on the incredible experimentation and energy that is happening presently. I do not want to look back on this as a lost golden age of crisis mapping — we are only barely getting started!
Toward this goal, a close examination of the Chilean dynamics is really valuable — we just need to keep it in perspective as a snapshot of the early heydays.
Hiya Chris, you make some stellar points, as always. I share your concern re:
“…a single high profile failure will damage “the reputation of the crowd” in a way that will have a strong chilling effect on the incredible experimentation and energy that is happening presently. I do not want to look back on this as a lost golden age of crisis mapping — we are only barely getting started!”
Pingback: Top 3 FAQs for Social Media and EM Beginners « idisaster 2.0
Pingback: 7 Ways to Be a Good Twitter Citizen During a Crisis | eVentures in Cyberland
Maybe one day we’ll be able to govern ourselves instead of relying on centralized institutions.
Pingback: Seeking the Trustworthy Tweet: Can “Tweetsourcing” Ever Fit the Needs of Humanitarian Organizations? | iRevolution
Pingback: How to Verifying Social Media Content: Some Tips and Tricks on Information Forensics | iRevolution
Pingback: Detecting Emerging Conflicts with Web Mining and Crisis Mapping | iRevolution
What about deliberate obfuscation, which might be salient during political violence mapping?
Pingback: Communicating with Disaster Affected Communities | iRevolution
Pingback: Communicating with Disaster Affected Communities (CDAC) | iRevolution
Reblogged this on Caledoniyya and commented:
Measuring the immeasurable?
Pingback: Truth in the Age of Social Media: A Social Computing and Big Data Challenge | iRevolution
Pingback: Analyzing the Veracity of Tweets during a Major Crisis « Transmedia Camp 101
Pingback: What Was Novel About Social Media Use During Hurricane Sandy? | iRevolution
Pingback: Predicting the Credibility of Disaster Tweets Automatically | iRevolution
Pingback: Forscher entwickeln Lügendetektor für Twitter | BASIC thinking
Pingback: Parenting: Just let the Robots handle it. « With a Wiffle Ball Bat
Pingback: Forscher entwickeln Lügendetektor für Twitter - YEPA NEWS
Pingback: Desarrollan nuevo sistema para descubrir veracidad de tuits en Twitter | La mirada de José Altimiras Lampré
Pingback: How To Tell If A Tweet Is Credible - AllTwitter
Pingback: An Algorithm to Limit Inaccurate Tweets : Stephen E. Arnold @ Beyond Search
Pingback: Building a Better Truth Machine | ForensicsPress.com
Pingback: Using Crowdsourcing to Counter the Spread of False Rumors on Social Media During Crises | iRevolution
Pingback: Twitter Under Crisis: Can we trust what we RT? | kredible.net
Pingback: Twitter no volverá a matar - Aurora Ferrer | Aurora Ferrer
Pingback: Crowd reporting puts the squeeze on traditional journalism - Social Media - English - DW.DE
Pingback: The Good, Bad, and In-Between of Social Media In Crisis Situations | iStrategyLabs – A Digital Agency That Invents Solutions Online and Off
Pingback: Russia part 3 (long) | fightingstuff