My colleague Kalev Leetaru has just launched The Global Twitter Heartbeat Project in partnership with the Cyber Infrastructure and Geospatial Information Laboratory (CIGI) and GNIP. He shared more information on this impressive initiative with the CrisisMappers Network this morning.
According to Kalev, the project “uses an SGI super-computer to visualize the Twitter Decahose live, applying fulltext geocoding to bring the number of geo-located tweets from 1% to 25% (using a full disambiguating geocoder that uses all of the user’s available information in the Twitter stream, not just looking for mentions of major cities), tone-coding each tweet using a twitter-customized dictionary of 30,000 terms, and applying a brand-new four-stage heatmap engine (this is where the supercomputer comes in) that makes a map of the number of tweets from or about each location on earth, a second map of the average tone of all tweets for each location, a third analysis of spatial proximity (how close tweets are in an area), and a fourth map as needed for the percent of all of those tweets about a particular topic, which are then all brought together into a single heatmap that takes all of these factors into account, rather than a sequence of multiple maps.”
Kalev added that, “For the purposes of this demonstration we are processing English only, but are seeing a nearly identical spatial profile to geotagged all-languages tweets (though this will affect the tonal results).” The Twitterbeat team is running a live demo showing both a US and world map updated in realtime at Supercomputing on a PufferSphere and every few seconds on the SGI website here.”
So why did Kalev share all this with the CrisisMappers Network? Because he and his team created a rather unique crisis map composed of all tweets about Hurricane Sandy, see the YouTube video above. “[Y]ou can see how the whole country lights up and how tweets don’t just move linearly up the coast as the storm progresses, capturing the advance impact of such a large storm and its peripheral effects across the country.” The team also did a “similar visualization of the recent US Presidential election showing the chaotic nature of political communication in the Twittersphere.”
To learn more about the project, I recommend watching Kalev’s 2-minute introductory video above.
That is amazing!
Pingback: Cyberculture roundup: Strong Passwords, Google’s Transparency Report, Samsung vs. Apple, Cybersecurity Act… « Erkan's Field Diary
Beautiful! What might be the implications and the usefulness for the CrisisMapping community and SBTF? : )
This part made me drool on my keyboard: “applying fulltext geocoding to bring the number of geo-located tweets from 1% to 25% (using a full disambiguating geocoder that uses all of the user’s available information in the Twitter stream, not just looking for mentions of major cities), […] that makes a map of the number of tweets from or about [!!!] each location on earth, (…) which are then all brought together into a single heatmap that takes all of these factors into account, rather than a sequence of multiple maps.”. Forget the combined heatmap (why would you do that?), that’s IMHO eye-candy and I find it the least impressive part. But a real-time sophisticated geo-coding and spatial analysis of all Tweets? Now that’s something!
Pingback: Social Media = Social Capital = Disaster Resilience? | iRevolution
Pingback: Social Media: Pulse of the Planet? | iRevolution
Pingback: Social Media: Pulse of the Planet? – News Watch
Pingback: Update: Twitter Dashboard for Disaster Response | iRevolution