Tag Archives: MIT

MAQSA: Social Analytics of User Responses to News

Designed by QCRI in partnership with MIT and Al-Jazeera, MAQSA provides an interactive topic-centric dashboard that summarizes news articles and user responses (comments, tweets, etc.) to these news items. The platform thus helps editors and publishers in newsrooms like Al-Jazeera’s better “understand user engagement and audience sentiment evolution on various topics of interest.” In addition, MAQSA “helps news consumers explore public reaction on articles relevant to a topic and refine their exploration via related entities, topics, articles and tweets.” The pilot platform currently uses Al-Jazeera data such as Op-Eds from Al-Jazeera English.

Given a topic such as “The Arab Spring,” or “Oil Spill”, the platform combines time, geography and topic to “generate a detailed activity dashboard around relevant articles. The dashboard contains an annotated comment timeline and a social graph of comments. It utilizes commenters’ locations to build maps of comment sentiment and topics by region of the world. Finally, to facilitate exploration, MAQSA provides listings of related entities, articles, and tweets. It algorithmically processes large collections of articles and tweets, and enables the dynamic specification of topics and dates for exploration.”

While others have tried to develop similar dashboards in the past, these have “not taken a topic-centric approach to viewing a collection of news articles with a focus on their user comments in the way we propose.” The team at QCRI has since added a number of exciting new features for Al-Jazeera to try out as widgets on their site. I’ll be sure to blog about these and other updates when they are officially launched. Note that other media companies (e.g., UK Guardian) will also be able to use this platform and widgets once they become public.

As always with such new initiatives, my very first thought and question is: how might we apply them in a humanitarian context? For example, perhaps MAQSA could be repurposed to do social analytics of responses from local stakeholders with respect to humanitarian news articles produced by IRIN, an award-winning humanitarian news and analysis service covering the parts of the world often under-reported, misunderstood or ignored. Perhaps an SMS component could also be added to a MAQSA-IRIN platform to facilitate this. Or perhaps there’s an application for the work that Internews carries out with local journalists and consumers of information around the world. What do you think?

From Gunfire at Sea to Maps of War: Implications for Humanitarian Innovation

MIT Professor Eric von Hippel is the author of Democratizing Innovation, a book I should have read when it was first published seven years ago. The purpose of this blog post, however, is to share some thoughts on “Gunfire at Sea: A Case Study in Innovation” (PDF), which Eric recently instructed me to read. Authored by Elting Morison in 1968, this piece is definitely required reading for anyone engaged in disruptive innovation, particularly in the humanitarian space. Morison was one of the most distinguished historians of the last century and the founder of MIT’s Program in Science, Technology and Society (STS). The Boston Globe called him “an educator and industrial historian who believed that technology could only be harnessed to serve human beings when scientists and poets could meet with mutual understanding.”

Morison details in intriguing fashion the challenges of using light artillery at sea in the late 1,800’s to illustrate how new technologies and new forms of power collide and indeed, “bombard the fixed structure of our habits of mind and behavior.” The first major innovative disruption in naval gunfire technology is the result of one person’s acute observation. Admiral Sir Percy Scott happened to watched his men during target practice one day while the ship they were on was pitching and rolling acutely due to heavy weather. The resulting accuracy of the shots was dismal save for one man who was doing something slightly different to account for the swaying. Scott observed this positive deviance carefully and cobbled existing to technology to render the strategy easier to repeat and replicate. Within a year, his gun crews were remarkable accurate.

Note that Scott was not responsible for the invention of the basic instruments he cobbled together to scale the positive deviance he observed. Scott’s contribution, rather, was  a mashup of existing technology made possible thanks to mechanical ingenuity and a keen eye for behavioral processes. As for the personality of the innovator, Scott possessed “a savage indignation directed ordinarily at the inelastic intelligence of all constituted authority, especially the British Admiralty.” Chance also plays a role in this story. “Fortune (in this case, the unaware gun pointer) indeed favors the prepared mind, but even fortune and the prepared mind need a favorable environment before they can conspire to produce sudden change. No intelligence can proceed very far above the threshold of existing data or the binding combinations of existing data.”

Whilst stationed in China several years later, Admiral Scott crosses paths with William Sims, an American Junior Officer of similar temperament. Sims’s efforts to reform the naval service are perhaps best told in his own words: “I am perfectly willing that those holding views differing from mine should continue to live, but with every fibre of my being I loathe indirection and shiftiness, and where it occurs in high place, and is used to save face at the expense of the vital interests of our great service (in which silly people place such a child-like trust), I want that man’s blood and I will have it no matter what it costs me personally.” Sims built on Scott’s inventions and made further modifications, resulting in new records in accuracy. “These elements were brought into successful combination by minds not interested in the instruments for themselves but in what they could do with them.”

“Sure of the usefulness of his gunnery methods, Sims then turned to the task of educating the Navy at large.” And this is where the fun really begins. His first strategy was to relay in writing the results of his methods “with a mass of factual data.” Sims authored over a dozen detailed data-driven reports on innovations in naval gunfire strage which he sent from his China Station to the powers that be in Washington DC. At first, there was no response from DC. Sims thus decided to change his tone by using deliberately shocking language in subsequent reports. Writes Sims: “I therefore made up my mind I would give these later papers such a form that they would be dangerous documents to leave neglected in the files.” Sims also decided to share his reports with other officers in the fleet to force a response from the men in Washington.

The response, however, was not exactly what Sims had hoped. Washington’s opinion was that American technology was generally as good as the British, which implied that the trouble was with the men operating the technology, which thus meant that ship officers ought to conduct more training. What probably annoyed Sims most, however, was Washington’s comments vis-a-vis the new records in accuracy that Sims claimed to have achieved. Headquarters simply waived these off as impossible. So while the first reaction was dead silence, DC’s second strategy was to try and “meet Sims’s claims by logical, rational rebuttal.”

I agree with the author, Elting Morison, that this second stage reaction, “the apparent resort to reason,” is the “most entertaining and instructive in our investigation of the responses to innovation.” That said, the third stage, name-calling, can be just as entertaining for some, and Sims took the argumentum ad hominem as evidence that “he was being attacked by shifty, dishonest men who were the victims, as he said, of insufferable conceit and ignorance.” He thus took the extraordinary step of writing directly to the President of the United States, Theodore Roosevelt, to inform him of the remarkable achievements in accuracy that he and Admiral Scott had achieved. “Roosevelt, who always liked to respond to such appeals when he conveniently could, brought Sims back from China late in 1902 and installed him as Inspector of Target Practice […]. And when he left, after many spirited encounters […], he was universally acclaimed as ‘the man who taught us how to shoot.'”

What fascinates Morison in this story is the concerted resistance triggered by Sims’s innovation. Why so much resistance? Morison identifies three main sources: “honest disbelief in the dramatic but substantiated claims of the new process; protection of the existing devices and instruments with which they identified themselves; and maintenance of the existing society with which they were identified.” He argues that the latter explanation is the most important, i.e., resistance due to the “fixed structure of our habits of mind and behavior” and the fact that relatively small innovations in gunfire accuracy could quite conceivably unravel the entire fabric of naval doctrine. Indeed,

“From changes in gunnery flowed an extraordinary complex of changes: in shipboard routines, ship design, and fleet tactics. There was, too, a social change. In the days when gunnery was taken lightly, the gunnery officer was taken lightly. After 1903, he became one of the most significant and powerful members of a ship’s company, and this shift of emphasis nat- urally was shortly reflected in promotion lists. Each one of these changes provoked a dislocation in the naval society, and with man’s troubled foresight and natural indisposition to break up classic forms, the men in Washington withstood the Sims onslaught as long as they could. It is very significant that they withstood it until an agent from outside, outside and above, who was not clearly identified with the naval society, entered to force change.”

The resistance to change thus “springs from the normal human instinct to protect oneself, and more especially, one’s way of life.” Interestingly, the deadlock between those who sought change and those who sought to retain things as they were was broken only by an appeal to superior force, a force removed from and unidentified with the mores, conventions, devices of the society. This seems to me a very important point.”  The appeal to Roosevelt suggests perhaps that no organization “should or can undertake to reform itself. It must seek assistance from outside.”

I am absolutely intrigued by what these insights might imply vis-a-vis innovation (and resistance to innovation) in the humanitarian sector. Whether it be the result of combining existing technologies to produce open-source crisis mapping platforms or the use of new information management processes such as crowdsourcing, is concerted resistance to such innovation in the humanitarian space inevitable as well? Do we have a Roosevelt equivalent, i..e, an external and somewhat independent actor who might disrupt the resistance? I can definitely trace the same stages of resistance to innovations in humanitarian technology as those identified by Morison: (1) dead silence; (2) reasoned dismissal; and (3) name-calling. But as Morison himself is compelled to ask: “How then can we find the means to accept with less pain to ourselves and less damage to our social organization the dislocations in our society that are produced by innovation?”

This question, or rather Morison’s insights in tackling this question are profound and have important implications vis-a-vis innovation in the humanitarian space. Morison hones in on the imperative of “identification” in innovation:

“It cannot have escaped notice that some men identified themselves with their creations- sights, gun, gear, and so forth-and thus obtained a presumed satisfaction from the thing itself, a satisfaction that prevented them from thinking too closely on either the use or the defects of the thing; that others identified themselves with a settled way of life they had inherited or accepted with minor modification and thus found their satisfaction in attempting to maintain that way of life unchanged; and that still others identified themselves as rebellious spirits, men of the insurgent cast of mind, and thus obtained a satisfaction from the act of revolt itself.”

This purely personal identification is a powerful barrier to innovation. So can this identifying process be tampered in order to facilitate change that is ultima-tely in everyone’s interest? Morison recommends that we “spend some time and thought on the possibility of enlarging the sphere of our identifications from the part to the whole.” In addition, he suggests an emphasis on process rather than product. If we take this advice to heart, what specific changes should we seek to make in the humanitarian technology space? How do we enlarge the sphere of our identifications and in doing so focus on processes rather than products? There’s no doubt that these are major challenges in and of themselves, but ignoring them may very well mean that important innovations in life-saving technologies and processes will go un-adopted by large humanitarian organiza-tions for many years to come.

Time-Critical Crowdsourcing for Social Mobilization and Crowd-Solving

My good friend Riley Crane just co-authored a very interesting study entitled “Time-Critical Social Mobilization” in the peer-reviewed journal Science. Riley spearheaded the team at MIT that won the DARPA Red Balloon competition last year. His team found the locations of all 10 weather balloons hidden around the continental US in under 9 hours. While we were already discussing alternative approaches to crowdsourcing for social impact before the competition, the approach he designed to win the competition certainly gave us a whole lot more to talk about given the work I’d been doing on crowd sourcing crisis information and near real-time crisis mapping.

Crowd-solving non-trivial problems in quasi real-time poses two important challenges. A very large number of participants is typically required couple with extremely fast execution. Another common challenge is the need for some sort of search process. “For example, search may be conducted by members of the mobilized community for survivors after a natural disaster.” Recruiting large numbers of participants, however, requires that individuals be motivated to actually conduct the search and participate in the information diffusion. Clearly, “providing appropriate incentives is a key challenge in social mobilization.”

This explains the rationale behind DARPA decision to launch their Red Balloon Challenge: “to explore the roles the Internet and social networking play in the timely communication, wide-area team-building, and urgent mobilization required to solve broad-scope, time-critical problems.” So 10 red weather balloons were discretely placed at different locations in the continental US. A senior analyst at the National Geospatial-Intelligence Agency is said to have characterized the challenge is impossible for conventional intelligence-gathering methods. Riley’s team found all 10 balloons in 8 hours and 36 minutes. How did they do it?

Some 36 hours before the start of the challenge, the team at MIT had already recruited over 4,000 participants using a “recursive incentive mechanism.” They used the $40,000 prize money that would be awarded by the winners of the challenge as a “financial incentive structure rewarding not only the people who correctly located the balloons but also those connecting the finder [back to the MIT team].” If Riley and colleagues won:

we would allocate $4000 in prize money to each of the 10 balloons. We promised $2000 per balloon to the first person to send in the cor- rect balloon coordinates. We promised $1000 to the person who invited that balloon finder onto the team, $500 to whoever invited the in- viter, $250 to whoever invited that person, and so on. The underlying structure of the “recursive incentive” was that whenever a person received prize money for any reason, the person who in- vited them would also receive money equal to half that awarded to their invitee

In other words, the reward offers by Team MIT “scales with the size of the entire recruitment tree (because larger trees are more likely to succeed), rather than depending solely on the immediate recruited friends.” What is stunning about Riley et al.’s approach is that their “attrition rate” was almost half the rate of other comparable social network experiments. In other words, participants in the MIT recruitment tree were about twice as likely to “play the game” so-to-speak rather than give up. In addition, the number recruited by each individual followed a power law distribution, which suggests a possible tipping point dynamic.

In conclusion, the mechanism devised by the winning team “simultaneously provides incentives for participation and for recruiting more individuals to the cause.” So what insights does this study provide vis-a-vis live crisis mapping initiatives that are volunteer-based, like those spearheaded by the Standby Volunteer Task Force (SBTF) and the Humanitarian OpenStreetMap (HOT) communities? While these networks don’t have any funding to pay volunteers (this would go against the spirit of volunteerism in any case), I think a number of insights can nevertheless be drawn.

In the volunteer sector, the “currency of exchange” is credit. That is, the knowledge and acknowledgement that I participated in the Libya Crisis Map to support the UN’s humanitarian operations, for example. I recently introduced SBTF “deployment badges” to serve in part the public acknowledgment incentive. SBTF volunteers can now add badges for deployments there were engaged in, e.g., “Sudan 2011”; “New Zealand 2011”, etc.

What about using a recursive credit mechanism? For example, it would be ideal if volunteers could find out how a given report they worked on was ultimately used by a humanitarian colleague monitoring a live map. Using the Red Balloon analogy, the person who finds the balloon should be able to reward all those in her “recruitment tree” or in our case “SBTF network”. Lets say Helena works for the UN and used the Libya Crisis Map whilst in Tripoli. She finds an important report on the map and shares this with her colleagues on the Tunisian border who decide to take some kind of action as a result. Now lets say this report came from a tweet that Chrissy in the Media Monitoring Team found while volunteering on the deployment. She shared the tweet with Jess in the GPS Team who found the coordinates for the location referred to in that tweet. Melissa then added this to the live map being monitored by the UN. Wouldn’t be be ideal if each could be sent an email letting them know about Helena’s response? I realize this isn’t trivial to implement but what would have to be in place to make something like this actually happen? Any thoughts?

On the recruitment side, we haven’t really done anything explicitly to incentivize current volunteers to recruit additional volunteers. Could we incentivize this beyond giving credit? Perhaps we could design a game-like point system? Or a fun ranking system with different titles assigned according to the number of volunteers recruited? Another thought would be to simply ask existing volunteers to recruit one or two additional volunteers every year. We currently have about 700 volunteers in the SBTF, so this might be one way to increase substantially in size.

I’m not sure what type of mechanism we could devise to simultaneously provide incentives for participation and recruitment. Perhaps those incentives already exist, in the sense that the SBTF response to international crises, which perhaps serves as a sufficient draw. I’d love to hear what iRevolution readers think, especially if you have good ideas that we could realistically implement!

LIFT09: Visualizing City Dynamics (Updated)

There were two neat presentations on data visualization of communication dynamics in urban environments. The first, by Stéphane Distinguin from UrbanMobs, included the following visualization of text messages sent throughout Paris during World Music Day:

The visualization below is of “mobile phone calls in Barcelona during the European Football Championship 2008 final and the day after the victory. You can easily notice the different game phases: kick off, half time, goal, end of the match and celebration of the Spanish team victory.”

Carlo Ratti from MIT’s SENSEable City Lab also gave a really neat talk on dynamic visualizations within cities and the patterns that arise.


Carlo showed engaging visualizations are a series of cities. Take the Real Time Rome project which aggregated data from mobile phones over different periods in Rome. The video represents the communication patterns across Rome during a Madonna concert.

Time zones influence the global rhythm of communications. In the video below, international calls between New York and 255 countries are visualized over a 24-hour period. “Areas of the world receiving and making fewer phone calls shrink while areas experiencing a greater amount of voice call activity expand.”

Carlo also showed an animation of “The Water Pavilion” located at the entrance to Expo Zaragoza 2008. Carlo and his team wanted to convey the sense of water in digital terms and therefore designed an interactive building made of water. Think of digital water like an inkjet printer on a large scale but with water instead of ink.

Patrick Philippe Meier