Tag Archives: Conflict Early Warning

NeoGeography and Crisis Mapping Analytics

WarViews is Neogeography

Colleagues at the Swiss Federal Institute of Technology Zurich (ETH) are starting to publish their research on the WarViews project. I first wrote about this project in 2007 as part of an HHI deliverable on Crisis Mapping for Humanity United. What appeals to me about WarViews is the initiative’s total “Neogeography” approach.



What is Neogeography? Surprisingly, WarViews‘s first formal publication (Weidmann and Kuse, 2009) does not use the term but my CrisisMappers colleague Andrew Turner wrote the defining chapter on Neogeography for O’Reilly back in 2006:

Neogeography means ‘new geography’ and consists of a set of techniques and tools that fall outside the realm of traditional GIS, Geographic Information Systems. Where historically a professional cartographer might use ArcGIS, talk of Mercator versus Mollweide projections, and resolve land area disputes, a neogeographer uses a mapping API like Google Maps, talks about GPX versus KML, and geotags his photos to make a map of his summer vacation.

Essentially, Neogeography is about people using and creating their own maps, on their own terms and by combining elements of an existing toolset. Neogeography is about sharing location information with friends and visitors, helping shape context, and conveying understanding through knowledge of place.

Compare this language with Wiedmann and Kuse, 2009:

[The] use of geographic data requires specialized software and substantial training and therefore involves high entry costs for researchers and practitioners. [The] War Views project [aims] to create an easy-to-use front end for the exploration of GIS data on conflict. It takes advantage of the recent proliferation of Internet-based geographic software and makes geographic data on conflict available for these tools.

With WarViews, geographic data on conflict can be accessed, browsed, and time-animated in a few mouse clicks, using only standard software. As a result, a wider audience can take advantage of the valuable data contained in these databases […].

The team in Zurich used the free GIS server software GeoServer, which reads “vector data in various formats, including the shapefile format used for many conflict-related GIS data sets.” This way, WarViews allows users to visualize data both statically and dynamically using Google Earth.

Evidently, the WarViews project is not groundbreaking compared to many of the applied mapping projects carried out by the CrisisMappers Group. (Colleagues and I in Boston created a Google Earth layer of DRC and Colombia conflict data well before WarViews came online).

That said the academic initiative at the University of Zurich is an important step forward for neogeography and an exciting development for political scientists interested in studying the geographic dimensions of conflict data.

Geographic Data

Geo-tagged conflict data is becoming more widely available. My Alma Matter, the Peace Research Institute in Oslo (PRIO), has made an important contribution with the Armed Conflict Location and Event Dataset (ACLED). This dataset includes geo-tagged conflict data for 12 countries between 1946 to present time.

In addition to ACLED, Wiedman and Kus (2009) also reference two additional geo-tagged datasets. The first is the Political Instability Task Force’s Worldwide Atrocities Dataset (PITF), which comprises a comprehensive collection of violent events against noncombatants. The second is the Peacekeeping Operations Locations and Event Dataset (Doroussen 2007, PDF), which provides geo-tagged data on interventions in civil wars. This dataset is not yet public.

Weidmann and Kuse (2009) do not mention Ushahidi, a Mobile Crisis Mapping (CMC) platform nor do the authors reference HHI’s Google Earth Crisis Map of Kenya’s Post-Election violence (2008). Both initiatives provide unique geo-tagged peace and conflict data. Ushahidi has since been deployed in the DRC, Zimbabwe and Gaza.

Unlike the academic databases referenced above, the Ushahidi data is crowdsourced and geo-tagged in quasi-real time. Given Ushahidi’s rapid deployment to other conflict zones, we can expect a lot more geo-tagged information in 2009. The question is, will we know how to analyze this data to detect patterns?

Crisis Mapping Analytics (CMA)

The WarViews project is “not designed for sophisticated analyses of geographic data […].” This is perhaps ironic given that academics across multiple disciplines have developed a plethora of computational methods and models to analyze geographic data over time and space. These techniques necessarily require advanced expertise in spatial econometric analysis and statistics.

The full potential of neography will only be realized when we have more accessible ways to analyze the data visualized on platforms like Google Earth. Neogeography brought dynamic mapping to many more users, but spatial econometric analysis has no popular equivalent.

This is why I introduced the term Crisis Mapping Analytics (CMA) back in August 2008 and why I blogged about the need to develop the new field of CMA here and here. The Harvard Humanitarian Initiative (HHI) is now spearheading the development of CMA metrics given the pressing need for more accessible albeit rigorous methods to identify patterns in crisis mapping for the purposes of early warning. Watching data played on Google Earth over and over will only take us so far, especially as new volumes of disparate datasets become available in 2009.

HHI is still in conversation with a prospective donor to establish the new field of CMA so I’m unable to outline the metrics we developed here but hope the donor will provide HHI with some funding so we can partner and collaborate with other groups to formalize the field of CMA.

Crisis Mapping Conference

In the meantime, my colleague Jen Ziemke and I are exploring the possibility of organizing a 2-3 day conference on crisis mapping for Fall 2009. The purpose of the conference is to shape the future of crisis mapping by bridging the gap that exists between academics and practitioners working on crisis mapping.

In our opinion, developing the new field of Crisis Mapping Analytics (CMA) will require the close and collegial collaboration between academic institutes like PRIO and operational projects like Ushahidi.

Jen and I are therefore starting formal conversations with donors in order to make this conference happen. Stay tuned for updates in March. In the meantime, if you’d like to provide logistical support or help co-sponsor this unique conference, please email us.

Patrick Philippe Meier

Crowdsourcing Honesty?

I set an all-time personal record this past week: my MacBook was dormant for five consecutive days. I dedicate this triumph to the delightful friends with whom I spent New Year’s. Indeed, I had the pleasure of celebrating with friends from Digital Democracy, The Fletcher School and The Global Justice Center on a Caribbean island for some much needed time off.


We all brought some good reading along and I was finally able to enjoy a number of books on my list. One of these, Dan Ariely’s “Predictably Irrational” was recommended to me by Erik Hersman, and I’m really glad he did. MIT Professor Ariely specializes in behavioral economics. His book gently discredits mainstream economics. Far from being rational agents, we are remarkably irrational in our decision-making, and predictably so.

Ariely draws on a number of social experiments to explicate his thesis.

For social scientists, experiments are like microscopes or strobe lights. They help us slow human behavior to a frame-by-frame narration of events, isolate individual forces, and examine those forces carefully and in more detail. They let us test directly and unambiguously what makes us tick.

In a series of fascinating experiments, Ariely seeks to understand what factors influence our decisions to be honest, especially when we can get away with dishonesty. In one experiment, participants complete a very simple math exercise. When done, the first set of participants (control group) are asked to hand in their answers for independent grading but the second set are subsequently given the answers and asked to report their own scores. At no point do the latter hand in their answers; hence the temptation to cheat.

In this experiment, some students are asked to list the names of 10 books they read in high school while others are asked to write down as many of the Ten Commandments as they can recall prior to the math exercise. Ariely’s wanted to know whether this would have any effect on the honesty of those participants reporting their scores? The statistically significant results surprised even him: “The students who had been asked to recall the Ten Commandments had not cheated at all.”

In fact, they averaged the same score as the (control) group that could not cheat. In contrast, participants who were asked to list their 10 high school books and self-report their scores cheated: they claimed grades that were 33% higher than those who could not cheat (control group).

What especially impressed me about the experiment […] was that the students who could remember only one or two commandments were as affected by them as the students who remembered nearly all ten. This indicated that it was not the Commandments themselves that encouraged honestly, but the mere contemplation of a moral benchmark of some kind.

Ariely carried out a follow up experiment in which he asked some of his MIT students to sign an honor code instead of listing the Commandments. The results were identical. What’s more, “the effect of signing a statement about an honor code is particularly amazing when we take into account that MIT doesn’t even have an honor code.”

In short, we are far more likely to be honest when reminded of morality, especially when temptation strikes. Ariely thus concludes that the act of taking an oath can make all the difference.

I’m intrigued by this finding and it’s potential application to crowdsourcing crisis information, e.g., Ushahidi‘s work in the DRC. Could some version of an honor code be introduced in the self-reporting process? Could the Ushahidi team create a control group to determine the impact on data quality? Even if impact were difficult to establish, would introducing an honor code still make sense given Ariely’s findings on basic behavioral psychology?

Patrick Philippe Meier

Covering the DRC – opportunities for Ushahidi

This blog entry was inspired by Ory’s recent blog post on “Covering the DRC – challenges for Ushahidi.” The thoughts that follow were originally intended for the comments section of Ushahidi’s blog but they surreptitiously morphed into a more in depth reflection. First, however, many thanks to Ory for taking the time to share the team’s experience in the DRC over the past few weeks.

Much of what Ory writes resonates with my own experience in conflict early warning/response. While several factors contribute to the challenge of Ushahidi’s deployment, I think one in particular regrettably remains a constant in my own experience: the link to early response, or rather the lack thereof. The main point I want to make is this: if Ushahidi addresses the warning-response gap, then Ushahidi-DRC is likely to get far more traction on the ground than it currently is.

To explain, if Ushahidi is going to provide a platform that enables the crowdsourcing of crisis information, then it must also facilitate the crowdsourcing of response. Why? For otherwise the tool is of little added value to the individuals who constitute said crowd, ie, the Bottom of the Pyramid (BoP) in conflict zones. If individuals at the BoP don’t personally benefit from Ushahidi, then they should not be spending time/resources on communicating alerts. As one of my dissertation committee members, Peter Walker wrote in 1991 vis-a-vis famine early warning/response systems in Africa:

It must be clearly understood that the informants are the most important part of the information system. It is their information […] upon which the rest of the system is based […]. The informant must be made to feel, quite rightly, that he or she is an important part of the system, not just a minion at the base, for ever giving and never receiving.

In 1988 (that’s write ’88), Kumar Rupesinghe published a piece on disaster early warning systems in which he writes that civil society has

… a vital role to play in the development of a global, decentralized early warning system. They now need the capacity to build information systems and to provide the basis for rapid information exchange. In general [civil society] will have to confront the monopolization of information with a demand for the democratic access to information technology.

Information on local concerns must be available to the local structures in society. The right to be informed and the right to information have to find entry into international discussions.

Ushahidi’s crowdsourcing approach has the potential to reverse the monopolization of information and thereby create a demand for access to conflict information. Indeed, Ushahidi is starting to influence the international discourse on early warning (forthcoming reports by the EC and OECD). However, it is the mobile crowdsourcing of response that will create value and thereby demand by the BoP for Ushahidi.

Put it this way, Twitter would be far less useful if it were limited to one (and only one) global website on which all tweets were displayed. What makes Twitter valuable is the ability to select specific feeds, and to have those feeds pushed to us effortlessly, using Twhirl or similar programs, and displayed (in less than 141 characters) on our computer screens in real time. At the moment, Ushahidi does the equivalent of the former, but not the latter.

Yet the latter is precisely where the added value to the individual lies. An NGO may be perfectly content with Ushahidi’s current set up, but NGOs do not constitute the BoP; they are not the “fundamental unit” of crowdsourcing—individuals are. (Just imagine if Wikipedia entries could only be written/edited by NGOs).

This mismatch in fundamental units is particularly prevalent in the conflict early warning/response field. NGOs do not have the same incentive structures as individuals. If individuals in at-risk communities were to receive customized alerts on incidents in/near their own town (if they themselves send alerts to Ushahidi), then that response presents a far more direct and immediate return on investment. Receiving geo-specific alerts in quasi real-time improves situational awareness and enables an individual to take a more informed decision about how to respond to the alerts. That is added value. The BoP would have an incentive, empowerment, to crowdsource crisis information.

Here’s a scenario: if an individual texts in an alert for the first time, Ushahidi should: (1) contact that person as soon as possible to thank them for their alert and, (2) ask them what SMS alerts they would like to receive and for what town(s). I guarantee you this person will spread the word through their own social network and encourage others to send in alerts so that they too may receive alerts. (Incidentally, Erik, this is the strategy I would recommend in places like Jos, Nigeria).

In summary, while the Ushahidi team faces a multitude of other challenges in the DRC deployment, I believe that addressing the early response dimension will render the other challenges more manageable. While the vast majority of conflict early warning systems are wired vertically (designed by outsiders for outsiders), the genius of Ushahidi is the paradigm shift to horizontally wired, local early warning/response, aka crowdsourcing.

In a way, it’s very simple: If Ushahidi can create value for the BoP, the client base will necessarily expand (substantially). To this end, Ushahidi should not be pitched as an early warning system, but rather as an early response service. This is one of the reasons why I am trying hard to encourage the operationalization of mobile crisis mapping.

Policy Briefing: Information in Humanitarian Responses

The BBC World Service Trust just released an excellent Policy Brief (PDF) on “The Unmet Need for Information in Humanitarian Responses.” The majority of the report’s observations and conclusions are in line with the findings identified during Harvard Humanitarian Initiative’s (HHI) 18-month applied research project on Conflict Early Warning and Crisis Mapping.

I include below excerpts that resonated particularly strongly.

  • People need information as much as water, food, medicine or shelter. Information can save lives, livelihoods and resources. Information bestows power.
  • Effective information and communication exchange with affected populations are among the least understood and most complex challenges facing the humanitarian sector in the 21st century.
  • Disaster victims need information about their options in order to take any meaningful choices about their future. Poor information follow is undoubtedly the biggest source of dissatisfaction, anger and frustration among affected people.
  • Information—and just as important communication—is essential for people to start claiming a sense of power and purpose over their own destiny.

In this context, recall the purpose of people-centered early warning as defined by the UN International Strategy for Disaster Reduction (UNISDR) during the Third International Conference on Early Warning (EWC III) in 2006:

To empower individuals and communities threatened by hazards to act in sufficient time and in an appropriate manner so as to reduce the possibility of personal injury, loss of life, damage to property and the environment, and loss of livelihoods.


Other important observations worth noting from the Policy Brief:

  • Sometimes information is the only help that can be made available, especially when isolated populations are cut off and beyond the reach of aid.
  • There are still misplaced assumptions and confusion about how and what to think about information and communication—and where organizationally to locate it. Humanitarian actors systematically fail to see the difference between public relations and communications with affected populations, and thus funds neither the expertise nor infrastructure necessary.
  • The information needs of people affected by disasters remain largely unmet because the people, systems and resources that are required to meet them simply don’t exist in a meaningful way.
  • The humanitarian system is not equipped with either the capacity or the resources to begin tackling the challenge of providing information to those affected by crises.
  • A prior understanding of how populations in disaster prone areas source information is vital in determining the best channels for information flow: for example, local media, local religious networks and local civil society groups.
  • Studies have shown that affected populations go to great lengths to reinstate their media infrastructure and access to information at the earliest opportunity following a disaster. Relief efforts should recognize these community-driven priorities and response accordingly.

My one criticism of the report has to do with the comments in parentheses in this paragraph:

Rebuilding the local media infrastructure for sustained operations must be prioritized as aid efforts continue. This may be as simple as providing a generator to a radio station that has lost its electricity supply, using UN communications structures such as the World Food Program towers to relay local radio stations (though in politically complex environments this needs careful thought)…

The BBC’s Policy Brief focuses on the unmet need for information in humanitarian responses but leaves out of the equation “politically complex environments.” This is problematic. As the UNISDR remarked in it’s 2006 Global Survey of Early Warning Systems (PDF), “the occurrence of “natural” disasters amid complex political crises is increasingly widespread: over 140 natural disasters have occurred alongside complex political crises in the past five years alone.”

Operating in politically volatile, repressive environments and conflict zones presents a host of additional issues that the majority of policy briefs and reports tend to ignore. HHI’s research has sought to outline these important challenges and to highlight potential solutions both in terms of technology and tactics.


The importance technology design has been all but ignored in our field. We may continue to use every day communication tools and adopt them for our purposes, but these will inevitably come with constraints. Mobile phones were not designed for operation in hostile environments. This means, for example, that mobile phones don’t come preinstalled with encrypted SMS options. Nor are mobile phones designed to operate in a peer-to-peer (mesh) configuration, which would render mobile phones less susceptible to repressive regimes switching off entire mobile phone networks.

What are today’s most vexing problems in the field of humanitarian early warning and response? This is a question often posed by my colleague Ted Okada to remind us that we often avoid the most important challenges in the humanitarian field. It’s one thing to respond in a post-disaster environment with easy access to refugee populations and donor funding. It’s quite another to be operating in a conflict zone, with restrictions on mobility, with no clear picture of the effected population and with donors reluctant to fund experimental communication projects.

It is high time we focus our attention and resources on tackling the most vexing issues in our field since the solutions we develop there will have universal application elsewhere. In contrast, identifying solutions to the less vexing problems will be of little benefit to large humanitarian community operating in political complex environments. As my colleague Erik Hersman is fond of saying, “If it works in Africa, it’ll work anywhere.” I’ve been making a similar argument over the past year: “If it works in conflict zones, it’ll work anywhere.”

Patrick Philippe Meier

Crisis Mapping the Conflict in Georgia (Updated)

Update: Jon Thompson had initially mistakenly blogged that all roads/cities in Georgia had “disappeared” from Google Maps and Google Earth. A colleague of mine at Google has since informed me that they never had a roads layer for Georgia. According to this same contact, Google has just released this formal statement:

It is untrue to suggest, as some media reports have, that Google has removed data or imagery from our Google Maps product in Georgia, Armenia or Azerbaijan.  We have never had local data for those countries and that is why local details such as landmarks and cities do not appear.

An initial crisis map of the escalating violence between Georgia and Russia has been created for Google Earth. While dynamic maps add more value than static maps, we need a more interactive interface that permits for crowdsourcing crisis information in quasi-real time with fully geo-referenced information.

Here I am thinking of Ushahidi and the Humanitarian Sensor Web (HSW). Humanitarian organizations have already moved into the disputed region and have no doubt learned important information, which is likely changing every hour. But the one person behind these initial Google Earth maps may not have easy access to organizations on the ground.

This is precisely when we need a crisis mapping platform that enables field-based organizations and local communities to text in important information on events as they unfold. As more information surfaces, we’ll need that same platform to provide quantitative, time-stamped analysis within the same interface. Finally, we would want to let affected communities know how to receive or subscribe to this information as it is posted and validated. This is where Dial-Up-Radio could come in handy.

Patrick Philippe Meier

Crisis Mapping, Dynamic Visualization and Pattern Recognition

My interest in dynamic networks and data visualization dates back several years. Indeed, one of the reasons I participated in the Santa Fe Institute’s (SFI) Complex Systems Summer School (CSSS) back in 2006 was precisely because of my long-time interest in applying this area of research to conflict analysis. But it wasn’t until recently that I began to connect those dots to my current research on crisis mapping and pattern recognition in complex emergencies.

Below is one rendering of a dynamic network that I used when co-teaching a  graduate seminar on “Managing Complex Systems” in Fall 2007. The visualization depicts flight patterns across the US. I used this simply to illustrate that certain patterns emerge when data is visualized geographically and temporally across multiple scales.

Another example is the dynamic rendering of information flow in the Blogosphere. I’ve included a picture below but the video for this animation is also worth watching. What does this have to do with crisis mapping? The point is to provide comparable visual renderings of dynamic conflict data at multiple levels of analysis, both spatially and temporally. When doing so, potential patterns and linkages can emerge. Mapping the “fluid dynamics” of conflict, or contagion effects, can be particularly insightful.

One really stunning rendering of a dynamic network was recently posted on the BBC’s technology news site. Using satellite imagery and ground breaking computer imaging we can for the first time visualize stunning patterns that emerge across the UK as seen from the sky. Like the flight tracking video above, this short BBC video is also highly worth watching.

These examples may serve as worthy goals for the new field of crisis mapping analytics, or CMA. It remains to be seen whether we can pull this off. More importantly, however, the question is whether this exercise will get us any closer to saving lives in complex emergencies.

Patrick Philippe Meier

NearMap Better Than Google Maps for Crisis Mapping?

NearMap, a geospatial media company bought out by Ipernica this week, claims that its “breakthrough technology enables photomaps to be updated much more frequently than other providers such as Google Earth, which can be many months out of date.”

NearMap’s technology enables very high resolution aerial photomaps with multiple angle views to be created at a fraction of the cost of traditional solutions… For the first time, people will be able to see the environment change over time, as NearMap’s online photomaps allow users to move back and forward month by month to see changes occur, such as the construction of a home or development of a new road. [And] with NearMap’s revolutionary approach to high resolution photomaps, it has achieved its objective of a 20-fold operating cost reduction over current industry practices.

Ipernica says that NearMap’s ultimate goal is to cover over 20 percent of the world’s population (700 cities) with photomaps updated at least on a monthly basis.According to Ipernica, NearMap has fully automated the process of creating very high definition photomaps and has developed a complete chain of technologies to address these challenging requirements.

If NearMap (or a competing company) broadens its scope to rural populations, the technology could be a particularly useful tool for the purposes of data collection and crisis mapping.

Patrick Philippe Meier

Second Response to Paul Curion on Global Voices

This is cross-posted in the comments section on Paul’s blog as well.

Paul: Patrick, I think your bias is showing. Your use of the word “extremist” looks dangerously close to being a euphemism for “things that I disagree with”; corruption, for example, is not an “extremist” action.

I completely agree, corruption is not extremist action (although it depends whether you are facing the direct consequences of such action). But you’re not responding to the main point I’m making (understandably since I have not been as careful as I should be in choosing appropriate words in formulating my responses to your comments, my apologies for that). Citizen media, investigative journalism, the use of Web 2.0 tools to document instances of human rights violations, government corruption, etc. are ways to expose wrong doing. They are “new” sources of potentially important information for conflict early warning/response. We no longer have to rely strictly on state media or national media. I think that’s a good thing. This of course does not mean that citizen journalism will provide all the answers to continuing challenges in the field.

On the issue of bias, which you brought up more than once, I wouldn’t want to live in a world completely free of bias, there would be no learning, no creation of knowledge, etc. (the analogy I would use is entropy and the heat-death of the universe). But that’s just a side point, more of a philosophical issue which does not add to the conversation.

As regards legal actions, there was discussion about how bloggers could work together to start influencing change in legislation, but also how to use existing laws to expose governments as clearly violating their own laws. That said, I’m really hoping the GV folks will start contributing to this conversation, because the two of us could go on forever and I’m not qualified, nor is it my place, to speak on behalf of GV. I’m an outsider and they may very well take issue with some of my points as well. So I hope the conversation leads to more “global voices” participating.

But what is the response? I’m still not seeing it – not in the sense that it doesn’t appear in a peer-reviewed journals (I don’t actually read peer-reviewed journals…), but in the sense that I can’t see what the response could be. Let me be clear: blogging is a response, data visualisation is a response, but not the type of response that I think you’re talking about.

Indeed, blogging in itself is a response. The operational responses, which I hope our colleagues from Kenya will share with us in their own words, are more micro-level responses in the form of real time information sharing. Kenya’s bloggers filled a notable vacuum in the national media following the elections (I was in Nairobi during this time). I consider this an important response. FAST field monitors did not contribute to this type of information sharing. Events were coded and stored on servers in Bern.

I could be wrong, however. I get the sense that you believe that this activity is worthwhile simply for its own sake – as part of the democratic process – and I’d tend to agree. However what I read here – and in the other discussions around the summit – goes beyond simply blogging because it’s worthwhile. It has a programmatic element, a directional element – but that means that the bar is higher.

I completely fully agree.

Again, your bias is showing – who decides which blogs are to be “trusted”, and what does “trusted” mean in this context? How do you know that GV bloggers have a “vibrant and pro-active network”? And what about the voices on the other side – the “extremist” side, who may be “extremist” precisely because they lack a voice? These are deeper questions which I am sure were discussed at the Summit and elsewhere, but their existence should make you wary of proclaiming their superiority without at least some qualifications.

Biased Patrick: You decide which blogs are to be trusted, you develop your own community of trusted sources. The iRevolution is about you, the individual, who stands to be more empowered to make his/her own more-informed choices. There were some 80 GV bloggers in Budapest and I spent the better part of three days, from morning to dinner with them. They struck me as a vibrant and pro-active network. Much of this came from the side conversations during breaks, etc. As for the voices on the “extremist” side, they are doing really well in adopting new technology for disseminating their “extremist” points of views. Take Al Qaeda for example, they have a superb, first-rate communications department which has allowed them to make use of Web 2.0 platforms to increase visibility, recruitment and improve training.

Representative of who? I ask you because while I was reading David Sasaki’s excellent post on the GV summit, I was struck by the following passage:

As incredibly diverse as the global blogosphere is, the ‘blogger demographic’ tends to very homogenous. From Tanzania to Tasmania, most bloggers live in the wealthy neighborhoods of urban centers, most are well educated, and most belong to the majority groups of their countries.

which is something which I would have guessed in more general terms. I don’t know what the profile of FASTs field monitors was, but I’m guessing it wasn’t that much different to the current GV profile?

I misunderstood what you meant in your previous response, so I completely take your point. Representative of who remains an open question. But I also think that this misses the more important point that I was hoping to make. I don’t want to be cornered into arguing about what GV is or is not. What my original post argued was that we (the conflict early warning/response community) may gain from paying more attention to blogs as a source of local information for the purposes of early warning/response. Hence my contrast with FAST. Our colleagues in Kenya were blogging on a virtually real-time basis, providing up-to-date information on events taking place across the country. The point is that they delivered, and took it upon themselves to do so; regardless of whether they live in wealthy neighborhoods or not. Many of them were in the streets as events were unfolding. This is the kind of local information that I value.

Again, I understand that what you’re reacting to are my somewhat sweeping claims about democracy, etc. But I don’t want this to distract from the main point I’m trying to get across, ie, that our community has some things to learn from the GV community and vice versa. Hence my hoping that this dialogue will prompt our GV colleagues to contribute (and possibly correct some of my own statements).

What you’ve outlined isn’t accountability in any strong sense – all of the actions that you describe here are certainly part of a dialogue, but I’m not sure they’re accountability mechanisms. I may be being unfair in my accusation here – it’s hard to know what I want GV to be accountable for – but you can be certain that this will be an issue which it will face in future.

I grant you that my take on what constitutes accountability is not the traditional, institutional, centralized version. Perhaps I’m too biased (again ; ) given that I identify more with the open source, decentralized philosophy of the Web 2.0 generation. Again, the piece by Benkler will hopefully convince you that there is a real significant change occurring, but perhaps I’m getting ahead of myself vis-a-vis the probable impact for the conflict early warning community.

I wasn’t at the GV Summit, and I haven’t had the discussions you’ve had with people like Ushaidi, so I am not as well-placed as you to talk about their status and plans. However my complaint is that I’m not seeing the evidence that these projects are having the impact that they (you?) claim, and I just want to be persuaded of that impact before I make any claims about them.

We’re definitely on the same page vis-a-vis the critical importance of demonstrating impact. This has been the very basis of my criticisms with respect to the majority of operational conflict early warning systems. So I’m equally interested in identifying whatever impact Ushahidi has had. But that was not the purpose of my post. See this post on crisis mapping analytics where I ask the same question as you do regarding impact.

I’m going to give this thread a rest now in the hopes that our GV and Ushahidi colleagues may jump in with their comments. Thanks again for the reality-check, Paul.

Patrick Philippe Meier

Global Voices and Conflict Early Warning

I’d like to follow up on my previous blog, “Global Voices and Humanitarian Action,” and focus specifically on the link between bloggers at Global Voices and the field of conflict early warning/response.

Early warning signals appear most clearly to those immediately around the disputants. “One cannot solely rely on the statistics produced by leading international development agencies” to monitor potential for conflict escalation (1). In fact, “according to 1994 World Bank data, Rwanda was the most egalitarian country among all low-income and middle-income countries in the world” (2). To this end, more micro level analysis is needed to capture “The View from Below,” i.e.,  the underlying web of complex political, social and economic networks. In addition, “if we are to make a difference for the majority of the people who suffer the horrible effects of civil wars, we ought to also focus our research on how ordinary people adjust their lives to cope with the constraints and opportunities brought about by civil war” (3).

But most conventional conflict early warning systems generate “macro level analysis and policy prescriptions that are generally based on a snapshot rather than a dynamic view of the changing situations on the ground” (4). In fact, the majority of references to conflict early warning are to top-down, inter-governmental  early warning systems with limited (if any) links to local communities. The field of conflict early warning is therefore shifting towards a more bottom-up approach, stressing the need for something like an indigenous “local information network” to get a better glimpse of “the view from below”. For sure, “a democratic flow of information is the first condition for a democratic and open system of warning and resolution” (5).

Enter Global Voices:

At a time when the international English-language media ignores many things that are important to large numbers of the world’s citizens, Global Voices aims to redress some of the inequities in media attention by leveraging the power of citizens’ media. We’re using a wide variety of technologies – weblogs, podcasts, photos, video, wikis, tags, aggregators and online chats – to call attention to conversations and points of view that we hope will help shed new light on the nature of our interconnected world.

This is precisely what the FAST early warning project at Swisspeace attempted to do. FAST drew on “Local Information Networks” (LINs) of field monitors to code event-data as reported by the local news media. These would then be aggregated and visualized as a time series to determine whether any patterns of conflict escalation could be identified. The process, however, was tedious and hierarchical. Field monitors were not included in the analysis (which was done only in Bern, Switzerland), nor were they included in galvanazing response or even formulating response options.

Long-distance expertise and “analytical capacity alone will never be sufficient for generating effective response,”  since “to have significance operationally, analysis cannot simply be factual but also has to address the issue of perception (e.g., perceived needs, values and symbols); Indeed, prevent[ing] violent conflict requires not merely identifying causes and testing policy instruments but building a political movement” since “the framework for response is inherently political, and the task of advocacy for such response cannot be separated from the analytical tasks of warning” (6).  These form part of the lessons recently learned in the field of conflict early warning.

Global Voices is a far more effective local information and response network than FAST ever was. FAST’s organizational structure was hierarchical, compared to the decentralized nature of the Global Voices network. Bloggers at Global Voices are directly linked to local social and political networks. They have their ears to the ground. They are some of the first to know when “Something is not right,” as Kenyan blogger Daudi remarked on the morning December 30th, 2007 in Nairobi. As more of the irregularities of the voting surfaced, bloggers quickly found themselves as citizen reporters, using twitter, photoblogging and other tools to document and respond to the escalating violence. Ethan Zuckerman writes,

Daudi argues that Kenya was especially prepared to cover the situation due to the richness and maturity of the blogosphere. There are at least 800 Kenyan bloggers, who are both fiercely independent and tightly linked together. “If you build a new Kenyan blog, if you put it into the webring, you’ll have a thousand viewers the first day.” Many of these bloggers were anxious to cover the elections. Daudi tells us he was out on the streets at 6am, photographing lines and polling places; other bloggers were out at 3am. Some bloggers were actually standing for election, others were embedded with foreign diplomats, visiting polling sites as election monitors.

FAST’s field monitors were limited in the technologies there were provided with. Bloggers, on the contrary, make use of all social media and Web 2.0 tools available. They are the new citizen field monitors. Unlike the local information networks at FAST and are conventional conflict early warning systems, they are not paid informants. They volunteer their time because they are dedicated to a more  transparent and democratic society. They are engaged and have a direct stake in peace. Why have we in the conflict prevention community not paid more attention to the rich information these bloggers provide? Why are we not subscribing to Global Voices? Why are we not using our sophisticated natural language parsers to quantity subtle changes in bloggers’ opinions and perceptions in real time?

The answer? Because the conflict early warning field is still in the middle ages when it comes to the use of emerging information communication technologies. A comprehensive OECD report (PDF) on existing operational early warning systems concluded in May 2008 that “most inter-governmental and non-governmental systems […] have not gone beyond the use of email and websites for dissemination, and communication technology for data collection.”

In addition, as the Center for Strategic International Studies (CSIS) recently reported in a “Review of Conflict Prediction Models and Systems,” one the most significant findings from the study is that a “small pool of [academic] experts dominate the field.” Both these factors are antithetical to the observation made by Rupesinghe exactly 20 years ago (!) vis-a-vis conflict early warning and response systems: “a democratic flow of information is the first condition for a democratic and open system of warning and resolution.” Stress on democratic and flow. It is high time we in the humanitarian community pay more attention to Global Voices.

Crimson Hexagon: Early Warning 2.0?

The future of automated textual analysis is Crimson Hexagon, a patent pending text reading technology that allows users to define the questions they want to ask, and crawl the blogosphere (or any text-based source) for fast, accurate answers. The technology was created under the aegis of Harvard University Professor Gary King.

I met with the new company’s CEO this week to learn more about the group’s parsing technology and underlying statistical models. Some UN colleagues and I are particularly interested in the technology’s potential application to conflict monitoring and analysis. At present, early warning units within the UN, and other international (regional) organizations such as the OSCE, use manual labor to collect relevant information from online sources. Most units employ full-time staff for this, often meaning that 80% of an analyst’s time is actually used to collect pertinent articles and reports, leaving only 20% of the time for actual analysis, interpretation and policy recommendations. We can do better. Analysts ought to be spending 80% of their time analyzing.

Crimson Hexagon is of course not the first company to carry out automated textual analysis. Virtual Research Associates (VRA) and the EC’s Joint Research Center (JRC) have both been important players in this space. VRA developed GeoMonitor, a natural language parser that reads the headlines of Reuters and AFP news wires and codes “who did what, to who, where and when?” for each event reported by the two media companies. According to an independent review of the VRA parser by Gary King and Will Lowe (2003),

The results are sufficient to warrant a serious reconsideration of the apparent bias against using events data, and especially automatically created events data, in the study of international relations. If events data are to be used at all, there would now seem to be little contest between the machine and human coding methods. With one exception, performance is virtually identical, and that exception (the higher propensity of the machine to find “events” when none exist in news reports) is strongly counterbalanced by both the fact that these false events are not correlated with the degree of conflict of the event category, and by the overwhelming strength of the machine: the ability to code huge numbers of events extremely quickly and inexpensively.

However, as Gary King mentioned in a recent meeting I had with him this month, VRA’s approach faces some important limitations. First, the parser can only parse the headline of each newswire. Second, adding new media sources such as BBC requires significant investment in adjusting the parser. Third, the parser cannot draw on languages other than English.

The JRC has developed the European Media Monitor (EMM). Unlike VRA’s tool, EMM is based on a key-word search algorithm, i.e., it uses a search engine like Google. EMM crawls online news media for key words and places each article into a corresponding category, such as terrorism. The advantage of this approach over VRA’s is that EMM can parse thousands of different news sources, and in different languages. The JRC recently set up an “African Media Monitor” for the African Union’s Continental Early Warning System (CEWS). However, this approach nevertheless faces limitations since analysts still need to read each article to understand the nature of the terrorist event.

Google.org is also pursuing text-based parsing. This initiative stems from Larry Brilliant’s TED 2006 prize to expand the Global Public Health Information Network (GPHIN) for the purposes of prediction and prevention:

Rapid ecological and social changes are increasing the risk of emerging threats, from infectious diseases to drought and other environmental disasters. This initiative will use information and technology to empower communities to predict and prevent emerging threats before they become local, regional, or global crises.

Larry’s idea led to the new non-profit InSTEDD, but last time I spoke with the team, they were not pursuing this initiative. In any case, I wouldn’t be surprised if Google.com were to express an interest in buying out Crimson Hexagon before year’s end. Hexagon’s immediate clients are private sector companies who want to monitor in real-time their brand perception as reported in the blogosphere. The challenge?

115 million blogs, with 120,000 more added each day. As pundits proclaim the death of email, social web content is exploding. Consumers are generating their own media through blogs and comments, social network profiles and interactions, and myriad microcontent publishing tools. How do we begin to know and accurately quantify the relevant opinion that’s out there? How can we get answers to specific questions about online opinion as it relates to a particular topic?

The accuracy and reliability of Crimson Hexagon is truly astounding. Equally remarkable is the fact that the technology developed by Gary King’s group parses every word in a given text. How does the system work? Say we were interested in monitoring the Iranian blogosphere—like the Berkman Center’s recent study. If we were interested in liberal bloggers and their opinion on riots (hypothetically taking place now in Tehran), we would select 10-30 examples of pro-democratic blog entries addressing the ongoing riots. These would then be fed into the system to teach the algorithm about what to look for. A useful analogy that Gary likes to give is speech recognition.

The Crimson Hexagon parser uses a stemming approach, meaning that every word in a given text is reduced to it’s root word. For example, “rioting”, “riots”, “rioters”, etc., is reduced to riot. The technology creates a vector of stem words to characterize each blog entry so that thousands of Iranian blogs can be automatically compared. By providing the algorithm with a sample of 10 or more blogs on, say, positive perceptions of rioting in Tehran were this happening now, the technology would be able to quantify the liberal Iranian bloggers’ changing opinion on the rioting in real time by aggregating the stem vectors.

Crimson Hexagon is truly pioneering a fundamental shift in the paradigm of textual analysis. Instead of trying to find the needle in the haystack as it were, the technology seeks to characterize the hay stack with astonishing reliability such that any changes in the hay stack (amount of hay, density, structure) can be immediately picked up by the parser in real time. Furthermore, the technology can parse any language, say Farsi, just as long as the sample blogs provided are in Farsi. In addition, the system has returned highly reliable results even when using less than 10 samples, and even when the actual blog entry had less than 10 words. Finally, the parser is by no means limited to blog entries, any piece of text will do.

The potential for significantly improving conflict monitoring and analysis is, in my opinion, considerable. Imagine parsing Global Voices in real time, or Reliefweb and weekly situation reports across all field-based agencies world wide. Crimson Hexagon’s CEO immediately saw the potential during our meeting. We therefore hope to carry out a joint pilot study with colleagues of mine at the UN and the Harvard Humanitarian Initiative (HHI). Of course, like any early warning initiative, the link to early response will dictate the ultimate success or failure of this project.

Patrick Phillipe Meier