Tag Archives: Haiti

Here Come the Crowd-Sorcerers: How Technology is Disrupting the Humanitarian Space and How Easy It Is

Posted on July 27, 2010 | 19 comments

I’ve recently been cc’d on an email thread in which a humanitarian group has started to “air out some latent issues and frustrations” vis-a-vis the use of crowdsourcing in emergencies. I applaud them for speaking up and credit them for coining the fantastic term “crowd-sorcerers” which is brilliant! The group is apparently preparing to publish a report concerning Humanitarian Information Management in Haiti. I really hope to appear in their chapter on “The Crowd-Sorcerers.”

I wonder which kind of sorcerer I am...

I too prefer candid conversations over diplomatic pillow talk. Lets be honest, it’s not actually difficult to disrupt the humanitarian system. It’s  hierarchical, overly bureaucratic, slow, often unaccountable and at times spectacularly corrupt. But I want to make sure my tone here is not misunderstood. I want to be constructive but playful and provocative at the same time, to “lighten things” up a bit. We often take ourselves way too seriously, too often. That’s why I absolutely love the term Crowd-Sorcerer! Lets use Muggles for our humanitarian friends.

I’ll first lay out some of the frustrations aired by the Muggles in their own words so I don’t  misrepresent their concerns—some of which are obviously valid (but not necessarily new). I’ll be reviewing these concerns in a series of blog posts, so stay tuned for future episodes in the new Crowd-Sorcerer Series! Caution: in case it’s not yet obvious, I will be deliberately provocative and playful in this series.

Muggles: Unless there are field personnel providing “ground truth” data, consumers will never have reliable information upon which to build decision support products. Crowdsourcing may be a quick way to get a message out, but it is not good information unless there is on-the-ground verification going on.

Not sure how you’d interpret these words but what they say to me is this: unless information comes from official field personnel, i.e., Muggles, it’s absolutely useless and should be dumped in the trash. I personally find that somewhat… is colonial too provocative?

Crisis information that was crowdsourced using the distributed short code 4636 in Haiti helped save hundreds of lives according to the Marine Corps. The vast majority of this information could not be verified and yet both the Marine Corps and Coast Guard used this as one of their feeds while FEMA encouraged the crowd-sorcerers to continue mapping, calling the crisis map of Haiti the most comprehensive and up-to-date source of information available to Muggles.

There’s another extraordinary story here, and that’s the story of Mission 4636. Tens of thousands of incoming text messages from disaster affected communities in Haiti were translated from Haitian Kreyol to English in near real-time thanks to crowdsourcing. These text messages were translated by thousands of Haitian Kreyol speaking volunteers from all around the world.

Map of volunteer locations

Without this crowdsourcing, the Marine Corps, Coast Guard, FEMA and others could not have used the information streaming in from 4636 as effectively as they did. And guess what? The original platform that was used to do this translation-by-crowdsourcing was built overnight by Brian Herbert, a 20-something tech developer at Ushahidi.

Where were the Muggles then? I’m sorry to put it in these terms but if we listened to (and waited for) Muggles all the time, then perhaps several hundred more people would have needlessly lost their lives in Haiti.

A forthcoming USIP report that reviews the deployment of the Ushahidi platform found that Haitian NGO’s and local civil society groups were physically barred from entering LogBase—the humanitarian community’s compound near the airport in Port-au-Prince. One Haitian NGO rep who was interviewed said he felt like a foreigner in his own country when he wasn’t allowed to enter LogBase and attend meetings where he could share vital information on urgent needs.

Now tell me, how is trashing Haitian text messages any different than  physically excluding Haitians from having a voice at LogBase? Because the so-called “unwashed masses” don’t have the “right” credentials as defined by the Muggles? Either way, they are excluded from having a stake in the hierarchical system that is supposed help them.

Incidentally, a fully independent evaluation led by a team of three accomplished experts in M&E  (monitoring and evaluation) are currently carrying out their impact assessment of the Ushahidi deployment during the emergency period. They will be in Haiti to for the field work and yes, one member of the team speaks fluent Kreyol. The PI from Tulane University has over 20 years of relevant experience. It would make absolutely no sense for Ushahidi to carry out this review.

Ushahidi has little to no expertise in M&E and such a review would likely be viewed as biased if Ushahidi was authoring it. In fact, Ushahidi didn’t even commission the evaluation, The Fletcher Team did, and they should be applauded for doing so. By the way, as I have blogged here, it is misguided to assume that experts in, say development, are by definition experts at evaluating development projects. M&E is a separate area of expertise and profession in it’s own right. Anyone who has taken M&E 101 will know this from the first lecture.

We’re going to a commercial break now, but stay tuned for the next episode: “Here Come the Crowd-Sorcerers: Is it Possible to Teach an Old (Humanitarian) Dog New Tech’s?”

Patrick Philippe Meier

19 Comments

Posted in Crisis Mapping, Crowdsourcing, Social Media

Tagged

Crowdsourcing Disaster Preparedness: Time for Some Disruption

Posted on July 9, 2010 | 22 comments

We’re well into hurricane season here in Haiti but good luck finding a map on hurricane shelters and evacuation routes. One UN agency was supposed to update a 2007 map but then dropped the ball. Another agency thought they’d take on the task but now there are legal concerns since only the government has the right to decide on official emergency routes and shelters. The result? A highly vulnerable population remains largely unprepared for what many expect will be a busy hurricane season.

Creating country wide maps of hurricane shelters and evacuation routes is obviously no easy task. Or is it? If we adopt the typical top down mentality, then yes, we’re talking about just a handful of people being charged with a huge project that will take them weeks to carry out. With this approach, the maps will completed well after the end of hurricane season. Great.

What if we distributed the task and crowdsourced the maps? We could use the 2007 map of hurricane shelters as a starting point and send out targeted text messages to hundreds of mobile phone users near each of these shelters asking them to report on the condition of each shelter and the access routes. We could triangulate the responses for validation purposes. This could be done tomorrow by using a free short code just like we did during the disaster response operations earlier this year. Since the lottery is big in Haiti, this could serve as an incentive: “timely and accurate replies will qualify you for a raffle.” DigiCel has already conducted SMS raffles in the past, so there is a precedent.

The SMS replies could then be analyzed over the weekend and the results shared with local radio stations early next week. The latter could then broadcast this information on a daily basis. In the meantime, government and UN officials could conduct site visits to improve the shelters and evacuation routes.

An on-line competition could also be launched to have volunteers use Google Earth and other web-based resources to identify areas of land that are elevated in case of flooding. These volunteers could also trace viable roads/paths that lead to and from these areas and mark places that may be vulnerable to landslides and other hazards.

What about the fact that only the government has the legal right to do this? Big deal. The system is not working so it’s time to disrupt it. Would you rather have a crowdsourced disaster preparedness plan now or a government certified plan after the hurricane season? I’m tempted to ask this during tomorrow’s BarCamp Haiti which I am co-organizing with the Haitian tech company Solutions and the Ushahidi Haiti Project.

Patrick Philippe Meier

22 Comments

Posted in Crisis Mapping, Crowdsourcing, Humanitarian Technologies

Tagged ,

Crisis Response and SMS Systems Management for NGOs and Governments

Posted on April 15, 2010 | 1 comment

Guest blog post: Bart Stidham is an enterprise architect committed to bringing positive change to the world via better information systems architecture. He has served as CTO of four companies including one of the largest communications companies in the world, been a senior executive at Accenture, and served as CIO of the largest NGO funded by USAID. He is an independent consultant and can be found in Washington, DC when he is not traveling.

This blog post builds off of and supports Patrick Meier’s previous post on developing an SMS Code of Conduct for Humanitarian Response. Patrick raises many important issues in his post and it is clear that with the success of Ushahidi-Haiti it is likely we will see a vast increase in the use of similar SMS based information management systems in the future. While the deployment of such systems and all communications systems is likely to be orderly and well structured in normal circumstances, it is likely that during crises such order may break down and these systems may negatively impact one another. For this reason I applaud Patrick’s effort to raise this issue but my hope is that the “normal order” imposed by governments and societies will help prevent the potential disruption of communications systems from occurring in disasters, emergencies, and crises.

I believe Patrick’s concerns are best discussed in the larger issue of frequency spectrum management. This is a huge issue and one that needs substantial education within the entire response space. It is a growing problem across each and every communications system not just in crises but also globally as we humans desire to communicate more in more ways and with more devices. There are limits to the amount of information that can be “pushed” through any communications system and those limits increasingly have to do with the laws of physics, not just the design of the systems.

The electromagnetic frequency spectrum (EF) is the basis of all wireless communication. We started our use of it the late 1800s with the first use of radio. Long ago we exhausted the entire spectrum and are now trying to find ways to reuse parts of it more efficiently. However it is critical that we protect this “public commons” on which so many of our communications systems depend.

Every communications system needs a “physical channel” and this varies widely but they all share some common characteristics. One is the problem of “collisions” which are bad because that means that the information is not delivered successfully. As humans using the physical channel of sound and speech we encounter this in our normal conversations whenever we meet in groups. A simple example of a collision is when two or more people are talking loudly over each other with the result being that no one understands what either is saying.

There are multiple ways to deal with collisions and every communications system must manage collisions or the system collapses. One way is to have a token and you are only allowed to talk if you hold the token. This method of managing communications was used brilliantly by various Native American tribes when discussing heated issues such as war – if you are not in possession of the peace pipe (the token) you are not allowed to speak. This forces everyone to listen to what you are saying and to politely take turns speaking. It is passed back and forth and everyone gets a turn. There are several types of network architecture that use this exact method for avoiding collisions.

Another method is collision avoidance by assigning each speaker a window of time to speak in. This is roughly the approach used by GSM for instance. Yet another method is collision detection where you allow for a certain statistical overlap and all parties know that the last “conversation” collided with another and the information was lost. The system then corrects the problem. This is not as efficient but is easy to do and cheap to implement. This is what Ethernet uses.

Finally as systems are deployed and interact with each other in a certain physical space they need to divide up the space. This can be done by frequency or cables or physical area or by time or all of the above.

In our discussion SMS are best likened to frequencies (although this is not an exact analogy). The advantage of them is that no two NGOs can ever end up with the same long code as this is handled by the carriers and their agreements. Internationally no two carriers can ever issue the same phone number or long code globally. If all NGOs stuck with long codes or full phone numbers we could avoid the problem Patrick is rightly concerned with.

NGOs and other organizations can problematically and mistakenly issue the same short code within a geographic area and we should all be concerned about this exactly as Patrick is. This problem can happen because short codes are for humans – not for the system itself. If the carriers are using different underlying cell phone technologies they can both issue the same short code and neither will interfere technically with the other one. Unfortunately it could have disastrous consequences for the socialization of the short codes to the local or larger population if they cross either technical or geographic lines. This is a problem largely unique to SMS and the plethora of technologies,carriers, bands (or frequencies) that can be deployed in a large physical area and the fact that short codes are for human convenience.

Right now there are 14 frequency bands just within the GSM voice system (thankfully quad band phones support all the widely used ones) and another 14 for data (furthermore the data “bands” actually cover a huge range of frequencies). This is why it is possible to have within one region, country or city with two “overlapping” short codes on two or more different carriers – the codes will each work only on the carrier that operates on that actual GSM frequency band. The system doesn’t care but it can be confusing to us humans.

Another thing Patrick has raised as a concern is actually “subject matter frequency” overlap (or collisions) and the confusion that can result to us simpleminded humans.

It makes no difference how many SMS codes are used as long as they are long codes OR if private and on short codes. The only time there is a problem is when two groups set up short codes that become public (meaning they are advertised in some way to the general public) as “the right number for X” where X is the same subject matter area.

In order to speed response many countries do NOT follow the US 911 system which uses a single short number for all emergencies. For instance Austria uses no less than 9 “short code” voice numbers each for a separate emergency type. That’s great if you live there and have them all memorized and know that for extreme sports there is a number just for “alpine rescue” to get your friend off some ledge that he crashed into in his para-glider. It speeds vital response and gets the right team dispatched in the least time. It does however require a massive amount of public education.

In the US the government decided to have a “one number fits all” system. This was in response to the fact that previously we had thousands of local numbers for each fire and police department, hospital and ambulance service. Without a local phone book it wasn’t possible to know who to call in an emergency. We designed the 911 system as a way to solve this problem and looped all three major responders into the one system. This was then deployed on a county by county basis across the US. There is no national 911 system. The system scales by dividing itself into small geographic sections.

SMS systems tend to be larger in size because SMS carriers are geographically larger that the old local phone POP (point of presence) that became the basis of the US 911 system. Another major concern for SMS design is the total carrying capacity of the carrier SMS system itself. SMS is NOT designed to be use for “one to many” messages. That was never part of the design and the system can be knocked out if the overall limits of the system are exceeded. At that point the SMS systems themselves collapse under the load and start failing and can cause a cascade failure of the entire carrier network in a region – this means that SMS can knock out voice. It does appear that such a failure occurred in Haiti to one of the local carriers that implemented an SMS emergency broadcast system in conjunction with an NGO so this is a real problem.

Getting back to Patrick’s identified concern – we should be worried when multiple SMS “subject channels” are socialized via the mass media and it confuses the public. In Haiti that didn’t happen because there was only one due largely to the work and efforts of the 4636 Haiti.Ushahidi community.

I believe in the future that is also unlikely to happen because I hope the mass media outlets will simply refuse to say “use any of the following SMS codes for health and these for x and y and z.” I think they won’t do this haphazardly. Doing so (meaning confusing the public) could endanger their (mass media) operating license from the host country.

Furthermore countries and cities are typically aware of this whole discussion and carefully control the distribution of short codes (but this does vary widely from region to region). The country that issues the carrier the license to operate the infrastructure is the ultimate authority for this and reserves the right to yank someone off the air or kick them out of the country for failure to follow the rules. Frequency spectrum must be managed for the greater public good or the classic “crisis of the commons” will result. This is the concern Patrick has brought to light.
One can not and should not assume that the rules, laws and policies we (individuals) are used to operating under in our home country apply elsewhere. The term “sovereign nation” means exactly that – they set their own laws concerning how things operate – including technology and communications systems. For instance WiFi is NOT WiFi everywhere and a WiFI router sold in Japan is illegal to operate in the US.

Some well meaning but largely uneducated NGOs deployed systems in Haiti that badly broke rules, laws, policies, etc and the Government of Haiti (and the US Government on behalf of the Haitian Government) was very polite to them. They stepped all over local businesses and disrupted them. Had this happened in the US the FCC would have issued huge fines to them – fines that likely would drive them out of business – and for good reason. They are exploiting the “public commons” for their own advantage. Whether they meant to or not is irrelevant just as ignorance of the law is no excuse.

In the past most responders to such emergencies were large NGOs with trained communications teams that knew they must coordinate their use of various communications platforms with each other or everyone would suffer. In this past this was easy and obvious because NGOs, governments and businesses made extensive use of UHF and VHF radios for communications. Because these systems were voice based it was obvious when you had a problem and when someone was on your assigned frequency. Furthermore you frequently had the opportunity to yell at them over that same communications system.

In the era of digital communications systems we no longer have the ability to yell at anyone and in fact both the designed legal and official user and the illegal user may be unaware that they are colliding and causing both systems to fail. This is a huge problem because it means that both parties have no way to know even know they are interfering with each other much less how or where to resolve the problem.

In conclusion, I applaud Patrick’s efforts as he has raised an important issue that all NGOs that respond to emergencies (both in the US and abroad) must to be aware of. Education is critical. Please tell your organization that they must contact and coordinate with the official frequency manager, typically the local government’s communications agency or ministry, prior to deploying any communications equipment. Failing to do so is typically illegal and can have grave consequences in emergencies, crises and disasters.

1 Comment

Posted in Crowdsourcing, Early Warning, Humanitarian Technologies

Tagged ,

Sentiment Analysis of Haiti Text Messages (Updated)

Posted on March 8, 2010 | 7 comments

The field of sentiment analysis is one that I’ve long been interested in. See my previous post on the use of sentiment analysis for early warning here. So when we began receiving thousands of text messages from Haiti, I decided to ask my colleagues at the EC’s Joint Research Center (JRC) whether they could run some of their sentiment analysis software on the incoming SMS’s.

The 4636 SMS initiative in Haiti was a collaboration between many organizations and was coordinated by Josh Nesbit of FrontlineSMS. The system allowed individuals in Haiti to text in their location and urgent needs. These would then be shared with some of the humanitarian actors on the ground and also mapped on the Ushahidi-Haiti platform, which was used by first responders such as the Marine Corps.

Here’s how the JRC in partnership with the University of Alicante carried out their analysis on the incoming SMS’s:

As many individual words are ambiguous (e.g. the word ‘help’ probably predominantly indicates a negative situation, but it may also be positive, as in “help has finally arrived”), they looked at the most frequent word groups, or word n-grams (sizes 2 to 5 words). Out of these, they identified about 100 n-grams that they felt are (high) negative or (high) positive. These were added to the sentiment analysis tool.

The graph below depicts the changing sentiment reflected in the SMS data between January 17th and February 5th.

Sentiment Analysis of Haiti SMS’s

There is, of course, no way to tell whether the incoming text messages reflect the general feeling of the population. It is also important to emphasize that the number of individuals sending in SMS’s increased during this time period. Still, it would be interesting to go through the sentiment analysis data and identify what may have contributed to the peaks and troughs of the above graph.

Incidentally, the lowest point on this graph is associated with the date of January 21. The data reveals that a major aftershock took place that day. There are subsequent reports of trauma, food/water shortages, casualties, need for medication, etc., which drive the sentiment analysis scores down.

—

Update 1: My colleague Ralf Steinberger and the Ushahidi-Haiti group is looking into the reasons behind the spike around January 30th. Ralf notes the following:

I checked the news a bit, using the calendar function in EMM NewsExplorer (http://emm.newsexplorer.eu/). I checked both the English and the French news for the day. One certainly positive news item accessible to Haitians on that day was that Haiti leaders pointed to progress. Another (French) positive news item is that the WFP (PAM) put in place a structured food aid system aiming at feeding up to 2 million people via women only. People were given food coupons (25kg of rice per family), starting Saturday 30.1.

Ralf also found that many of the original SMS’s received on that day had not been translated into English. So we’re looking into why that might have been. Hopefully we can get them translated retro-actively for the purposes of this analysis.

Update 2: Josef Steinberger from JRC has produced a revised sentiment analysis graph through to mid March.

—

This kind of sentiment analysis can be done in real-time. In future deployments where SMS becomes the principle source to communicate with disaster affected populations, using this kind of approach may  eventually provide an overall score for how the humanitarian community is doing.

Patrick Philippe Meier

7 Comments

Posted in Digital Activism, Early Warning

Tagged ,

Haiti and the Tyranny of Technology

Posted on March 2, 2010 | 10 comments

How quickly we forget what has come before us. Is it because our technologies are so new or different that recommendations in the disaster literature don’t apply to us? The technology community repeatedly emphasized the unprecedented nature of the response in Haiti, particularly with respect to communication with disaster effected populations. Was is it really a complete departure? To a large extent yes, but were there really no guidelines available?

The challenges that materialized in the response to Haiti included:

  • Raising of expectations
  • Lack of formal complaints mechanism
  • Absence of downward accountability
  • Coordination and clarity of messaging

Guidelines to address these challenges do exist. I’ll draw on two documents that are both 4 years old. The first is my colleague Imogen Wall’s study “The Right to Know: The Challenge of Public Information and Accountability in Aceh and Sri Lanka”. The second is the Final Report of the Global Symposium+5.

Here are some of the main points to take home from Imogen’s 60-page study:

  • Many organizations are still paying for mistakes made in communicating with communities in the early days of the tsunami recovery effort, resulting in what many call the ‘broken promises’ phenomenon. The inherent problems of managing expectations were exacerbated by a widespread use of translators and jargon and the extreme levels of trauma experienced by beneficiaries.
  • Confusion about policies, an inability to report misuse of aid, ignorance about where to turn for assistance, cynicism and anger stemming from broken promises about aid, and mismanaged expectations were all noted in the 2004 Tsunami. But while cases of actual broken promises undoubtedly occurred, the majority of perceived broken promises actually seem to have resulted from communications problems.
  • Managing communications with communities is key to successful community-driven development. Knowledge is power: without information, communities cannot participate, make choices, or ask questions. Good communication is also about trust and partnership, and is thus at the heart of successful community partnerships.
  • Putting communities at the center of disaster response requires that adequate provision be made for community access to information about projects, channels through which they can ask questions, and mechanisms by which they can register dissatisfaction or complaints. Such efforts both supply information and create space for dialogue between communities and aid agencies, a two way information flow tht is beneficial to both parties. No accountability and transparency system is complete without a strong complaints mechanism. Complaints mechanisms have an important role in conflict mitigation at a community level and in the prevention of violence.
  • Donors must require downward accountability and communications strategies in projects they fund and by exploring ways in which they can receive feedback on projects from beneficiaries.
  • Until information is properly shared with beneficiaries, they will never be equal partners. And until they are provided with a voice and the ability to judge a project’s viability, organizations will never be able to claim that they enabled survivors to rebuild and move on to a as bright a future as possible.
  • Primarily, beneficiaries want practical information that explains what aid is available, what assistance they can expect, how to apply for it, when aid will arrive, why what they have received might differ from their neighbor, and what to do if they are not satisfied. They are not interested in materials that simply promote a particular organization. Secondly, they are very interested in hearing how the aid effort is going, how money is being spent, what problems are being experienced elsewhere, and what solutions are being found.
  • Low-tech solutions are almost invariably better. A simple bulletin board can  do more to enhance transparency and accountability towards beneficiaries than any website. All IDP locations should be required to have a bulletin board, and aid organizations should be required to display basic project information and contact details.
  • Broadly speaking, the aim of public relations (PR) is to promote an organization; the aim of public information (PI) is to channel information to the relevant audiences. But most communications expertise within international aid organizations is geared toward public relations.
  • Aid organizations have a tendency to regard communication with beneficiaries as an optional extra rather than seeing information as a vital commodity and a humanitarian right, the key to empowerment, better relationships with beneficiaries and a more effective recovery effort. There has also been a failure to understand that information deprivation causes stress and exacerbates trauma.
  • Organizations should consider, where appropriate, incorporating some form of community-bsaed monitoring and evaluation systems into their projects.
  • Written community contracts between organizations and beneficiaries should be adopted wherever possible.
  • There will always be questions regarding how much information can or should be shared with beneficiaries. Communities should be the driver of how much information will be shared.
  • Always leave a contact name, number and address, ideally of the liaison person responsible for the community.
  • With text messaging, a simple, short message can be sent to a list of phone numbers simultaneously. And while it is easy to compile a list of phone numbers of key people in beneficiary communities, collating and managing wider lists of numbers and making them available to aid organizations could be undertaken by a body such as OCHA’s Humanitarian Information Centers.
  • SMS is a powerful medium that can be harnessed for otherwise very difficult tasks, such as providing information quickly that is available only at the last minute, such as times for aid deliveries or changes in a medical clinic’s arrival time in a certain area. The list of recipients can be easily tailored to include only those in a certain geographical area, enabling messages to be very precisely targetted.

The Final Report of the 2006 Symposium+5 event that I participated includes a review of lessons learned, best practices and recommendations on the topic of communicating with disaster affected communities. Here are the main points:

  • Programs designed to enhance two-way information-sharing and communication with affected populations are not mainstreamed into all phases of the humanitarian continuum or the UN cluster system. More needs to be done to financially support the establishment of these projects in the preparedness and early response phase.
  • Provide easily understandable information to affected communities to encourage and empower people to take action to build and strengthen their resilience. The information should be developed with affected populations, incorporate relevant traditional and indigenous knowledge and cultural heritage and be tailored to different target audiences through both media and non-media communication channels, taking into account cultural and social factors.
  • Provide funding and support to local media and journalistic organizations that have a role in providing information to affected populations in all phases, from preparedness, during response, and into recovery and reconstruction.

Yes, we have new integrated platforms that allow for 2-way communication with disaster affected populations in near real time. But do these new tools render the above lessons learned and recommendations obsolete? If not, then why did the technology community not draw on them to guide their work in Haiti?

Patrick Philippe Meier

10 Comments

Posted in Crisis Mapping, Humanitarian Technologies

Tagged , ,

Ushahidi & The Unprecedented Role of SMS in Disaster Response

Posted on February 20, 2010 | 19 comments

What if we could communicate with disaster affected communities in real-time just days after a major disaster like the quake in Haiti? That is exactly what happened thanks to a partnership between the Emergency Information Service (EIS), InSTEDD, Ushahidi, Haitian Telcos and the US State Department. Just 4 days after the earthquake, Haitians could text their location and urgent needs to “4636” for free.

I will focus primarily on the way that Ushahidi used 4636. Since the majority of incoming text messages were in Creole, we needed a translation service. My colleague Brian Herbert from Ushahidi and Robert Munro of Energy for Opportunity thus built a dedicated interface for crowdsourcing this step and reached out to dozens of Haitian communities groups to aid in the translation, categorization and geo-location of every message, quickly mobilizing 100s of motivated and dedicated volunteers. So not only was Ushahidi crowdsourcing crisis information in near real-time but also crowdsourcing translation in near real-time.

Text messages are translated into English just minutes after they leave a mobile phone in Haiti. The translated messages then appear directly on the Ushahidi platform. The screenshots below (click on graphics to enlarge) illustrates how the process works. The original SMS in Creole (or French) is displayed in the header. In order to view the translation, one simply clicks on “Read More”.

Ushahidi Back End

–

Incoming Text Messages

–

If further information is required, then one can reply to the sender of the text message directly from the Ushahidi platform. This is an important feature for several reasons. First, this allows for two-way communication with disaster affected communities. Second, an important number of messages we received were not actionable because of insufficient location information. The reply feature allowed us to get more precise information.

The screenshots below show how the “Send Reply” feature works. We weren’t sure if Universite Wayal was the same as Royal University. So we replied and asked for more location information. Note the preset replies in both English and Creole. The presets include thanks & requests for more location information, for example. Of course, one is not limited to these presets. Any text can be typed in and sent back to the sender of the original SMS. This feature has been part of the Ushahidi for almost two years now. We send off the request for more information and receive the following reply within minutes.

Preset Replies

When we receive an urgent and actionable SMS like this one, we can immediately create a report. By actionable, we mean there is sufficient location information and the description of the need is specific enough to respond to, just like the example above.

Creating a Report

First, the GPS coordinates for the location is identified. This can be done directly from the Ushahidi platform by entering the street address or town name. Sometimes a bit of detective work is needed to pinpoint the exact coordinates. Next, a title and description for the report is included–the latter usually comprising the text of the SMS. This is what we mean by structured information. The report is then tagged based on the category framework. Pictures can be uploaded with the report, and links to videos can also be included. Finally the report is saved and then approved for publication.

This is how the Ushahidi-Haiti @ Tufts team mapped 1,500+ text messages on the Ushahidi platform. We are now working with Samasource and Crowdflower to have the translation work serve as a source of income for Haitians inside Haiti. But how does all this connect to response?

Ushahidi’s “Get Alerts” feature is one of my favorite because it allows responders themselves to customize the specific type of actionable information that is important to them; i.e., demand driven situational awareness in near real-time. Not only can responders elect to receive automated alerts via email, but they can also do so via SMS. Responders can also specify their geographic area of interest.

Subscribe to Alerts

For example, if a relief worker from the Red Cross has a field office in neighborhood of Delmas, they can subscribe to Ushahidi to receive information on all reports originating from their immediate vicinity by specifying a radius, as shown below.

Selecting Area of Interest

The above Alerts feature is now being upgraded to the one depicted below, which was designed by my colleague Caleb Bell from Ushahidi. Not only are responders able to specify their geographic area of interest, but they can also select the type of alert (e.g., collapsed building, food shortage, looting, etc.) they want to receive. They can even add key words of interest to them, such as “water”, “violence” or “UN”. The goal is to provide responders with an unprecedented degree of customization to ensure they receive exactly the kind of alerts that they can respond to.

Highly Customized Alerts

On a more “macro” level, I recently reached out to colleagues at the EC’s Joint Research Center (JRC) to leverage their automated sentiment (“mood”) analysis platform. Sentiment Analysis is a branch of natural language processing (NLP) that seeks to quantify positive vs negative perceptions; akin to “tone” analysis. I suggested that we use their platform on the incoming text messages from Haiti to get a general sense of changing mood on an hourly basis. I’ll blog about the results shortly. In the meantime, here’s a previous blog post on the use of Sentiment Analysis for early warning.

Patrick Philippe Meier

19 Comments

Posted in Crisis Mapping, Crowdsourcing, Early Warning, Humanitarian Technologies

Tagged , ,

Location Based Mobile Alerts for Disaster Response in Haiti

Posted on February 6, 2010 | 5 comments

Using demand-side and supply-side economics as an analogy for the use of communication and information technology (ICT) in disaster response may yield some interesting insights. Demand-side economics (a.k.a. Keynesian economics) argues that government policies should seek to “increase aggregate demand, thus increasing economic activity and reducing unemployment.” Supply-side economics, in contrast, argues that “overall economic well-being is maximized by lowering the barriers to producing goods and services.”

I’d like to take this analogy and apply it to the subject of text messaging in Haiti. The 4636 SMS system was set up in Haiti by the Emergency Information Service or EIS (video) with InSTEDD (video), Ushahidi (video) and the US State Department. The system allows for both demand-side and supply-side disaster response. Anyone in the country can text 4636 with their location and needs, i.e., demand-side. The system is also being used to supply some mobile phone users with important information updates, i.e., supply-side.

Both communication features are revolutionizing disaster response. Lets take the supply-side approach first. EIS together with WFP, UNICEF, IOM, the Red Cross and others are using the system to send out SMS to all ~7,500 mobile phones (the number is increasing daily) with important information updates. Here are screen shots of the latest messages sent out from the EIS system:

The supply-side approach is possible thanks to the much lower (technical and financial) barriers to disseminating this information in near real-time. Providing some beneficiaries with this information can serve to reassure them that aid is on the way and to inform them where they can access various services thus maximizing overall economic well-being.

Ushahidi takes both a demand-side and supply-side approach by using the 4636 SMS system. 4636 is used to solicit text messages from individuals in urgent need. These SMS’s are then geo-tagged in near real-time on Ushahidi’s interactive map of Haiti. In addition, Ushahidi provides a feature for users to receive alerts about specific geographic locations. As the screen shot below depicts, users can specify the location and geographical radius they want to receive information on via automated email and/or SMS alerts; i.e., supply-side.

The Ushahidi Tech Team is currently working to allow users to subscribe to specific alert categories/indicators based on the categories/indicators already being used to map the disaster and humanitarian response in Haiti. See the Ushahidi Haiti Map for the list. This will enable subscribers to receive even more targeted location based mobile alerts,  thus further improving their situational awareness, which will enable them to take more informed decisions about their disaster response activities.

Both the demand- and supply-side approaches are important. They comprise an unprecedented ability to provide location-based mobile alerts for disaster response; something not dissimilar to location based mobile advertising, i.e., targeted communication based on personal preferences and location. The next step, therefore, is to make all supply-side text messages location based when necessary. For example, the following SMS broadcast would only go to mobile phone subscribers in Port-au-Prince:

It is important that both demand- and supply-side mobile alerts be location based when needed. Otherwise, we fall prey to Seeing Like a State.

“If we imagine a state that has no reliable means of enumerating and locating its population, gauging its wealth, and mapping its land, resources, and settlements, we are imagining a state whose interventions in that society are necessarily crude.”

In “Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed,” James Scott uses the following elegant analogy to emphasize the importance of locality.

“When a large freighter or passenger liner approaches a major port, the captain typically turns the control of his vessel over to a local pilot, who brings it into the harbor and to its berth. The same procedure is followed when the ship leaves its berth until it is safely out into the sea-lanes. This sensible procedure, designed to avoid accidents, reflects the fact that navigation on the open sea (a more “abstract” space) is the more general skill. While piloting a ship through traffic in a particular port is a highly contextual skill. We might call the art of piloting a “local and situated knowledge.”

An early lesson learned in the SMS deployment in Haiti is that more communication between the demand- and supply-side organizations need to happen. We are sharing the 4636 number,  so we are dependent on each other and need to ensure that changes to the system be up for open discussion. This lack of joint outreach has been the single most important challenge in my opinion. The captains are just not talking to the local pilots.

Patrick Philippe Meier

5 Comments

Posted in Crisis Mapping, Crowdsourcing, Humanitarian Technologies

Tagged , , , ,

Using Mechanical Turk to Crowdsource Humanitarian Response

Posted on February 5, 2010 | 17 comments

I’m increasingly intrigued by the idea of applying Mechanical Turk services to humanitarian response. Mechanical Turk was first developed by Amazon to crowdsource and pay for simple tasks.

An excellent example of a Mechanical Turk service in the field of ICT for Development (ICT4D) is txteagle, a platform that enables mobile phone subscribers in developing countries to earn money and accumulate savings by completing simple SMS-based micro-tasks for large corporate clients. txteagle has been used to translate pieces of text by splitting them into individual words and sending these out by SMS. Subscribers can then reply with the translation and earn some money in the process. This automatic compensation system uses statistical machinery to automatically evaluate the value of submitted work.

In Haiti, Samasource and Crowdflower have partnered with Ushahidi and FrontlineSMS to set up a Mechanical Turk service called “Mission 4636“. The system that Ushahidi and partners originally set up uses the generosity of Haitian volunteers in the US to translate urgent SMS’s from the disaster affected population in near real-time. Mission 4636 will relocate the translation work to Haiti and become an automatic compensation system for Haitian’s in-country.

At Ushahidi, we aggregate and  categorize urgent, actionable information from multiple sources including SMS and geo-tag this information on the Ushahidi’s interactive mapping platform. In the case of Haiti, this work is carried out by volunteers in Boston, Geneva, London and Portland coordinated by the Ushahidi-Haiti Situation Room at Tufts University. Volunteer retention is often a challenge, however. I wonder whether we an automated compensation system could be used to sustain future crisis mapping efforts.

Another challenge of crowdsourcing crisis information is tracking response. We know for a fact that a number of key responders are following our near real-time mapping efforts but knowing which reports they respond to is less than automatic. We have been able to document a number of success stories and continue to receive positive feedback from responders themselves but this information is hard to come by.

In a way, by crisis mapping actionable information in near real-time and in the public domain, we are in effect trying to crowdsource response. This, by nature, is a distributed and decentralized process, hence difficult to track. The tracking challenge is further magnified when the actors in question are relief and aid organizations responding to a large disaster. As anyone who has worked in disaster response knows, communicating who is doing what, where and when is not easy. Responders don’t have the bandwidth to document which reports they’ve responded to on Ushahidi.

This is problematic for several reasons including coordination. Organizations don’t necessarily know who is responding to what and whether this response is efficient. I wonder whether a Mechanical Turk system could be set up to crowdsource discrete response tasks based on individual organizations’ mandates. Sounds a little far out and may not be feasible, but the idea nevertheless intrigues me.

The automatic compensation system could be a public way to compensate response. Incoming SMS’s could be clustered along the UN Cluster system. The Shelter Cluster, for example, would have a dedicated website to which all shelter-related SMS’s would be pushed to. Organizations working in this space would each have access to this password protected website and tag the alerts they can and want to respond to.

In order to “cash in” following a response, a picture (or text based evidence) has to be submitted as proof, by the organization in question e.g., of new shelters being built. The number of completed responses could also be made public and individuals compelled to help, could send donations via SMS to each organization to reward and further fund the responses.

The task of evaluating the evidence of responses can also be crowdsource à la Mechanical Turk and serve as a source of revenue for beneficiaries.

For example, local Haitian subscribers to the system would receive an SMS notifying them that new shelters have been set up near Jacmel. Only subscribers in the Jacmel area would receive the SMS. They would then have a look for themselves to see whether the new shelters were in fact there and text back accordingly. Dozens of individuals could send in SMS’s to describe their observations which would further help triangulate the veracity of the evaluation à la Swift River. Note that the Diaspora could also get involved in this. And like txteagle, statistical machinery could also  be used to automatically evaluate the response and dispense the micro-compensations.

I have no doubt there are a number of other important kinks to be ironed out but I wanted to throw this out there now to get some preliminary feedback. This may ultimately not be feasible or worthwhile. But I do think that a partnership between Ushahidi and Crowdflower makes sense, not only in Haiti but for future deployments as well.

See also:

  • Digital Humanitarian Response: Moving from Crowdsourcing to Microtasking [Link]

17 Comments

Posted in Crisis Mapping, Crowdsourcing, Humanitarian Technologies

Tagged , , , ,

How To Royally Mess Up Disaster Response in Haiti

Posted on February 3, 2010 | 30 comments

I have to find an outlet other than this one to vent my frustrations at this time, which is why I deleted the 5 paragraphs that followed about 3 times. Not to worry, I saved them in a Word document. Good, now that I’ve got the venting part over with, lets play a crowdsourcing game.

I’d love to get your thoughts on the Top 10 ways to mess up disaster response in Haiti using information and communication technology. Suggestions can be completely made up, they can be jokes, serious commentary, witty remarks, predictions, actual observations, and so on, you get the idea. Feel free to post your comments below (anonymously if you wish), but no insults or accusations please, or else I’ll have to delete them.

I’ll keep this game open for 7 days and will post the best results on a new blog post. The person with the best comment will get a free invitation to the:

2nd International Conference on Crisis Mapping (ICCM 2010):
Haiti and Beyond

Patrick Philippe Meier

30 Comments

Posted in Humanitarian Technologies

Tagged

The Role of Live Skype Chats in the Disaster Response to Haiti

Posted on January 31, 2010 | 12 comments

Live Skype chats played an invaluable role in the disaster response to Haiti but this has gone largely unnoticed by both mainstream and citizen media. I have a Word document with over 2,000 pages worth of Skype chat messages exchanged  in various groups during the first 2.5 weeks after the earthquake. I have no doubt that this data will become a source of major interest for scholars seeking to evaluate the disaster response in Haiti.

The Skype chats reveal a minute-by-minute account of the actions and decisions that organizations like Ushahidi, FrontlineSMS, InSTEDD, Sahana, Google, Thomson-Reuters and others took following the earthquake. Search and Rescue (SAR) teams in Port-au-Prince also participated in these Skype chats:

For the full story behind the above exchange between Anna, Eric and myself, please see my previous blog post. In addition to SAR staff, the US State Department,  a White House liaison contact, SOUTHCOM, DAI, UN/OCHA, WFP, the US Coast Guard, a Telecom company, and so on were all on live Skype chats at one point or another. It’s actually hard to keep track of everyone who has used the various Skype chats since the earthquake.

The most active and critical Skype Chat Groups were/are:

  • Haiti Tech Ushahidi Situation Room (72 users)
  • GPS Conversations for the SAR Dispatch (21 users)
  • SMS Logistics (37)
  • Ushahidi + US Coast Guard + SOUTHCOM (11 users)
  • Urgent Response Group (13 users)
  • Ushahidi Volunteer Task Force (168)

I would really like to see a discourse analysis and social network analysis of this data. Not to mention different visualizations of the data. In fact, I’d love to partner with anyone who has the time and expertise in these areas to do this. For now, lets take the first Skype chat group above, which was the most critical group during the first week, and just focus on the growth of this group in terms of users during the first week. And then lets create some Wordl visualizations based on data in this chat group.

I started the Haiti Tech Ushahidi Situation Room a couple hours after David Kobia and I launched the Ushahidi-Haiti platform. The second person I called (on my cell) after David was Chris Blow from Meedan. Chris got started on the icons for the platform right away. In the meantime, we used color-coded dots to represent the different categories/indicators.

I checked in with Chris on Skype a couple hours later. Below is the progression of users added to the Skype chat during the first week in case anyone wants to start on some simple social network analysis:

[1/12/10 9:09:55 PM] Patrick Meier: hey Chris, you there?

[1/12/10 10:00:27 PM] Patrick Meier added Brian Herbert to this chat

[1/12/10 10:02:30 PM] Patrick Meier added David Kobia to this chat

[1/12/10 10:10:39 PM] Patrick Meier added Jeffrey Villaveces to this chat

[1/12/10 10:47:41 PM] Jeffrey Villaveces added Luishernando to this chat

[1/12/10 11:42:01 PM] Jeffrey Villaveces added Gabriel Dicelis to this chat

[1/12/10 11:48:49 PM] Brian Herbert added Ory Okolloh to this chat

[1/13/10 2:00:03 AM] Patrick Meier added Kennedy Kasina to this chat

[1/13/10 2:00:11 AM] Patrick Meier: just added Ken to this chat

[1/13/10 2:00:43 AM] Ory Okolloh added Henry Addo to this chat

[1/13/10 2:02:54 AM] Brian Herbert added Henry Addo to this chat

[1/13/10 3:30:13 AM] Patrick Meier added Kaushal Jhalla to this chat

[1/13/10 8:46:07 AM] Patrick Meier added Claire U to this chat

[1/13/10 10:06:59 AM] Brian Herbert added Pablo Destefanis to this chat

[1/13/10 10:09:41 AM] Brian Herbert added Oscar Salazar to this chat

[1/13/10 10:22:02 AM] Patrick Meier added Emily Jacobi to this chat

[1/13/10 10:47:32 AM] Oscar Salazar added Nicolas et Alice BIais- Bonhomme to this chat

[1/13/10 10:51:59 AM] Patrick Meier added Rob Baker to this chat

[1/13/10 11:22:54 AM] Emily Jacobi added Mark Belinsky to this chat

[1/13/10 12:05:59 PM] Patrick Meier added Josh Marcus to this chat

[1/13/10 12:08:10 PM] Patrick Meier added Shoreh Elhami to this chat

[1/13/10 12:11:54 PM] Jeffrey Villaveces added Luke Beckman to this chat

[1/13/10 12:18:52 PM] Luke Beckman added Eduardo Jezierski, Eric Rasmussen to this chat

[1/13/10 12:38:09 PM] Brian Herbert added Erik Hersman to this chat

[1/13/10 1:29:05 PM] Luke Beckman added Brian Steckler to this chat

[1/13/10 1:41:03 PM] Erik Hersman added Caleb Bell to this chat

[1/13/10 2:32:19 PM] Erik Hersman added Jason Mule to this chat

[1/13/10 2:40:31 PM] Luke Beckman added Josh Nesbit to this chat

[1/13/10 5:32:31 PM] Claire U added Fabienne to this chat

[1/13/10 6:13:48 PM] Eduardo Jezierski added Mark Prutsalis to this chat

[1/13/10 7:28:34 PM] David Kobia added Andrew Turner to this chat

[1/13/10 10:59:49 PM] Josh Marcus added Tim Schwartz to this chat

[1/14/10 12:25:58 AM] Tim Schwartz added Ryan Brown to this chat

[1/14/10 4:00:41 AM] Erik Hersman added Meryn Stol to this chat

[1/14/10 4:25:46 AM] Erik Hersman added Victor Miclovich to this chat

[1/14/10 4:29:46 AM] Kennedy Kasina added Charles Kithika to this chat

[1/14/10 5:02:55 AM] Erik Hersman added Brian Joel Conley to this chat

[1/14/10 5:14:35 AM] Kennedy Kasina added lisudza to this chat

[1/14/10 5:21:03 AM] Erik Hersman added aliveinbaghdad to this chat

[1/14/10 1:26:56 PM] Erik Hersman added Dale Zak to this chat

[1/14/10 1:43:43 PM] Dale Zak added benrigby to this chat

[1/14/10 1:51:00 PM] benrigby added Boris Korsunsky to this chat

[1/14/10 2:00:21 PM] Brian Herbert added Abdallah Chamas to this chat

[1/14/10 3:09:37 PM] Josh Nesbit added Paul Goodman to this chat

[1/14/10 3:24:11 PM] Brian Herbert added Satchit Balsari to this chat

[1/14/10 3:37:02 PM] Satchit Balsari added ritwikdey to this chat

[1/14/10 3:50:38 PM] Satchit Balsari added Selvam Velmurugan to this chat

[1/14/10 4:12:47 PM] Josh Marcus added Sharda Sekaran to this chat

[1/14/10 5:53:19 PM] Ory Okolloh added Jonathan Greenblatt to this chat

[1/14/10 7:30:37 PM] Tim Schwartz added wendell_iii to this chat

[1/14/10 10:02:25 PM] Tim Schwartz added Christopher Csikszentmihalyi to this chat

[1/14/10 11:43:09 PM] Josh Nesbit added Robert Munro to this chat

[1/15/10 4:59:07 AM] Brian Steckler added Ryan Burke to this chat

[1/15/10 5:03:45 AM] Kennedy Kasina added joanwmaina to this chat

[1/15/10 5:39:11 AM] David Kobia added Cooper Quintin to this chat

[1/15/10 10:38:10 AM] mark.prutsalis added Chamindra de Silva to chat

[1/15/10 11:24:43 AM] Sharda Sekaran added Amir Reavis-Bey to this chat

[1/15/10 11:27:07 AM] Josh Nesbit added David Wade to this chat

[1/15/10 3:30:51 PM] Paul Goodman added Tapan Parikh to this chat

[1/15/10 7:02:37 PM] Mark Belinsky added Philip Ashlock to this chat

[1/15/10 8:02:39 PM] Brian Steckler added Michael D. McDonald to chat

[1/15/10 10:15:38 PM] mark.prutsalis added David Bitner to this chat

[1/16/10 12:26:11 PM] mark.prutsalis added lifeeth to this chat

[1/16/10 9:23:24 PM] Rob Baker added Rachel Weidinger to this chat

[1/17/10 11:36:20 AM] Josh Nesbit added Lisa Lamanna to this chat

[1/18/10 3:54:06 PM] Luke Beckman added doshi.sd to this chat

[1/18/10 4:18:09 PM] Tim Schwartz added Christina Xu to this chat

[1/19/10 12:10:19 PM] Jeffrey Villaveces added Amaury to this chat

[1/19/10 6:05:19 PM] Ryan Burke added Randy Maule to this chat

[1/19/10 10:24:21 PM] Tapan Parikh added david.notkin to this chat

Here’s the Wordl rendition of the above text:

Below is the Wordl visualization of all the data in the Haiti Chat group, i.e., not only users being added but also the full content of all the chats between 9pm on January 12th through 9pm on January 30th.  This constitutes over 300 pages of content in a Word document. Of course, dates and individual names still come up most frequently.

The Wordl visualization below draws on the first week of data but with all names, dates and times removed. This enables us to focus exclusively on the content or dialogue exchanged between users.

I like the fact that the word “thanks” stands out fairly prominently. Stay tuned for more Wordl visualizations on the other Skype chat groups. In the meantime, if you want to get started on some more statistical discourse analysis or social network analysis, please feel free to get in touch. Thanks!

Patrick Philippe Meier

12 Comments

Posted in Crisis Mapping, Humanitarian Technologies

Tagged ,