Category Archives: Humanitarian Technologies

Innovation and the State of the Humanitarian System

Posted on August 29, 2012 | 3 comments

Published by ALNAP, the 2012 State of the Humanitarian System report is an important evaluation of the humanitarian community’s efforts over the past two years. “I commend this report to all those responsible for planning and delivering life saving aid around the world,” writes UN Under-Secretary General Valerie Amos in the Preface. “If we are going to improve international humanitarian response we all need to pay attention to the areas of action highlighted in the report.” Below are some of the highlighted areas from the 100+ page evaluation that are ripe for innovative interventions.

Accessing Those in Need

Operational access to populations in need has not improved. Access problems continue and are primarily political or security-related rather than logistical. Indeed, “UN security restrictions often place sever limits on the range of UN-led assessments,” which means that “coverage often can be compromised.” This means that “access constraints in some contexts continue to inhibit an accurate assessment of need. Up to 60% of South Sudan is inaccessible for parts of the year. As a result, critical data, including mortality and morbidity, remain unavailable. Data on nutrition, for example, exist in only 25 of 79 countries where humanitarian partners have conducted surveys.”

Could satellite and/or areal imagery be used to measure indirect proxies? This would certainly be rather imperfect but perhaps better than nothing? Could crowdseeding be used?

Information and Communication Technologies

“The use of mobile devices and networks is becoming increasingly important, both to deliver cash and for communication with aid recipients.” Some humanitarian organizations are also “experimenting with different types of communication tools, for different uses and in different contexts. Examples include: offering emergency information, collecting information for needs assessments or for monitoring and evaluation, surveying individuals, or obtaining information on remote populations from an appointed individual at the community level.”

“Across a variety of interventions, mobile phone technology is seen as having great potential to increase efficiency. For example, […] the governments of Japan and Thailand used SMS and Twitter to spread messages about the disaster response.” Naturally, in some contexts, “traditional means like radios and call centers are most appropriate.”

In any case, “thanks to new technologies and initiatives to advance commu-nications with affected populations, the voices of aid recipients began, in a small way, to be heard.” Obviously, heard and understood are not the same thing–not to mention heard, understood and responded to. Moreover, as disaster affected communities become increasingly “digital” thanks to the spread of mobile phones, the number of voices will increase significantly. The humanitarian system is largely (if not completely) unprepared to handle this increase in volume (Big Data).

Consulting Local Recipients

Humanitarian organizations have “failed to consult with recipients […] or to use their input in programming.” Indeed, disaster-affected communities are “rarely given opportunities to assess the impact of interventions and to comment on performance.” In fact, “they are rarely treated as end-users of the service.” Aid recipients also report that “the aid they received did not address their ‘most important needs at the time.'” While some field-level accountability mechanisms do exist, they were typically duplicative and very project oriented. To this end, “it might be more efficient and effective to have more coordination between agencies regarding accountability approaches.”

While the ALNAP report suggests that these shortcomings could “be addressed in the near future by technical advances in methods of needs assessment,” the challenge here is not simply a technical one. Still, there are important efforts underway to address these issues.

Improving Needs Assessments

The Inter-Agency Standing Committee’s (IASC) Needs Assessment Task Force (NAFT) and the International NGO-led Assessment Capacities Project (ACAPS) are two such exempts of progress. OCHA serves as the secretariat for the NAFT through its Assessment and Classification of Emergencies (ACE) Team. ACAPS, which is a consortium of three international NGOs (X, Y and Z) and a member of NATF, aims to “strengthen the capacity of the humanitarian sector in multi-sectoral needs assessment.” ACAPS is considered to have “brought sound technical processes and practical guidelines to common needs assessment.” Note that both ACAPS and ACE have recently reached out to the Digital Humanitarian Network (DHNetwork) to partner on needs-assessment projects in South Sudan and the DRC.

Another promising project is the Humanitarian Emergency Settings Perceived Needs Scale (HESPER). This join initiative between WHO and King’s College London is designed to rapidly assess the “perceived needs of affected populations and allow their views to be taken into consideration. The project specifically aims to fill the gap between population-based ‘objective’ indicators […] and/or qualitative data based on convenience samples such as focus groups or key informant interviews.” On this note, some NGOs argue that “overall assessment methodologies should focus far more at the community (not individual) level, including an assessment of local capacities […],” since “far too often international aid actors assume there is no local capacity.”

Early Warning and Response

An evaluation of the response in the Horn of Africa found “significant disconnects between early warning systems and response, and between technical assessments and decision-makers.” According to ALNAP, “most commentators agree that the early warning worked, but there was a failure to act on it.” This disconnect is a concern I voiced back in 2009 when UN Global Pulse was first launched. To be sure, real-time information does not turn an organization into a real-time organization. Not surprisingly, most of the aid recipients surveyed for the ALNAP report felt that “the foremost way in which humanitarian organizations could improve would be to: ‘be faster to start delivering aid.'” Interestingly, “this stands in contrast to the survey responses of international aid practitioners who gave fairly high marks to themselves for timeliness […].”

Rapid and Skilled Humanitarians

While the humanitarian system’s surge capacity for the deployment of humanitarian personnel has improved, “findings also suggest that the adequate scale-up of appropriately skilled […] staff is still perceived as problematic for both operations and coordination.” Other evaluations “consistently show that staff in NGOs, UN agencies and clusters were perceived to be ill prepared in terms of basic language and context training in a significant number of contexts.” In addition, failures in knowledge and understanding of humanitarian principles were also raised. Furthermore, evaluations of mega-disasters “predictably note influxes or relatively new staff with limited experience.” Several evaluations noted that the lack of “contextual knowledge caused a net decrease in impact.” This lend one senior manager noted:

“If you don’t understand the political, ethnic, tribal contexts it is difficult to be effective… If I had my way I’d first recruit 20 anthropologists and political scientists to help us work out what’s going on in these settings.”

Monitoring and Evaluation

ALNAP found that monitoring and evaluation continues to be a significant shortcoming in the humanitarian system. “Evaluations have made mixed progress, but affected states are still notably absent from evaluating their own response or participating in joint evaluations with counterparts.” Moreover, while there have been important efforts by CDAC and others to “improve accountability to, and communication with, aid recipients,” there is “less evidence to suggest that this new resource of ground-level information is being used strategically to improve humanitarian interventions.” To this end, “relatively few evaluations focus on the views of aid recipients […].” In one case, “although a system was in place with results-based indicators, there was neither the time nor resources to analyze or use the data.”

The most common reasons cited for failing to meet community expectations include the “inability to meet the full spectrum of need, weak understanding of local context, inability to understand the changing nature of need, inadequate information-gathering techniques or an inflexible response approach.” In addition, preconceived notions of vulnerability have “led to inappropriate interventions.” A major study carried out by Tufts University and cited in the ALNAP report concludes that “humanitarian assistance remains driven by ‘anecdote rather than evidence’ […].” One important exception to this is the Danish Refugee Council’s work in Somalia.

Leadership, Risk and Principles

ALNAP identifies an “alarming evidence of a growing tendency towards risk aversion” and a “stifling culture of compliance.” In addition, adherence to humanitarian principles were found to have weakened as “many humanitarian organizations have willingly compromised a principled approach in their own conduct through close alignment with political and military activities and actors.” Moreover, “responses in highly politicized contexts are viewed as particularly problematic for the retention of humanitarian principles.” Humanitarian professionals who were interviewed by ALNAP for this report “highlighted multiple occasions when agencies failed to maintain an impartial response when under pressure from strong states, such as Pakistan and Sri Lanka.”

3 Comments

Posted in Big Data, Humanitarian Technologies

Tagged , , , ,

How People in Emergencies Use Communication to Survive

Posted on August 28, 2012 | 2 comments

“Still Left in the Dark? How People in Emergencies Use Communication to Survive — And How Humanitarian Agencies Can Help” is an excellent report pub-lished by the BBC World Service Trust earlier this year. It is a follow up to the BBC’s 2008 study “Left in the Dark: The Unmet Need for Information in Humanitarian Emergencies.” Both reports are absolute must-reads. I highlight the most important points from the 2012 publication below.

Are Humanitarians Being Left in the Dark?

The disruptive impact of new information and communication technologies (ICTs) is hardly a surprise. Back in 2007, researchers studying the use of social media during “forest fires in California concluded that ‘these emergent uses of social media are pre-cursors of broader future changes to the institutional and organizational arrangements of disaster response.'” While the main danger in 2008 was that disaster-affected communities would continue to be left in the dark since humanitarian organizations were not prioritizing information delivery, in 2012, “it may now be the humanitarian agencies themselves […] who risk being left in the dark.” Why? “Growing access to new technologies make it more likely that those affected by disaster will be better placed to access information and communicate their own needs.” Question is: “are humanitarian agencies prepared to respond to, help and engage with those who are communicating with them and who demand better information?” Indeed, “one of the consequences of greater access to, and the spread of, communications technology is that communities now expect—and demand—interaction.”

Monitoring Rumors While Focusing on Interaction and Listening

The BBC Report invites humanitarian organizations to focus on meaningful interaction with disaster-affected communities, rather than simply on message delivery. “Where agencies do address the question of communication with affected communities, this still tends to be seen as a question of relaying infor-mation (often described as ‘messaging’) to an unspecified ‘audience’ through a channel selected as appropriate (usually local radio). It is to be delivered when the agency thinks that it has something to say, rather than in response to demand. In an environment in which […] interaction is increasingly expected, this approach is becoming more and more out of touch with community needs. It also represents a fundamental misunderstanding of the nature and potential of many technological tools particularly Twitter, which work on a real time many-to-many information model rather than a simple broadcast.”

Two-way communication with disaster-affected communities requires two-way listening. Without listening, there can be no meaningful communication. “Listening benefits agencies, as well as those with whom they communicate. Any agency that does not monitor local media—including social media—for misinformation or rumors about their work or about important issues, such as cholera awareness risks, could be caught out by the speed at which information can move.” This is an incredibly important point. Alas, humanitarian organ-izations have not caught up with recent advances in social computing and big data analytics. This is one of the main reasons I joined the Qatar Computing Research Institute (QCRI); i.e., to spearhead the development of next-generation humani-tarian technology solutions.

Combining SMS with Geofencing for Emergency Alerts

Meanwhile, in Haiti, “phone company Digicel responded to the 2010 cholera outbreak by developing methods that would send an SMS to anyone who travelled through an identified cholera hotspot, alerting them to the dangers and advising on basic precautions.” The later is an excellent example of geofencing in action. That said, “while responders tend to see communication as a process either of delivering information (‘messaging’) or extracting it, disaster survivors seem to see the ability to communicate and the process of communication itself as every bit as important as the information delivered.”

Communication & Community-Based Disaster Response Efforts

As the BBC Report notes, “there is also growing evidence that communities in emergencies are adept at leveraging communications technology to organize their own responses.” This is indeed true as these recent examples demonstrate:

“Communications technology is empowering first responders in new and extremely potent ways that are, at present, little understood by international humanitarians. While aid agencies hesitate, local communities are using commu-nications technology to reshape the way they prepare for and respond to emergencies.” There is a definite payoff to those agencies that employ an “integrated approach to communicating and engaging with disaster affected communities […]” since they are “viewed more positively by beneficiaries than those that [do] not.” Indeed, “when disaster survivors are able to communicate with aid agencies their perceptions become more positive.”

Using New Technologies to Manage Local Feedback Mechanisms

So why don’t more agencies follow suite? Many are concerned that establishing feedback systems will prove impossible to manage let alone sustain. They fear that “they would not be able to answer questions asked, that they [would] not have the skills or capacity to manage the anticipated volume of inputs and that they [would be] unequipped to deal with people who would (it is assumed) be both angry and critical.”

I wonder whether these aid agencies realize that many private sector companies have feedback systems that engage millions of customers everyday; that these companies are using social media and big data analytics to make this happen. Some are even crowdsourcing their customer service support. It is high time that the humanitarian community realize that the challenges they face aren’t that unique and that solutions have already been developed in other sectors.

There are only a handful of examples of positive deviance vis-a-vis the setting up of feedback systems in the humanitarian space. Oxfam found that simply com-bining the “automatic management of SMS systems” with “just one dedicated local staff member […] was enough to cope with demand.” When the Danish Refugee Council set up their own SMS complaints mechanism, they too expected be overwhelmed with criticisms. “To their surprise, more than half of the SMS’s they received via their feedback system […] have been positive, with people thanking the agency for their assistance […].” This appears to be a pattern since “many other agencies reported receiving fewer ‘difficult’ questions than anticipated.”

Naturally, “a systematic and resourced approach for feedback” is needed either way. Interestingly, “many aid agencies are in fact now running de facto feedback and information line systems without realizing it. […] most staff who work directly with disaster survivors will be asked for contact details by those they interact with, and will give their own personal mobile numbers.” These ad hoc “systems” are hardly efficient, well-resourced or systematic, however.

User-Generated Content, Representativeness and Ecosystems

Obviously, user-generated content shared via social media may not be represen-tative. “But, as costs fall and coverage increases, all the signs are that usage will increase rapidly in rural areas and among poorer people. […] As one Somali NGO staff member commented […], ‘they may not have had lunch — but they’ll have a mobile phone.'” Moreover, there is growing evidence that individuals turn to social media platforms for the first time as a result of crisis. “In Thailand, for example, the use of social media increased 20% when the 2010 floods began–with fairly equal increases found in metropolitan Bangkok and in rural provinces.”

While the vast majority of Haitians in Port-au-Prince are not on Twitter, “the city’s journalists overwhelmingly are and and see it as an essential source of news and updates.” Since most Haitians listen to radio, “they are, in fact, the indirect beneficiaries of Twitter information systems.” Another interesting fact: “In Kenya, 27% of radio listeners tune in via their mobile phones.” This highlights the importance of an ecosystem approach when communicating with disaster-affected communities. On a related note, recent statistics reveal that individuals in developing countries spend about 17.5% of their income on ICTs compared to just 1.5% in developing countries.

2 Comments

Posted in Big Data, Crowdsourcing, Humanitarian Technologies, Social Computing, Social Media

Tagged , , , , , , , ,

Crowdsourcing Community-Based Disaster Relief in Indonesia

Posted on August 21, 2012 | 6 comments

I just came across a very neat example of crowdsourced, community-based crisis response in this excellent report by the BBC World Service Trust: “Still Left in the Dark? How People in Emergencies Use Communication to Survive—And How Humanitarian Agencies Can Help.” I plan to provide a detailed summary of this important report in a forthcoming blog post. In the meantime, this very neat example below (taken directly from said BBC report) is well worth sharing.

“In Indonesia during the eruption of Mount Merapi in November 2010, a local radio community known as Jalin Merapi began to share information via Twitter and used the network to organize community-based relief to over 700 shelters on the side of the mountain […].”

“The Jalin Merapi network was founded following an eruption of the Mount Merapi volcano on Java, Indonesia in 2006. Three community radio stations who felt that the reporting of the eruption by the mainstream media had been inaccurate and unhelpful to those affected joined up with a group of local NGOs and other radio networks to produce accurate information on volcanic activity for those living on the mountain’s slopes. By the time of the 2010 eruption the network involved 800 volunteers, a presence online, on Twitter and on Face-book, and a hotline.”

“During the first eruption on 26 October 2010, the team found that their online accounts–especially Twitter–had become extremely busy. Ten volunteers were assigned to manage the information flow: sorting incoming information (they agreed 27 hashtags to share information), cross referencing it and checking for veracity. For example, when one report came in about a need for food for 6,000 internally displaced people, the team checked the report for veracity then redistributed it as a request for help, a request re-tweeted by followers of the Jalin Merapi account. Within 30 minutes, the same volunteer called and said that enough food had now been supplied, and asked people to stop sending food – a message that was distributed by the team immediately.”

“Interestingly, two researchers who analyzed information systems during the Merapi eruption found that many people believed traditional channels such as television to be ‘less satisfying’. In many cases they felt that television did not provide proper information at the time, but created panic instead.” […] “The success of initiatives such as the Jalin Merapi is based on the levels of trust, community interaction and person-to-person relationships on which participants can build. While technology facilitated and amplified these, it did not replace them.” […] “The work of Jalin Merapi continues today, using the time between eruptions to raise awareness of dangers and help communities plan for the next incident.”

 

6 Comments

Posted in Crowdsourcing, Humanitarian Technologies, Social Media

Tagged , , , , , , ,

Crisis Mapping for Disaster Preparedness, Mitigation and Resilience

Posted on August 13, 2012 | 8 comments

Crisis mapping for disaster preparedness is nothing new. In 2004, my colleague Suha Ulgen spearheaded an innovative project in Istanbul that combined public participation and mobile geospatial technologies for the purposes of disaster mitigation. Suha subsequently published an excellent overview of the project entitled “Public Participation Geographic Information Sharing Systems for Co-mmunity Based Urban Disaster Mitigation,” available in this edited book on Geo-Information for Disaster Management. I have referred to this project in count-less conversations since 2007  so it is high time I blog about it as well.

Suha’s project included a novel “Neighborhood Geographic Information Sharing System,” which “provided volunteers with skills and tools for identification of seismic risks and response assets in their neighborhoods. Field data collection volunteers used low-cost hand-held computers and data compiled was fed into a geospatial database accessible over the Internet. Interactive thematic maps enabled discussion of mitigation measures and action alternatives. This pilot evolved into a proposal for sustained implementation with local fire stations.” Below is a screenshot of the web-based system that enabled data entry and query.

There’s no reason why a similar approach could not be taken today, one that uses a dedicated smart phone app combined with integrated gamification and social networking features. The idea would be to make community mapping fun and rewarding; a way to foster a more active and connected community—which would in turn build more social capital. In the event of a disaster, this same smart phone app would allow users to simply “check in” to receive information on the nearest shelter areas (response assets) as well as danger zones such as overpasses, etc. This is why geo-fencing is so important for crisis mapping.

(Incidentally, Suha’s project also included a “School Commute Contingency Pilot” designed to track school-bus routes in Istanbul and thus “stimulate contingency planning for commute-time emergencies when 400,000 students travel an average of 45 minutes each way on 20,000 service buses. [GPS] data loggers were used to determine service bus routes displayed on printed maps high-lighting nearest schools along the route.” Suha proposed that “bus-drivers, parents and school managers be issued route maps with nearest schools that could serve as both meeting places and shelters”).

Fast forward to 2012 and the Humanitarian OpenStreetMap’s (HOT) novel project “Community Mapping for Exposure in Indonesia,” which resulted in the mapping of over 160,000 buildings and numerous village level maps in under ten months. The team also organized a university competition to create incentives for the mapping of urban areas. “The students were not only tasked to digitize buildings, but to also collect building information such building structure, wall type, roof type and the number of floors.” This contributed to the mapping and codification of some 30,000 buildings.

As Suha rightly noted almost 10 years ago, “for disaster mitigation measures to be effective they need to be developed in recognition of the local differences and adopted by the active participation of each community.” OSM’s work in Indonesia fully embodies the importance of mapping local differences and provides important insights on how to catalyze community participation. The buildup of social capital is another important outcome of these efforts. Social capital facilitates collective action and increases local capacity for self-organization, resulting in greater social resilience. In sum, these novel projects demonstrate that technologies used for crisis mapping can be used for disaster preparedness, mitigation and resilience.

8 Comments

Posted in Crisis Mapping, Crowdsourcing, Disaster Resilience, Humanitarian Technologies

Tagged , , , , , ,

CrisisTracker: Collaborative Social Media Analysis For Disaster Response

Posted on July 30, 2012 | 8 comments

I just had the pleasure of speaking with my new colleague Jakob Rogstadius from Madeira Interactive Technologies Institute (Madeira-TTI). Jakob is working on CrisisTracker, a very interesting platform designed to facilitate collaborative social media analysis for disaster response. The rationale for CrisisTracker is the same one behind Ushahidi’s SwiftRiver project and could be hugely helpful for crisis mapping projects carried out by the Standby Volunteer Task Force (SBTF).

From the CrisisTracker website:

“During large-scale complex crises such as the Haiti earthquake, the Indian Ocean tsunami and the Arab Spring, social media has emerged as a source of timely and detailed reports regarding important events. However, indivi-dual disaster responders, government officials or citizens who wish to access this vast knowledge base are met with a torrent of information that quickly results in information overload. Without a way to organize and navigate the reports, important details are easily overlooked and it is challenging to use the data to get an overview of the situation as a whole.”

We (Madeira University, University of Oulu and IBM Research) believe that volunteers around the world would be willing to assist hard-pressed decision makers with information management, if the tools were available. With this vision in mind, we have developed Crisis-Tracker.”

Like SwiftRiver, CrisisTracker combines some automated clustering of content with the crowdsourced curation of said content for further filtering. “Any user of the system can directly contribute tags that make it easier for other users to retrieve information and explore stories by similarity. In addition, users of the system can influence how tweets are grouped into stories.” Stories can be filtered by Report Category, Keywords, Named Entities, Time and Location. CrisisTracker also allows for simple geo-fencing to capture and list only those Tweets displayed on a given map.

Geolocation, Report Categories and Named Entities are all generated manually. The clustering of reports into stories is done automatically using keyword frequencies. So if keyword dictionaries exist for other languages, the platform could be used in these other languages as well. The result is a list of clustered Tweets displayed below the map, with the most popular cluster at the top.

Clicking on an entry like the row in red above opens up a new page, like the one below. This page lists a group of tweets that all discuss the same specific event, in this case an explosion in Syria’s capital.

What is particularly helpful about this setup is the meta-data displayed for this story or event: the number of people who tweeted about the story, the number of tweets about the story, the first day/time the story was shared on twitter. In addition, the first tweet to report the story is listed along, which is very helpful. This list can be ranked according to “Size” which is a figure that reflects the minimum number of original tweets and the number of Twitter users who shared these tweets. This is a particularly useful metric (and way to deal with spammers). Users also have the option of listing the first 50 tweets that referenced the story.

As you may be able to tell from the “Hide Story” and “Remove” buttons on the righthand-side of the display above, each clustered story and indeed tweet can be hidden or removed if not relevant. This is where crowdsourced curation comes in. In addition, CrisisTracker enable users to geo-tag and categorize each tweets according to report type (e.g., Violence, Deaths, Request/Need, etc.), general keywords (e.g., #assad, #blasts, etc.) and named entities. Note the the keywords can be removed and more high-quality tags can be added or crowdsourced by users as well (see below).

CrisisTracker also suggests related stories that may be of interest to the user based on the initial clustering and filtering—assisted manual clustering. In addition, the platform’s API means that the data can then be exported in XML using a simple parser. So interoperability with platforms like Ushahidi’s would be possible. After our call, Jakob added a link on each story page in the system (a small XML icon below the related stories) to get the story in XML format. Any other system can now take this URL and parse the story into its own native format. Jakob is also looking to build a number of extensions to CrisisTracker and a “Share with Ushahidi” button may be one such future extension. Crisis-Tracker is basically Jakob’s core PhD project, which is very cool, so he’ll be working on this for at least one more year.

In sum, this could very well be the platform that many of us in the crisis mapping space have been waiting for. As I wrote in February 2012, turning the Twitter-sphere “into real-time shared awareness will require that our filtering and curation platforms become more automated and collaborative. I believe the key is thus to combine automated solutions with real-time collaborative crowd-sourcing tools—that is, platforms that enable crowds to collaboratively filter and curate real-time information, in real-time. Right now, when we comb through Twitter, for example, we do so on our own, sitting behind our laptop, isolated from others who may be seeking to filter the exact same type of content. We need to develop free and open source platforms that allow for the distributed-but-networked, crowdsourced filtering and curation of information in order to democratize the sense-making of the firehose.”

Actually, I’ve been advocating for this approach since early 2009. So I’m really excited about Jakob’s project. We’ll be partnering with him and the Standby Volunteer Task Force (SBTF) in September 2012 to test the platform and provide him with expert feedback on how to further streamline the tool for collaborative social media analysis and crisis mapping. Jakob is also looking for domain experts to help on this study. In the meantime, I’ve invited Jakob to present Crisis-Tracker at the 2012 CrisisMappers Conference in Washington DC and very much hope he can join us to demo his tool to us in person. In the meantime, the video above provides an excellent overview of CrisisTracker, as does the project website. Finally, the project is also open source and available on Github here.

Epilogue: The main problem with CrisisTracker is that it is still too manual; it does not include any machine learning & artificial intelligence features; and has only focused on Syria. This may explain why it has not gained traction in the humanitarian space so far.

8 Comments

Posted in Big Data, Crisis Mapping, Crowdsourcing, Humanitarian Technologies, Social Computing, Social Media

Tagged , , , , ,

Introducing GeoXray for Crisis Mapping

Posted on July 29, 2012 | 3 comments

My colleague Joel Myhre recently pointed me to Geosemble’s GeoXray platform, which “automatically filters content to your geographic area of interest and to your keywords of interest to provide you with timely, relevant information that enables you and your organization to make better decisions faster.” While I haven’t tested the platform, it seems similar to what Geofeedia offers.

Perhaps the main difference, beyond user-interface and maybe ease-of-use, is that GeoXray pulls in both external public content (from Twitter, Facebook, Blogs, News, PDFs, etc.) and internal sources such as private databases, documents etc. The platform allows users to search content by keyword, location and time. GeoXray also works off the Google Earth Engine, which enables visual-ization from different angles. The tool can also pull in content from Wikimapia and allows users to tag mapped content according to perceived veracity. One of the strengths of the platform appears to be the tool’s automated geo-location feature. For more on GeoXray:

3 Comments

Posted in Crisis Mapping, Crowdsourcing, Humanitarian Technologies, Social Media

Tagged ,

Enhanced Messaging for the Emergency Response Sector (EMERSE)

Posted on July 24, 2012 | 6 comments

My colleague Andrea Tapia and her team at PennState University have developed an interesting iPhone application designed to support humanitarian response. This application is part of their EMERSE project: Enhanced Messaging for the Emergency Response Sector. The other components of EMERSE include a Twitter crawler, automatic classification and machine learning.

The rationale for this important, applied research? “Social media used around crises involves self-organizing behavior that can produce accurate results, often in advance of official communications. This allows affected population to send tweets or text messages, and hence, make them heard. The ability to classify tweets and text messages automatically, together with the ability to deliver the relevant information to the appropriate personnel are essential for enabling the personnel to timely and efficiently work to address the most urgent needs, and to understand the emergency situation better” (Caragea et al., 2011).

The iPhone application developed by PennState is designed to help humanitarian professionals collect information during a crisis. “In case of no service or Internet access, the application rolls over to local storage until access is available. However, the GPS still works via satellite and is able to geo-locate data being recorded.” The Twitter crawler component captures tweets referring to specific keywords “within a seven-day period as well as tweets that have been posted by specific users. Each API call returns at most 1000 tweets and auxiliary metadata […].” The machine translation component uses Google Language API.

The more challenging aspect of EMERSE, however, is the automatic classification component. So the team made use of the Ushahidi Haiti data, which includes some 3,500 reports about half of which came from text messages. Each of these reports were tagged according to a specific (but not mutually exclusive category), e.g., Medical Emergency, Collapsed Structure, Shelter Needed, etc. The team at PennState experimented with various techniques from (NLP) and Machine Learning (ML) to automatically classify the Ushahidi Haiti data according to these pre-existing categories. The results demonstrate that “Feature Extraction” significantly outperforms other methods while Support Vector Machine (SVM) classifiers vary significantly depending on the category being coded. I wonder whether their approach is more or less effective than this one developed by the University of Colorado at Boulder.

In any event, PennState’s applied research was presented at the ISCRAM 2011 conference and the findings are written up in this paper (PDF): “Classifying Text Messages for the Haiti Earthquake.” The co-authors: Cornelia Caragea, Nathan McNeese, Anuj Jaiswal, Greg Traylor, Hyun-Woo Kim, Prasenjit Mitra, Dinghao Wu, Andrea H. Tapia, Lee Giles, Bernard J. Jansen, John Yen.

In conclusion, the team at PennState argue that the EMERSE system offers four important benefits not provided by Ushahidi.

“First, EMERSE will automatically classify tweets and text messages into topic, whereas Ushahidi collects reports with broad category information provided by the reporter. Second, EMERSE will also automatically geo-locate tweets and text messages, whereas Ushahidi relies on the reporter to provide the geo-location information. Third, in EMERSE, tweets and text messages are aggregated by topic and region to better understand how the needs of Haiti differ by regions and how they change over time. The automatic aggregation also helps to verify reports. A large number of similar reports by different people are more likely to be true. Finally, EMERSE will provide tweet broadcast and GeoRSS subscription by topics or region, whereas Ushahidi only allows reports to be downloaded.”

In terms of future research, the team may explore other types of abstraction based on semantically related words, and may also “design an emergency response ontology […].” So I recently got in touch with Andrea to get an update on this since their ISCRAM paper was published 14 months ago. I’ll be sure to share any update if this information can be made public.

6 Comments

Posted in Crowdsourcing, Humanitarian Technologies, Social Computing, Social Media

Tagged , , , ,

PeopleBrowsr: Next-Generation Social Media Analysis for Humanitarian Response?

Posted on July 11, 2012 | 2 comments

As noted in this blog post on “Data Philanthropy for Humanitarian Response,” members of the Digital Humanitarian Network (DHNetwork) are still using manual methods for media monitoring. When the United Nations Office for the Coordination of Humanitarian Affairs (OCHA) activated the Standby Volunteer Task Force (SBTF) to crisis map Libya last year, for example, SBTF volunteers manually monitored hundreds of Twitter handles, news sites for several weeks.

SBTF volunteers (Mapsters) do not have access to a smart microtasking platform that could have distributed the task in more efficient ways. Nor do they have access to even semi-automated tools for content monitoring and information retrieval. Instead, they used a Google Spreadsheet to list the sources they were manually monitoring and turned this spreadsheet into a sign-up sheet where each Mapster could sign on for 3-hour shifts every day. The SBTF is basically doing “crowd computing” using the equivalent of a typewriter.

Meanwhile, companies like Crimson Hexagon, NetBase, RecordedFuture and several others have each developed sophisticated ways to monitor social and/or mainstream media for various private sector applications such as monitoring brand perception. So my colleague Nazila kindly introduced me to her colleagues at PeopleBrowsr after reading my post on Data Philanthropy. Last week, Marc from PeopleBrowsr gave me a thorough tour of the platform. I was definitely impressed and am excited that Marc wants us to pilot the platform in support of the Digital Humanitarian Network. So what’s the big deal about PeopleBrowsr? To begin with, the platform has access to 1,000 days of social media data and over 3 terabytes of social data per month.

To put this in terms of information velocity, PeopleBrowsr receives 10,000 social media posts per second from a variety of sources including Twitter, Facebook, fora and blogs. On the latter, they monitor posts from over 40 million blogs including all of Tumblr, Posterious, Blogspot and every WordPress-hosted site. They also pull in content from YouTube and Flickr. (Click on the screenshots below to magnify them).

Lets search for the term “tsunami” on Twitter. (One could enter a complex query, e.g., and/or, not, etc., and also search using twitter handles, word or hashtag clouds, top URLs, videos, pictures, etc). PeopleBrowsr summarizes the result by Location and Community. Location simply refers to where those generating content referring to a tsunami are located. Of course, many Twitter users may tweet about an event without actually being eye-witness accounts (think of Diaspora groups, for example). While PeopleBrowsr doesn’t geo-tag the location of reports events, you can very easily and quickly identify which twitter users are tweeting the most about a given event and where they are located.

As for Community, PeopleBrowsr has  indexed millions of social media users and clustered them into different communities based on their profile/bio information. Given our interest in humanitarian response, we could create our own community of social media users from the humanitarian sector and limit our search to those users only. Communities can also be created based on hashtags. The result of the “tsunami” search is displayed below.

This result can be filtered further by gender, sentiment, number of twitter followers, urgent words (e.g., alert, help, asap), time period and location, for example. The platform can monitor and view posts in any language that is posted. In addition, PeopleBrowsr have their very own Kred score which quantifies the “credibility” of social media users. The scoring metrics for Kred scores is completely transparent and also community driven. “Kred is a transparent way to measure influence and outreach in social media. Kred generates unique scores for every domain of expertise. Regardless of follower count, a person is influential if their community is actively listening and engaging with their content.”

Using Kred, PeopleBrows can do influence analysis using Twitter across all languages. They’ve also added Facebook to Kred, but only as an opt in option.  PeopleBrowsr also has some great built-in and interactive data analytics tools. In addition, one can download a situation report in PDF and print that off if there’s a need to go offline.

What appeals to me the most is perhaps the full “drill-down” functionality of PeopleBrowsr’s data analytics tools. For example, I can drill down to the number of tweets per month that reference the word “tsunami” and drill down further per week and per day.

Moreover, I can sort through the individual tweets themselves based on specific filters and even access the underlying tweets complete with twitter handles, time-stamps, Kred scores, etc.

This latter feature would make it possible for the SBTF to copy & paste and map individual tweets on a live crisis map. In fact, the underlying data can be downloaded into a CSV file and added to a Google Spreadsheet for Mapsters to curate. Hopefully the Ushahidi team will also provide an option to upload CSVs to SwiftRiver so users can curate/filter pre-existing datasets as well as content generated live. What if you don’t have time to get on PeopleBrowsr and filter, download, etc? As part of their customer support, PeopleBrowsr will simply provide the data to you directly.

So what’s next? Marc and I are taking the following steps: Schedule online demo of PeopleBrowsr of the SBTF Core Team (they are for now the only members of the Digital Humanitarian Network with a dedicated and experienced Media Monitoring Team); SBTF pilots PeopleBrowsr for preparedness purposes; SBTF deploys  PeopleBrowsr during 2-3 official activations of the Digital Humanitarian Network; SBTF analyzes the added value of PeopleBrowsr for humanitarian response and provides expert feedback to PeopleBrowsr on how to improve the tool for humanitarian response.

2 Comments

Posted in Big Data, Crowdsourcing, Humanitarian Technologies, Social Computing, Social Media

Tagged , , ,

Surprising Findings: Using Mobile Phones to Predict Population Displacement After Major Disasters

Posted on July 8, 2012 | 9 comments

Rising concerns over the consequences of mass refugee flows during several crises in the late 1970’s is what prompted the United Nations (UN) to call for the establishment of early warning systems for the first time. “In 1978-79 for example, the United Nations and UNHCR were clearly overwhelmed by and unprepared for the mass influx of Indochinese refugees in South East Asia. The number of boat people washed onto the beaches there seriously challenged UNHCR’s capability to cope. One of the issues was the lack of advance information. The result was much human suffering, including many deaths. It took too long for emergency assistance by intergovernmental and non-governmental organizations to reach the sites” (Druke 2012 PDF).

Forty years later, my colleagues at Flowminder are using location data from mobile phones to nowcast and predict population displacement after major disasters. Focusing on the devastating 2010 Haiti earthquake, the team analyzed the movement of 1.9 million mobile users before and after the earthquake. Naturally, the Flowminder team expected that the mass exodus from Port-au-Prince would be rather challenging to predict. Surprisingly, however, the predictability of people’s movements remained high and even increased during the three-month period following the earthquake.

The team just released their findings in a peer-reviewed study entitled: “Predictability of population displacement after the 2010 Haiti earthquake” (PNAS 2012). As the analysis reveals, “the destinations of people who left the capital during the first three weeks after the earthquake was highly correlated with their mobility patterns during normal times, and specifically with the locations in which people had significant social bonds, as measured by where they spent Christmas and New Year holidays” (PNAS 2012).

For the people who left Port-au-Prince, the duration of their stay outside the city, as well as the time for their return, all followed a skewed, fat-tailed distribution. The findings suggest that population movements during disasters may be significantly more predictable than previously thought” (PNAS 2012). Intriguingly, the analysis also revealed the period of time that people in Port-au-Prince waited to leave the city (and then return) was “power-law distributed, both during normal days and after the earthquake, albeit with different exponents (PNAS 2012).” Clearly then, “[p]eople’s movements are highly influenced by their historic behavior and their social bonds, and this fact remained even after one of the most severe disasters in history” (PNAS 2012).

 

I wonder how this approach could be used in combination with crowdsourced satellite imagery analysis on the one hand and with Agent Based Models on the other. In terms of crowdsourcing, I have in mind the work carried out by the Standby Volunteer Task Force (SBTF) in partnership with UNHCR and Tomnod in Somalia last year. SBTF volunteers (“Mapsters”) tagged over a quarter million features that looked liked IDP shelters in under 120 hours, yielding a triangulated country of approximately 47,500 shelters.

In terms of Agent Based Models (ABMs), some colleagues and I  worked on “simulating population displacements following a crisis”  back in 2006 while at the Santa Fe Institute (SFI). We decided to use an Agent Based Model because the data on population movement was simply not within our reach. Moreover, we were particularly interested in modeling movements of ethnic populations after a political crisis and thus within the context of a politically charged environment.

So we included a preference for “safety in numbers” within the model. This parameter can easily be tweaked to reflect a preference for moving to locations that allow for the maintenance of social bonds as identified in the Flowminder study. The figure above lists all the parameters we used in our simple decision theoretic model.

The output below depicts the Agent Based Model in action. The multi-colored panels on the left depict the geographical location of ethnic groups at a certain period of time after the crisis escalates. The red panels on the right depict the underlying social networks and bonds that correspond to the geographic distribution just described. The main variable we played with was the size or magnitude of the sudden onset crisis to determine whether and how people might move differently around various ethnic enclaves. The study long with the results are available in this PDF.

In sum, it would be interesting to carry out Flowminder’s analysis in combination with crowdsourced satellite imagery analysis and live sensor data feeding into an Agent Base Model. Dissertation, anyone?

9 Comments

Posted in Big Data, Crisis Mapping, Crowdsourcing, Humanitarian Technologies

Tagged , , , , , ,

Finally, A Decision-Support Platform for SMS Use in Disaster Response

Posted on July 3, 2012 | 2 comments

Within weeks of the 2010 Haiti Earthquake, I published this blog post entitled “How to Royally Mess Up Disaster Response in Haiti.” A month later, I published another post on “Haiti and the Tyranny of Technology.” I also called for an SMS Code of Conduct as described here. Some of the needs and shortcomings expressed in these blog posts have finally been answered by InfoAsAid‘s excellent Message Library, “an online searchable database of messages that acts as a reference for those wanting to disseminate critical information to affected populations in an emergency.”

“If used in the correct way, the library should help improve communication with crisis-affected populations.” As my colleague Anahi Ayala explains with respect to the disaster response in Haiti,

“One of the main problem that emerged was not only the need to communicate but the need for a coordinated and homogeneous message to be delivered to the affected communities. The problem was posed by the fact that as agencies and organizations were growing in number and size, all of them were trying in different ways to deliver messages to the beneficiaries of aid, with the result of many messages, sometimes contradicting each other, delivered to many people, sometimes not the right receiver for that message.”

This platform can be used for both disaster response and preparedness. In the latter case, preparedness exercises can “Involve communities to identify threats and draft appropriate messages using the message library as a reference.” Organizations can also “Pre-test the messages with different segments of society (consider differences in gender, rural/urban, education levels, age) to ensure comprehension.” In terms of disaster response, the platform can be used to disseminate information on the “scale, nature and impact of the disaster (humanitarian news); Alerts about secondary disasters such as aftershocks, landslides or flooding; Messages about how to stay safe and mitigate risk in the face of anticipated threats.”

At PeaceTXT, we’re taking a very similar approach to SMS messaging. In our case, we are developing an SMS Library specifically for the purposes of changing recipients’ behaviors and perceptions vis-a-vis peace and conflict issues in Kenya. This shift towards a culture of preparedness is really important, both for disaster response and conflict prevention. We are currently organizing a series of focus groups with local communities to develop the content of our SMS Library. We plan to review this content in August for inclusion in the library. I very much look forward to scheduling a conference call between InfoAsAid and PeaceTXT in the coming months to share lessons learned thus far in the development of our respective message libraries.

For more on InfoAsAid’s absolutely critical resource, this short video provides a very good summary, including how sensitive messages are managed and how you can contribute SMS content to this very important service. Some serious thanks and praise are in order for InfoAsAid’s work. I do hope that the team at InfoAsAid will join us at the International Crisis Mappers Conference  (ICCM 2012) to share the latest on their excellent initiatives.

2 Comments

Posted in Humanitarian Technologies

Tagged , , , , , , , , ,