Tag Archives: QCRI

QED – Goodbye Doha, Hello Adventure!

Quod Erat Demonstrandum (QED) is Latin for “that which had to be proven.” This abbreviation was traditionally used at the end of mathematical proofs to signal the completion of said proofs. I joined the Qatar Computing Research Institute (QCRI) well over 3 years ago with a very specific mission and mandate: to develop and deploy next generation humanitarian technologies. So I built the Institute’s Social Innovation Program from the ground up and recruited the majority of the full-time experts (scientists, engineers, research assistants, interns & project manager) who have become integral to the Program’s success. During these 3+years, my team and I partnered directly with humanitarian and development organizations to empirically prove that methods from advanced computing can be used to make sense of Big (Crisis) Data. The time has thus come to add “QED” to the end of that proof and move on to new adventures. But first a reflection.

Over the past 3.5 years, my team and I at QCRI developed free and open source solutions powered by crowdsourcing and artificial intelligence to make sense of Tweets, text messages, pictures, videos, satellite and aerial imagery for a wide range of humanitarian and development projects. We co-developed and co-deployed these platforms (AIDR and MicroMappers) with the United Nations and the World Bank in response to major disasters such as Typhoons Haiyan and RubyCyclone Pam and both the Nepal & Chile Earthquakes. In addition, we carried out peer-reviewed, scientific research on these deployments to better understand how to meet the information needs of our humanitarian partners. We also tackled the information reliability question, experimenting with crowd-sourcing (Verily) and machine learning (TweetCred) to assess the credibility of information generated during disasters. All of these initiatives were firsts in the humanitarian technology space.

We later developed AIDR-SMS to auto-classify text messages; a platform that UNICEF successfully tested in Zambia and which the World Food Program (WFP) and the International Federation of the Red Cross (IFRC) now plan to pilot. AIDR was also used to monitor a recent election, and our partners are now looking to use AIDR again for upcoming election monitoring efforts. In terms of MicroMappers, we extended the platform (considerably) in order to crowd-source the analysis of oblique aerial imagery captured via small UAVs, which was another first in the humanitarian space. We also teamed up with excellent research partners to crowdsource the analysis of aerial video footage and to develop automated feature-detection algorithms for oblique imagery analysis based on crowdsourced results derived from MicroMappers. We developed these Big Data solutions to support damage assessment efforts, food security projects and even this wildlife protection initiative.

In addition to the above accomplishments, we launched the Internet Response League (IRL) to explore the possibility of leveraging massive multiplayer online games to process Big Crisis Data. Along similar lines, we developed the first ever spam filter to make sense of Big Crisis Data. Furthermore, we got directly engaged in the field of robotics by launching the Humanitarian UAV Network (UAViators), yet another first in the humanitarian space. In the process, we created the largest repository of aerial imagery and videos of disaster damage, which is ripe for cutting-edge computer vision research. We also spearheaded the World Bank’s UAV response to Category 5 Cyclone Pam in Vanuatu and also directed a unique disaster recovery UAV mission in Nepal after the devastating earthquakes. (I took time off from QCRI to carry out both of these missions and also took holiday time to support UN relief efforts in the Philippines following Typhoon Haiyan in 2013). Lastly, on the robotics front, we championed the development of international guidelines to inform the safe, ethical & responsible use of this new technology in both humanitarian and development settings. To be sure, innovation is not just about the technology but also about crafting appropriate processes to leverage this technology. Hence also the rationale behind the Humanitarian UAV Experts Meetings that we’ve held at the United Nations Secretariat, the Rockefeller Foundation and MIT.

All  of the above pioneering-and-experimental projects have resulted in extensive media coverage, which has placed QCRI squarely on the radar of international humanitarian and development groups. This media coverage has included the New York Times, Washington Post, Wall Street Journal, CNN, BBC News, UK Guardian, The Economist, Forbes and Times Magazines, New Yorker, NPR, Wired, Mashable, TechCrunch, Fast Company, Nature, New Scientist, Scientific American and more. In addition, our good work and applied research has been featured in numerous international conference presentations and keynotes. In sum, I know of no other institute for advanced computing research that has contributed this much to the international humanitarian space in terms of thought-leadership, strategic partnerships, applied research and operational expertise through real-world co-deployments during and after major disasters.

There is, of course, a lot more to be done in the humanitarian technology space. But what we have accomplished over the past 3 years clearly demonstrates that techniques from advanced computing can indeed provide part of the solution to the pressing Big Data challenge that humanitarian & development organizations face. At the same time, as I wrote in the concluding chapter of my new book, Digital Humanitarians, solving the Big Data challenge does not alas imply that international aid organizations will actually make use of the resulting filtered data or any other data for that matter—even if they ask for this data in the first place. So until humanitarian organizations truly shift towards both strategic and tactical evidence-based analysis & data-driven decision-making, this disconnect will surely continue unabated for many more years to come.

Reflecting on the past 3.5 years at QCRI, it is crystal clear to me that the number one most important lesson I (re)learned is that you can do anything if you have an outstanding, super-smart and highly dedicated team that continually goes way above and beyond the call of duty. It is one thing for me to have had the vision for AIDR, MicroMappers, IRL, UAViators, etc., but vision alone does not amount to much. Implementing said vision is what delivers results and learning. And I simply couldn’t have asked for a more talented & stellar team to translate these visions into reality over the past 3+years. You each know who you are, partners included; it has truly been a privilege and honor working with you. I can’t wait to see what you do next at/with QCRI. Thank you for trusting me; thank you for sharing my vision; thanks for your sense of humor, and thank you for your dedication and loyalty to science and social innovation.

So what’s next for me? I’ll be lining up independent consulting work with several organizations (likely including QCRI). In short, I’ll be open for business. I’m also planning to work on a new project that I’m very excited about, so stay tuned for updates; I’ll be sure to blog about this new adventure when the time is right. For now, I’m busy wrapping up my work as Director of Social Innovation at QCRI and working with the best team there is. QED.

MicroMappers Launched for Pakistan Earthquake Response (Updated)

Update 1: MicroMappers is now public! Anyone can join to help the efforts!
Update 2: Results of MicroMappers Response to Pakistan Earthquake [Link]

MicroMappers was not due to launch until next month but my team and I at QCRI received a time-sensitive request by colleagues at the UN to carry out an early test of the platform given yesterday’s 7.7 magnitude earthquake, which killed well over 300 and injured hundreds more in south-western Pakistan.

pakistan_quake_2013

Shortly after this request, the UN Office for the Coordination of Humanitarian Affairs (OCHA) in Pakistan officially activated the Digital Humanitarian Network (DHN) to rapidly assess the damage and needs resulting from the earthquake. The award-winning Standby Volunteer Task Force (SBTF), a founding member of the DHN. teamed up with QCRI to use MicroMappers in response to the request by OCHA-Pakistan. This exercise, however, is purely for testing purposes only. We made this clear to our UN partners since the results may be far from optimal.

MicroMappers is simply a collection of microtasking apps (we call them Clickers) that we have customized for disaster response purposes. We just launched both the Tweet and Image Clickers to support the earthquake relief and may also launch the Tweet and Image GeoClickers as well in the next 24 hours. The TweetClicker is pictured below (click to enlarge).

MicroMappers_Pakistan1

Thanks to our partnership with GNIP, QCRI automatically collected over 35,000 tweets related to Pakistan and the Earthquake (we’re continuing to collect more in real-time). We’ve uploaded these tweets to the TweetClicker and are also filtering links to images for upload to the ImageClicker. Depending on how the initial testing goes, we may be able to invite help from the global digital village. Indeed, “crowdsourcing” is simply another way of saying “It takes a village…” In fact, that’s precisely why MicroMappers was developed, to enable anyone with an Internet connection to become a digital humanitarian volunteer. The Clicker for images is displayed below (click to enlarge).

MicroMappers_Pakistan2

Now, whether this very first test of the Clickers goes well remains to be seen. As mentioned, we weren’t planning to launch until next month. But we’ve already learned heaps from the past few hours alone. For example, while the Clickers are indeed ready and operational, our automatic pre-processing filters are not yet optimized for rapid response. The purpose of these filters is to automatically identify tweets that link to images and videos so that they can be uploaded to the Clickers directly. In addition, while our ImageClicker is operational, our VideoClicker is still under development—as is our TranslateClicker, both of which would have been useful in this response. I’m sure will encounter other issues over the next 24-36 hours. We’re keeping track of these in a shared Google Spreadsheet so we can review them next week and make sure to integrate as much of the feedback as possible before the next disaster strikes.

Incidentally, we (QCRI) also teamed up with the SBTF to test the very first version of the Artificial Intelligence for Disaster Response (AIDR) platform for about six hours. As far as we know, this test represents the first time that machine learning classifiers for disaster resposne were created on the fly using crowdsourcing. We expect to launch AIDR publicly at the 2013 CrisisMappers conference this November (ICCM 2013). We’ll be sure to share what worked and didn’t work during this first AIDR pilot test. So stay tuned for future updates via iRevolution. In the meantime, a big, big thanks to the SBTF Team for rallying so quickly and for agreeing to test the platforms! If you’re interested in becoming a digital humanitarian volunteer, simply join us here.

Bio

Introducing MicroMappers for Digital Disaster Response

The UN activated the Digital Humanitarian Network (DHN) on December 3, 2012 to carry out a rapid damage needs assessment in response to Typhoon Pablo in the Philippines. More specifically, the UN requested that Digital Humanitarians collect and geo-reference all tweets with links to pictures or video footage capturing Typhoon damage. To complete this mission, I reached out to my colleagues at CrowdCrafting. Together, we customized a microtasking app to filter, classify and geo-reference thousands of tweets. This type of rapid damage assessment request was the first of its kind, which means that setting up the appropriate workflows and technologies took a while, leaving less time for the tagging, verification and analysis of the multimedia content pointed to in the disaster tweets. Such is the nature of innovation; optimization takes place through iteration and learning.

Microtasking is key to the future of digital humanitarian response, which is precisely why I am launching MicroMappers in partnership with CrowdCrafting. MicroMappers, which combimes the terms Micro-Tasking and Crisis-Mappers, is a collection of free & open source microtasking apps specifically customized and optimized for digital disaster response. The first series of apps focus on rapid damage assessment activations. In other words, the apps include Translate, Locate and Assess. The Translate & Locate Apps are self-explanatory. The Assess App enables digital volunteers to quickly tag disaster tweets that link to relevant multimedia that captures disaster damage. This app also invites volunteers to rate the level of damage in each image and video.

For example, say an earthquake strikes Mexico City. We upload disaster tweets with links to the Translate App. Volunteer translators only translate tweets with location information. These get automatically pushed to the Assess App where digital volunteers tag tweets that point to relevant images/videos. They also rate the level of damage in each. (On a side note, my colleagues and I at QCRI are also developing a crawler that will automatically identify whether links posted on twitter actually point to images/videos). Assessed  tweets are then pushed in real-time to the Locate App for geo-referencing. The resulting tweets are subsequently published to a live map where the underlying data can also be downloaded.  Both the map & data download feature can be password protected.

The plan is to have these apps online and live 24/7 in the event of an activation request. When a request does come in, volunteers with the Digital Humanitarian Network will simply go to MicroMappers.com (not yet live) to start using the apps right away. Members of the public will also be invited to support these efforts and work along side digital humanitarian volunteers. In other words, the purpose of the MicroMappers Apps is not only to facilitate and accelerate digital humanitarian efforts but also to radically democratize these efforts by increasing the participation base. To be sure, one doesn’t need prior training to microtask, simply being able to read and access the web will make you an invaluable member of the team.

We plan to have the MicroMappers Apps completed in May/June September for testing by members of the Digital Humanitarian Network. In the meantime, huge thanks to our awesome partners at CrowdCrafting for making all of this possible! If you’re a coder and interested in contributing to these efforts, please feel free to get in touch with me. We may be able to launch and test these apps earlier with your help. After all, disasters won’t wait until we’re ready and we have several more disaster response apps that are in need of customization.

bio

MAQSA: Social Analytics of User Responses to News

Designed by QCRI in partnership with MIT and Al-Jazeera, MAQSA provides an interactive topic-centric dashboard that summarizes news articles and user responses (comments, tweets, etc.) to these news items. The platform thus helps editors and publishers in newsrooms like Al-Jazeera’s better “understand user engagement and audience sentiment evolution on various topics of interest.” In addition, MAQSA “helps news consumers explore public reaction on articles relevant to a topic and refine their exploration via related entities, topics, articles and tweets.” The pilot platform currently uses Al-Jazeera data such as Op-Eds from Al-Jazeera English.

Given a topic such as “The Arab Spring,” or “Oil Spill”, the platform combines time, geography and topic to “generate a detailed activity dashboard around relevant articles. The dashboard contains an annotated comment timeline and a social graph of comments. It utilizes commenters’ locations to build maps of comment sentiment and topics by region of the world. Finally, to facilitate exploration, MAQSA provides listings of related entities, articles, and tweets. It algorithmically processes large collections of articles and tweets, and enables the dynamic specification of topics and dates for exploration.”

While others have tried to develop similar dashboards in the past, these have “not taken a topic-centric approach to viewing a collection of news articles with a focus on their user comments in the way we propose.” The team at QCRI has since added a number of exciting new features for Al-Jazeera to try out as widgets on their site. I’ll be sure to blog about these and other updates when they are officially launched. Note that other media companies (e.g., UK Guardian) will also be able to use this platform and widgets once they become public.

As always with such new initiatives, my very first thought and question is: how might we apply them in a humanitarian context? For example, perhaps MAQSA could be repurposed to do social analytics of responses from local stakeholders with respect to humanitarian news articles produced by IRIN, an award-winning humanitarian news and analysis service covering the parts of the world often under-reported, misunderstood or ignored. Perhaps an SMS component could also be added to a MAQSA-IRIN platform to facilitate this. Or perhaps there’s an application for the work that Internews carries out with local journalists and consumers of information around the world. What do you think?

Muḥammad ibn Mūsā al-Khwārizmī: An Update from the Qatar Computing Research Institute

I first heard of al-Khwārizmī in my ninth-grade computer science class at the International School of Vienna (AIS) back in 1993. Dr. Herman Prossinger who taught the course is exactly the kind of person one describes when answering the question: which teacher had the most impact on you while growing up? I wonder how many other 9th graders in the world had the good fortune of being taught computer science by a full-fledged professor with a PhD dissertation entitled “Isothermal Gas spheres in General Relativity Theory” (1976) and numerous peer-reviewed publications in top-tier scientific journals including Nature?

Muḥammad ibn Mūsā al-Khwārizmī was a brilliant mathematician & astronomer who spent his time as a scholar in the House of Wisdom in Baghdad (possibly the best name of any co-working space in history). “Al-Khwarithmi” was initially transliterated into Latin as Algoritmi. The manuscript above, for example, begins with “DIXIT algorizmi,” meaning “Says al-Khwārizmī.” And thus was born the world AlgorithmBut al-Khwārizmī’s fundamental contributions were not limited to the fields of mathematics and astronomy, he is also well praised for his important work on geography and cartography. Published in 833, his Kitāb ṣūrat al-Arḍ (Arabic: كتاب صورة الأرض) or “Book on the Appearance of the Earth” was a revised and corrected version of Ptolemy’s Geography. al-Khwārizmī’s book comprised an impressive list of 2,402 coordinates of cities and other geo-graphical features. The only surviving copy of the book can be found at Strasbourg University. I’m surprised the item has not yet been purchased by Qatar and relocated to Doha.

View of the bay from QCRI in Doha, Qatar.

This brings me to the Qatar (Foundation) Computing Research Institute (QCRI), which was almost called the al-Khwārizmī Computing Research Institute. I joined QCRI exactly two weeks ago as Director of Social Innovation. My first impression? QCRI is Doha’s “House of Whizzkids”. The team is young, dynamic, international and super smart. I’m already working on several exploratory research and development (R&D) projects that could potentially lead to initial prototypes by the end of the year. These have to do with the application of social computing and big data analysis for humanitarian response. So I’ve been in touch with several colleagues at the United Nations (UN) Office for the Coordination of Humanitarian Affairs (OCHA) to bounce these early ideas off and am thrilled that all responses thus far have been very positive.

My QCRI colleagues and I are also looking into collaborative platforms for “smart microtasking” which may be useful for the Digital Humanitarian Network. In addition, we’re just starting to explore potential solutions for quantifying veracity in social media, a rather non-trivial problem as Dr. Prossinger would often say with a sly smile in relation to NP-hard problems. In terms of partner-ship building, I will be in New York, DC and Boston next month for official meetings with the UN, World Bank and MIT to explore possible collaborations on specific projects. The team in Doha is particularly strong on big data analytics, social computing, data cleaning, machine learning and translation. In fact, most of the whizzkids here come from very impressive track records with Microsoft, Yahoo, Ivy Leagues, etc. So I’m excited by the potential.

View of Tornado Tower (purple lights) where QCRI is located.

The reason I’m not going into specifics vis-a-vis these early R&D efforts is not because I want to be secretive or elusive. Not at all. We’re still refining the ideas ourselves and simply want to manage expectations. There is a very strong and genuine interest within QCRI to contribute meaningfully to the humanitarian technology space. But we’re really just getting started, still hiring left, center and right, and we’ll be in R&D mode for a while. Plus, we don’t want to rush just for the sake of launching a new product. All too often, humanitarian technologies are developed without the benefit (and luxury) of advanced R&D. But if QCRI is going to help shape next-generation humanitarian technology solutions, we should do this in a way that is deliberate, cutting-edge and strategic. That is our comparative advantage.

In sum, the outcome of our R&D efforts may not always lead to a full-fledged prototype, but all the research and findings we carry out will definitely be shared publicly so we can move the field forward. We’re also committed to developing free and open source software as part of our prototyping efforts. Finally, we have no interest in re-inventing the wheel and far prefer working in partnerships than in isolation. So there we go, time to R&D  like al-Khwārizmī.

Using Rayesna to Track the 2012 Egyptian Presidential Candidates on Twitter

My (future) colleague at the Qatar Foundation’s Computing Research Institute (QCRI) have just launched a new platform that Al Jazeera is using to track the 2012 Egyptian Presidential Candidates on Twitter. Called Rayesna, which  means “our president” in colloquial Egyptian Arabic, this fully automated platform uses cutting-edge Arabic computational linguistics processing developed by the Arabic Language Technology (ALT) group at QCRI.

“Through Rayesna, you can find out how many times a candidate is mentioned, which other candidate he is likely to appear with, and the most popular tweets for a candidate, with a special category for the most retweeted jokes about the candidates. The site also has a time-series to explore and compares the mentions of the candidate day-by-day. Caveats: 1. The site reflects only the people who choose to tweet, and this group may not be representative of general society; 2. Tweets often contain foul language and we do not perform any filtering.”

I look forward to collaborating with the ALT group and exploring how their platform might also be used in the context of humanitarian response in the Arab World and beyond. There may also be important synergies with the work of the UN Global Pulse, particularly vis-a-vis their use of Twitter for real-time analysis of vulnerable communities.

Joining the Qatar Foundation to Advance Humanitarian Technology

Big news! I’ll be taking a senior level position at the Qatar Foundation to work on the next generation of humanitarian technology solutions. I’ll be based at the Foundation’s Computing Research Institute (QCRI) and be working alongside some truly amazing minds defining the cutting edge of social and scientific computing, computational linguistics, big data, etc. My role at QCRI will be to leverage the expertise within the Institute, the region and beyond to drive technology solutions for humanitarian and social impact globally—think of it as Computing for Good backed by some serious resources.  I’ll spend just part of the time in Doha. The rest of my time will be based wherever necessary to have the greatest impact. Needless to say, I’m excited!

My mission over the past five years has been to catalyze strategic linkages between the technology and humanitarian space to promote both innovation and change, so this new adventure feels like the perfect next chapter in this exciting adventure. I’ve had the good fortune and distinct honor of working with some truly inspiring and knowledgeable colleagues who have helped me define and pursue my passions over the years. Needless to say, I’ve learned a great deal from these colleagues; knowledge, contacts and partnerships that I plan to fully leverage at the Qatar Foundation.

It really has been an amazing five years. I joined the Harvard Humanitarian Initiative (HHI) in 2007 to co-found and co-direct the Program on Crisis Mapping and Early Warning. The purpose of the program was to assess how new technologies were changing the humanitarian space and how these could be deliberately leveraged to yield more significant impact. As part of my time at HHI, I consulted on a number of cutting-edge projects including the UNDP’s Crisis and Risk Mapping Analysis (CRMA) Program in the Sudan. I also leveraged this iRevolution blog extensively to share my findings and learnings with both the humanitarian and technology communities. In addition, I co-authored the UN Foundation & Vodafone Foundation Report on “New Technologies in Emergen-cies and Conflicts” (PDF).

Towards the end of HHI’s program in 2009, I co-launched the Humanitarian Technology Network, CrisisMappers, and have co-organized and curated each International Conference of Crisis Mappers (ICCM) since then. The Network now includes close to 4,000 members based in some 200 countries around the world. Last year, ICCM 2011 brought together more than 400 participants to Geneva, Switzerland to explore and define the cutting edge of humanitarian technology. This year, ICCM 2012 is being hosted by the World Bank and will no doubt draw an even greater number of experts from the humanitarian & technology space.

I joined Ushahidi as Director of Crisis Mapping shortly after launching the Crisis Mappers Network. My goal was to better understand the field of crisis mapping from the perspective of a technology company and to engage directly with international humanitarian, human rights and media organizations so they too could better understand how to leverage the technologies in the Ushahidi ecosystem. There, I spearheaded several defining crisis mapping projects including Haiti, Libya, Somalia and Syria in partnership with key humanitarian, human rights and media organizations. I also spoke at many high-profile conferences to share many of the lessons learned and best practices resulting from these projects. I am very grateful to these conference organizers for giving me the stage at so many important events, thank you very much. And of course, special thanks to the team at Ushahidi for the truly life-changing experience.

Whilst at Ushahidi, I also completed my PhD during my pre-doctoral fellowship at Stanford and co-founded the award-winning Standby Volunteer Task Force (SBTF) to provide partner organizations with surge capacity for live mapping support. I co-created the SBTF’s Satellite Imagery Team to apply crowdsourcing and micro-tasking to satellite imagery analysis in support of humanitarian operations. I also explored a number of promising data mining solutions for social media analysis vis-a-vis crisis response. More recently, I co-launched the Digital Humanitarian Network (DHN) in partnership with a UN colleague.

The words “co-founded,” “co-launched,” and “co-directed” appear throughout the above because all these initiatives are the direct result of major team-work, truly amazing partners and inspiring mentors. You all know who you are. Thank you very much for your guidance, expertise, friendship and for your camara-derie throughout. I look forward to collaborating with you even more once I get settled at the Qatar Foundation.

To learn more about QCRI’s work thus far, I recommend watching the above presentation given by the Institute’s Director who has brought together an incredible team—professionals who all share his ambition and exciting vision. When we began to discuss my job description at the Foundation, I was simply told: “Think Big.” The Institute’s Advisory Board is also a source of excitement for me: Joichi Ito (MIT) and Rich deMillo (GeorgiaTech), to name a few. 

Naturally, the Qatar Foundation also has access to tremendous resources and an amazing set of partners from multiple sectors in Doha, the region and across the globe. In short, the opportunity for QCRI to become an important contributor to the humanitarian technology space is huge. I look forward to collaborating with many existing colleagues and partners to turn this exciting opportunity into reality and look forward to continuing this adventure with an amazing team of experts in Doha who are some of the best in their fields. More soon!