The ICTD2009 conference in Doha, Qatar, had some excellent tech demo’s. I had the opportunity to interview Kuang Chen, a PhD student with UC Berkeley’s computer science department about his work on improving data quality using dynamic forms and machine learning.
I’m particularly interested in this area of research since ensuring data quality continues to be a real challenge in the fields of conflict early warning and crisis mapping. So I always look for alternative and creative approaches that address this challenge. I include below the abstract for Kuang’s project (which includes 5 other team members) and a short 2-minute interview.
Abstract
“Organizations in developing regions want to efficiently collect digital data, but standard data gathering practices from the developed world are often inappropriate. Traditional techniques for form design and data quality are expensive and labour-intensive. We propose a new data-driven approach to form design, execution (filling) and quality assurance. We demonstrate USHER, an end-to-end system that automatically generates data entry forms that enforce and maintain data quality constraints during execution. The system features a probabilistic engine that drives form-user interactions to encourage correct answers.”
In my previous post on data quality evaluation, I pointed to a study that suggests mobile-based data entry has significantly higher error rates. The study shows that a voice call to a human operator results in superior data quality—no doubt due to the human operator double-checking the respondent’s input verbally. USHER’s ability to dynamically adjust the user interface (form layout and data entry widgets) is one approach to provide some context-specific data-driven user feedback that is currently lacking in mobile forms, as an automated proxy of a human data entry person on the other end of the line.
Interview
This is my first video so many thanks to Erik Hersman for his tips on video editing! And many thanks to Kuang for the interview.
iRevolutions 
There were also a couple of oral paper presentations at the ICTD2009 conference on data quality via different methods of data entry (voice vs. touchtone, and voice vs. SMS vs. forms).
If you’ve got a very patient Internet connection, you can temporarily get a draft of the proceedings (all papers/posters plus one-page abstracts for demos) at: http://www.qatar.cmu.edu/~yonina/ICTD2009Proceedings.pdf
I notice the proceedings has also been loaded onto scribd.com
Thanks Richard, I already blogged about the voice vs. touchtone, and voice vs. SMS vs. forms presentation here:
https://irevolution.wordpress.com/2009/04/18/evaluating-accuracy-of-data-collection-on-mobile-phones/
Have you taken a look at http://your.flowingdata.com/? I think the next version will be hugely valuable for mobile data collection. I’m beta testing right now and am really impressed.
Thanks for the tip, David, starting to YFD now. Looking forward to getting an invite to see how we can use this in humanitarian contexts.
based on USHER we made presentation on-HURDEN-improving data quality using adaptive forms
Thank you for sharing