Simon@iSchool

My research (the speech-enabled interface to Hesperian Foundation’s healthcare content) was not just a driver for participation in the Developing Rural Computing Applications (DRCA) course. It also was the focus of my work in INFO 214 – Needs and Usability Assessment.

Throughout the semester, alongside my participation in DRCA, I have been working with Matt Gedigian of the School of Information and Alean Daniel of Mechanical Engineering on a usability study for Hesperian. The goals were to report on Hesperian’s vision for the project, evaluate a prototype we built (as part of research, but also for the sake of having something to test), and examine the field of automated phone systems in general to identify useful features to consider for future iterations of the project.

Our findings from three usability methods (interviews, a formal usability study, and a competitive analysis) have been compiled into a report suitable for digestion by Hesperian staff and the administrators of INFO 214.

You can access the final report here.

Future Plans

After taking this course, I plan to continue my existing work for the Hesperian Foundation (see the Research Abstract) with a renewed sense of inspiration and a better understanding of the landscape of rural computing applications. The project ideas I heard from fellow classmates were interesting, but I have committed to my current project for the forseeable future.

In many ways, the work I’ll be doing is much like Neil Patel’s work. I have been working on an automated telephone system based on VoiceXML, the same technology powering IBM’s “Spoken Web” and Patel’s call-in system for rural farmers to access recordings of radio programs.

Currently, I have three small VoiceXML programs that could represent three different components of a larger, encompassing automated telephone system.

• A program that reads short summaries of diseases given the names of diseases as input
• A program that walks through a diagnosis of acute diarrhea and provides specific advice based on yes/no inputs
• A program that provides a walkthrough on how to treat skin problems, outlining the individual steps to perform a hot/cold compress

I’m performing evaluation of the systems for a Needs and Usability Assessment course. Due to the impracticality of performing a real field study, I am currently only conducting usability studies with Hesperian’s local employees. (I ask them to imagine themselves in the position of a health worker while using these systems; their experience with Hesperian’s materials and their target audience helps them do this.) While this study is not as ecologically valid or thorough as Patel’s study, I hope to be able to gain enough insight into the strengths and weaknesses of each of my VoiceXML programs and apply the findings to produce a more polished system.

From there, I hope to take the project beyond Patel’s voice/telephone-only approach and move into using XHTML+Voice to provide a multi-modal interface to Hesperian’s materials. The idea is to take the spoken interaction model that is developed with VoiceXML and combine it with the visual elements of the Digital Library website. How this will be done is not entirely clear now, but I know that success will require many iterations, a lot of input from a sampling of potential users as well as Hesperian, and thoughtful application of the insights I learned through this course.

Feedback

Some feedback that came out of the group discussion in the last class:

If possible, have student presentations earlier in the semester so we get ‘warmed up’ to the idea and have more confidence in our own abilities before hearing about some of the major projects powered by whole organizations.

Form into groups early in the semester. Formation can be determined by area of interest (e.g. healthcare, education) or geographical area of study (e.g. Africa, India). These groups should do research into their respective areas throughout the semester, using what they learn in the class to enhance and inform their work. Later in the semester, there could be interactions between the groups to facilitate sharing of their findings.

Conclusion

This has been a very enlightening and worthwhile course at the School of Information. It’s clear that the field of ICT4D is one with much potential for significant impact, but only if practicioners learn from each other and sets of “lessons learned” are passed from one generation of successful researchers to the next. Courses like this one are a great way of doing that, and expanding the offering in the future seems like a natural next step.

Sonesh came this week to talk about TIER’s efforts in providing Internet access to rural communities through long-range wireless systems. Some of their systems have broken world records for wireless connectivity between two points, with towers getting as far as 279km away from each other in Venezuela!

Sonesh described the experiences he had while attempting to train and educate the people of the communities he served; he informed them of how to maintain the wireless satellites they set up on their rooftops and what to do if things stopped working. His group was considerate enough to set up hotlines and other methods of contacting them for support, and he told us stories of some of the bizarre things that he heard on these support lines.

One of the most interesting stories from the talk was about such a support request. These all begin the same way: Connectivity goes down and the Internet becomes inaccessible for the people on a wide scale, so they get a call. Just like any other decent support line, they walk their customers through a troubleshooting process, but nothing seems to be working to fix the problem. Eventually, someone is forced to go to the scene to see if there is something wrong with the satellite dish. It turns out that the dish itself had been occluded by the construction of a new elevator shaft that extended up to the rooftop where the device was installed! It completely blocked its line-of-sight to the next satellite in their system, rendering it completely unusable.

The need for the satellite dish to have a line-of-sight to the next satellite seemed obvious to the TIER group, but it probably was never detailed clearly to the people who maintained that node. Instead, they were trained on all the other network maintenance issues they might come across, while never really understanding the basics of how the system worked.

For me, this shows that nothing is ‘too obvious’ to withhold from customers who have never worked with your technology before. It is far too easy for us ‘techies’ to assume that something should be obvious or straightforward when it really isn’t to all people. Stating that the device needed a line-of-sight would have taken a few seconds during training, and would have saved hours of support time had this issue been prevented.

Neil Patel from Stanford University came and spoke about the projects he worked on over the past two summers in rural India, which included an organic certification system (Jatan) and an automated phone system that doled out advice for farmers. Neil pointed out that farming as a way of life was on the decline, since it was not seen as a modern or lucrative profession. His projects aimed to revitalize the industry and provide ways for worried farmers to find new ways to make their jobs more profitable. The end result involves organic farms that operate more efficiently as well as a farming population that is able to share and retrieve trade advice via electronic means.

I was particularly interested in Neil’s work on his automated phone system. It uses the same technology as the project I’m working on (VoiceXML for automated telephone interaction) and he performed quite a number of usability assessments of it with the farmers he worked with.

Some of his findings were surprising to me.

For example, he found that it was better for the system to name explicit voice prompts for unfamiliar users instead of letting them say anything and having the system try to interpret it. What this means is that the system would prompt with such statements as, “To ask a question, say ‘Question’; to listen to radio, say ‘Radio’,” as opposed to just letting the user have an open-ended prompt from which the system could try and detect the trigger words. This approach goes against conventional user interface guidelines for voice, which typically suggest going for a ‘conversational’ style in interaction models. Neil said that he found that, in particular, the women in the villages he encountered were very nervous about having an open-ended prompt; they would not know what to say and would end up just not saying anything at all. I wonder if this is a tendency that is reproducible with voice interaction systems here, or if it is a product of the culture in India.

Neil also tried comparing the efficacy of voice versus using dial tones (DTMF, push buttons) for his system. He developed two prototypes to evaluate, one requiring voice input in short bursts (i.e. mostly ‘yes’ and ‘no’) and one that asked for the user to push buttons (i.e. ‘1′ and ‘2′, respectively).

After writing up three tasks (easy, medium, and hard) to accomplish with his system and running a controlled study, what he found was that DTMF tended to outperform voice in nearly all his situations. His users tended to learn faster by using dial tones and were less likely to make mistakes or become frustrated. He attributed this to the weakness of the voice recognition (it was based on constructing non-English words from English phonemes) and expressed hope that his study could be repeated elsewhere in India to confirm or refute his results. Personally, I was disheartened to learn of the difficulty the users had with voice, but felt that his approach of using phonemes for recognition was the best that anyone could have done with the current state-of-the-art in voice recognition.

Perhaps the best answer to the speech vs. DTMF question is to use both – provide support for both saying the term desired as well as dial tone input when it fails repeatedly.

We had more guest presentations today in class, with some from the Haas School of Business and Blum Center for Developing Economies.

The first speaker discussed his experiences learning about Community Health Worker programs that are so common in the developing world today. ICT4D can have a huge impact on the lives of health workers (i.e. in aiding their medical service, record-keeping, etc.), which is my guess at why we’ve heard so much about them as a user group this semester.

(I, myself, have an interest in health workers because my research also is an attempt to make their lives easier.)

One aspect I find interesting: There are NGOs who recruit paid health workers and those who maintain a volunteer force. There wasn’t much discussion on how the results differed between the two paradigms, but I am guessing that there are usually higher retention rates and more requirements for those health workers who are paid. After all, they are apparently making a very nice salary (~USD$70 a month, which is quite good in developing regions).