October 14, 2013
Heuristic Evaluation Assignment
Due: Tuesday, November 12th, 2013 before Class
Objective: In this assignment, you will perform a heuristic evaluation of another group’s Balsamiq prototype. You will start by conducting a “think aloud” exercise. Next, you will conduct a heuristic evaluation, and integrate your results with other evaluators to generate an evaluation report. Finally, you will conduct another 1-2 “think aloud” exercises with other prospective users.
What to do:
- With your group, write down 2-3 tasks to be completed by evaluators using your interface.
- Look at the front whiteboard to see if you have been assigned to evaluate a project. If you find your name, prepare to be a test user by walking over and introducing yourself to the project you will be evaluating. If not, prepare to conduct a “Think-Aloud” exercise using your prototype. You will be an evaluator in the second half of the class.
- After introducing yourself as a test user, please step through the tasks as instructed by the demonstrator using the prototype. Utilize the “Think-Aloud” protocol. Demonstrators should take detailed notes of the pilot user’s observations. Do it quickly so that you have time for the heuristic evaluation. You will conduct additional think-aloud experiments with other users before the next assignment.
- Next, conduct a heuristic evaluation. You can use Nielsen’s heuristics, plus any others you have added. The demonstrator should begin by again outlining a task or scenario. After that, the evaluators are free to ask questions. When the evaluators are finished assessing one scenario, the demonstrators should restart the conversation by demonstrating the next task or scenario. Each evaluator should work independently, making sure to take detailed notes. Any evaluations that cannot be completed during class time should be done after class.
- Evaluators should use the following format to keep track of your observations: HE.xls. Each observation requires a numeric index, a heuristic that was violated, a location on the user interface, a description of the problem, a severity rating, and a possible fix (use the 0-4 scale presented in lecture for severity ratings). Each evaluator should aim to document at least 10 usability problems, covering 5 distinct heuristics. Some usability problems may not violate an established heuristic – in that case you can label them “misc” for miscellaneous. However, please make sure that an existing heuristic does not cover what you are describing.
- The next step is to combine your report with the other evaluators in your group. Please do this after class. Plan a time to meet with the other evaluators to consolidate your individual evaluation reports. The group whose project you evaluated need not be present, but you will probably find it helpful to have a version of their prototype available for reference. You will create one master Excel spreadsheet that contains each unique problem found. Remember – a duplicate is a violation of the same heuristic, in the same location. A violation of a different heuristic in the same location is considered distinct. For each unique problem found, you will need to discuss amongst yourselves to decide on a final severity rating and a possible fix.
- In collaboration with the other evaluators, write a short (less then one page) executive summary that outlines the major problems that you found, and possible solutions for the same, prioritizing those with the highest severity ratings.
- After class, you should conduct another “Think-Aloud” exercise with your own project, this time with 2-3 real prospective users. Feel free to use the same tasks you did in class, or come up with new ones. Document your observations, and any feedback and recommendations that you receive. You will find it useful to do this before your next prototype assignment.
What to turn in:
The preferred method of turn-in is a PDF document, including each of the following components. To avoid a late penalty, e-mail a link to your group’s submission to the professor, TA and the other group before class on Tuesday, November 12th:
- Cover sheet including yourself and the other evaluators’ names, and the project(s) you evaluated.
- The final consolidated evaluation report, using the following Excel format: HE.xls.
- A short executive summary that outlines the major problems you found, and possible solutions for the same, prioritizing those with the highest severity ratings [1 page].
Please also share your evaluation report with the other group(s) before class next Tuesday. Please note that the other group is depending on your timely feedback. For this reason, late submissions will not receive credit on this assignment.
Please contact the professor or the class TA if you have any questions with this assignment.