Syllabus
Last updated 4/12/10 Â 2:30 pm
Regarding links to commercial sites.
1a. Jan. 19. Intro to the course
Please fill out this student info form and bring to first class.
What is User Experience Research? Usability? — What does this course cover? — Course requirements
Assignment 1 This assignment will not be graded; I’ll simply record it as received (or not).
1b. Jan. 21. Intro to User Experience Research
Introductions
Interactive Design and usability — Place in the design process — Why this isn’t just intuitive — Roots in research methods — Adjusting methods to user experience research and the reality of the design world
Methods overview – Goals and methods are intertwined – What do you need to know? What can you know? What resources are available, needed? How much time is available, needed ? – How do you design a project?
Some introductions and defintions:
Nielsen, Usability 101
Usability Professionals’ Association Poster The text on this page is a pretty good overview of a possible design process.
http://www.usability.gov/ Step through much of this site to get an overview.
http://www.usability.gov/methods/index.html Look at this page in particular. Doesn’t exactly match what we’ll be doing, but useful. Click on the links to get a general sense of what these methods cover.
http://www.usabilityfirst.com/methods/index.txl Another overview of common methods — not the same set as usability.gov
http://www.usabilitynet.org/tools/methods.htm Note that this is an interactive chart that lets you specify three different sets of constraints.
http://www.theusabilityteam.com/overview.asp And another overview.
http://www.infodesign.com.au/usabilityresources
2a. Tues, Jan 26. (Updated 1/25) Overview of Methods; if time, more about projects
Assignment 1 Due and discussed in class
User experience research projects:
(1) in professional practice
(21) for this class: What are the requirements? What makes a good project? How do you go about choosing a project? Creating project teams?
Issues in taking on projects (for this course and in professional practice) – Scoping the work – Dealing with clients
2b. Jan 28. Expert Evaluation & Available Data: Competitor Analysis; Unobtrusive Measures
Competitive/Competitor Analysis
Some explanations and examples — note that these range from a simple chart to a lengthy report. For your projects, you’re more likely to do a chart:
- Digital Web Magazine: Competitive analysis (good overview of the process; nice illustrations)
- Competitive Analysis: Understanding the Market Context (from Boxes and Arrows; not that this includes heuristics — next topic)
- Sample written report (no graphics)
- Sample report (few graphics)
Unobtrusive measures – Traces of activity
- e.g., web analytics
First discussion of measurement issues
3a. Feb 2. Using Available data
Understanding and understanding useful data from existing sources
Example: Who’s using social networking?
Pew Internet and American Life project — read the full reports and look at the survey questions:
How diverse is Facebook? Facebook report Be sure you look at their methodology.
Come to class with at least one surprising or useful fact about who’s using social networking — or one myth debunked. Be ready to show us the data behind your finding.
3b. Feb. 4: Expert Evaluation, cont.: Heuristics
Assignment 2 posted: heuristics There’s not going to be a heuristics assignment.
Muller, Matheson, Page, and Gallup, Participatory Heuristic Evaluation, Interactions, Sep.-Oct., 1998
Nielsen, Jakob. How to conduct a heuristic evaluationand Severity Ratings for Usability Problems
Nielsen’s 10 Usability Heuristics The classics; everyone knows these, and many still use them. Here’s Instone’s piece on how to apply them to the web — again, oldie but goodie.
Seven usability guidelines for websites on mobile devices An example of heuristics for a special situation; in this case, for websites on mobile phones. We’re not saying these are the best; just good examples.
First principles, AskTog
International standards for HCI and usability
Optional, but recommended
- Research-based Guidelines
- Usability Heuristics for Rich Internet Applications
- Heurstic Evaluation of Ambient Displays
4a. Tues., Feb. 9. Intro to Working with Users
Identifying – Classifying – Selecting – Recruiting – Caring for. Ethics. Beginning discussion of sampling.
Courage & Baxter, Understanding Your Users: A Practical Guide to User Requirements Methods, Tools, and Techniques
Hackos and Redish, User and Task Analysis for Interface Design
Skim these — useful resources for ethics of working with users:
- Ethical Principles of Psychologists and Code of Conduct 2002; see especially sections on Privacy and Confidentiality.
- STC usability SIG on ethics especially the UPA Code of Professional Conduct (goes beyond working with users to usability work in general)
- UC Berkeley Committee for the Protection of Human Subjects – especially the guidelines on informed consent.
- International Visual Studies Association Code of Research Ethics and Guidelines Addresses issues related to multimedia recordings.
Assignment 2 due: heuristics
4b. Thurs, Feb. 11. Usability testing
slides
Rubin, Jeffrey, Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Wiley, 1994 (This book is worth buying — a classic.)
Optional — if and when you conduct usability tests, read the following:
5a. Tues, Feb 16. No class meeting.
5b. Feb. 18. Proposed project descriptions due.
We’ll spend class time discussing these. If we need to, we’ll take the entire class. If not, we’ll start talking about field work.
6a. Feb. 23: Discussion
Assignment 2: Usability Test due
Brief reports from people who went to User Experience Friday at Bolt Peters last Friday
Discussion of usability testing assignment
Finish discussion of projects
Assignment 3 posted: field observation
6b. Feb. 25. Guest speaker: Cyd Harrell, Bolt | Peters
She’ll talk about their methods but also the state of the UX research business these days.
Look at their website http://boltpeters.com/
http://boltpeters.com/services/live-remote-research/ — this is an overview of their methods
Look at http://www.rosenfeldmedia.com/books/remote-research/info/table_of_contents/ — TofC for the book they just published — think about whether you have questions, specific issues you want Cyd to address.
http://www.boxesandarrows.com/view/researching-video
http://entrepreneur.venturebeat.com/2010/02/11/bolt-peters-remote-research/
7a . Tues., March 2. Collecting Data from  Users: Field Studies
We’ll discuss these next two readings as case studies in class. Read these with attention to their methods .
Kirk, David S., Shahram Izadi, Abigail Sellen, Stuart Taylor, Richard Banks, and Otmar Hilliges. 2009. Opening up the family archive. In Proceedings of the 2010 ACM conference on Computer supported cooperative work. Savannah, Georgia, USA: ACM.
A good example of a field study implementing a new technology.  (If you don’t access from campus, you have to use Library Proxy.)
Li, Jane, Toni Robertson, Susan Hansen, Tim Mansfield, and Jesper Kjeldskov. 2008. Multidisciplinary medical team meetings: a field study of collaboration in health care. In Proceedings of the 20th Australasian Conference on Computer-Human Interaction: Designing for Habitus and Habitat. Cairns, Australia: ACM.
Background reading for this and the next several classes:
Blomberg, J., Burrell, M., Guest, G. An ethnographic approach to design. In Jacko J. A., Sears A. (eds.). The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications. Lawrence Erlbaum Associates, Inc.: Mahwah, New Jersey, 2003.  Long, but classic overview of much of what we’re talking about.
David R. Millen. Rapid ethnography: time deepening strategies for HCI field research. Conference proceedings on Designing interactive systems : processes, practices, methods, and techniques. ACM, 2000.
Hugh Beyer and Karen Holtzblatt, Contextual Design, San Francisco; Morgan-Kaufman, 1998. Chapter 1 – Introduction
We are not going to study contextual inquiry/contextual design per se, but it’s a method people talk about. Â The term “contextual inquiry” has become a more generic term for field-based observation, rather than a label for Beyer and Holtzblatt’s method.
7b & 8a. Thurs, March 4 & Tues, March 9. Interviewing and Related Interactions with Users
Assignment 3 field observation due March 4.
Assignment 4 posted: interviewing
Who to interview – how to decide what to ask – formulating questions – interview logistics
Conducting an interview
We will practice interviewing skills in class.
Robert S. Weiss, Learning from Strangers – The Art and Method of Qualitative Interview Studies, New York: The Free Press, 1994
Chapter 3 – Preparation for Interviewing
Chapter 4 – Interviewing – Part 1
Chapter 4 – Interviewing – Part 2 – Examples of Interviewing
Chapter 5 – Issues in Interviewing
Example of analyzing and reporting interview data: Â Â Shamma, David A., Elizabeth F. Churchill, Nikhil Bobb, and Matt Fukuda. 2009. Spinning online: a case study of internet broadcasting by DJs. In Proceedings of the fourth international conference on Communities and technologies. University Park, PA, USA: ACM.
8b & 9a. March 11 & 16. Surveys
Compared to interviews – complementing interviews – survey questions – survey sampling – survey design and implementation for user experience research – Capturing data at time of interaction
Key issues include the process of designing a survey; the design of the survey itself (contents, question wording, question ordering); and survey administration (including sampling). Â Paper surveys are increasingly rare in the kinds of domains we’re talking about, but online and paper surveys are fundamentally similar.
Introduction to surveys
There are lots of resources (online and published) on survey research generally. Here are some suggestions. Â Most emphasize some aspects more than others — at least skim all of them, so that you can pick up from each what it covers more thoroughly than the others do:
Research Methods Knowledge Base: Survey Research. Â You’ll see lots of similarities to interviews, but some important differences.
University of Reading Statistical Services Center, Guidelines for Planning Effective Surveys. This one has less on question construction than the above but more on other surveying issues.
Statnotes, Survey Research. Â Especially good on design considerations and item bias. Â Also good FAQs.
And usabilty
Questionnaires in Usability Engineering: FAQs (3rd ed)
Sources of questions and examples of questions (and data analysis)
Pew Internet and American Life project publishes their surveys. If they have written a question that works for you, use it! Â Also, by seeing how carefully they word questions and answers you get a sense of the kinds of problems in question wording that they had to deal with (which may help you think about your questions).
A recent example of their work – Â Online News Survey
Analyzing survey results:
University of Reading Statistical Services Center, Approaches to the Analysis of Survey Data
Free Survey Software
There are many. Â Be sure you check their limits on free or trial use — not just # of questions and respondents, but what sort of data analysis they do, and/or whether you can download data to Excel.
Assignment 4 due March 16 : Interviewing
Assignment 5 posted : Surveys
9b. March 18. Other Methods of Collecting Data with, by, and about Users
This discussion will likely run over into the following class. Â You need to know about these methods; read these articles for what they will tell you about these methods.
Diary Studies – participants use various media to collect real-time data
Carter, S. and Mankoff, J. 2005. When participants do the capturing: the role of media in diary studies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA, April 02 – 07, 2005). CHI ’05. ACM, New York, NY, 899-908.  Useful for their discussion of using media other than paper, with references to examples.
This one is useful for their method of having people use SMS to record diary entries: Sohn, T., Li, K. A., Griswold, W. G., and Hollan, J. D. 2008. A diary study of mobile information needs. InProceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy, April 05 – 10, 2008). CHI ’08. ACM, New York, NY, 433-442.
It’s quite difficult to find examples of HCI-related diary study data collection instruments.
Example: very elaborate diary study: USPS annual Household Diary Study: Mail Use and Attitudes in FY 2008 — see Appendix C2; Dairy Package.
Example: simple diary study. Â Arbitron Radio Listening Diary (and why they use paper)
Experience sampling – Users report their in-the-moment actions in response to some kind of alert.
Sunny Consolvo, Miriam Walker Using the Experience Sampling Method to Evaluate Ubicomp Applications Pervasive Computing April-June 2003 (Vol. 2, No. 2) pp. 24-31.
Cultural probes
This piece is a classic: Gaver, B., Dunne, T., and Pacenti, E. 1999. Design: Cultural probes. interactions 6, 1 (Jan. 1999), 21-29.
March 23 & 25. Spring Break
10a. March 30. Focus Groups
Many resources online.
10b. April 1: Analyzing results from field studies: qualitative data
Affinity Diagrams. Â Read http://www.usabilitybok.org/methods/affinity-diagram and then skim through these — all short; repetitive, but often one will add something that the others didn’t cover:
http://www.balancedscorecard.org/Portals/0/PDF/affinity.pdf
http://www.usabilitynet.org/tools/affinity.htm
http://www.infodesign.com.au/usabilityresources/affinitydiagramming
http://www.hiser.com.au/articles/dtech_affinity_diagrams.html
http://www.mindtools.com/pages/article/newTMC_86.htm
http://www.usabilitybok.org/methods/affinity-diagram
http://www.infodesign.com.au/ftp/AffinityDiagramming.pdf
http://www.uie.com/articles/kj_technique/
11a. April 6. Personas and Scenarios; Start Quantitative Data
Good, very short intro to scenarios: What is a scenario? from Information & Design.
Good intro to scenarios in HCI: The Blind Men and the Elephant: Views of Scenario-Based System Design, Interactions, Nov.-Dec. 2004
Good typology of scenarios: Cooper, A. (1999). The inmates are running the asylum. Indianapolis: Sams. ch 11: Designing for people. Â userid & pwd: is214
Pruitt, John and Grudin, Jonathan. Personas: Practice and Theory. 2002.
Action figure personas (really!): Nieters, J. E., Ivaturi, S., and Ahmed, I. 2007. Making personas memorable. In CHI ’07 Extended Abstracts on Human Factors in Computing Systems (San Jose, CA, USA, April 28 – May 03, 2007). CHI ’07. ACM, New York, NY, 1817-1824.
Personas pro and con:
Pro: Norman, D. A. 2008. THE WAY I SEE IT
Filling much-needed holes. interactions 15, 1 (Jan. 2008), 70-71.
Con: Portigal, S. 2008. TRUE TALES
Persona non grata. interactions 15, 1 (Jan. 2008), 72-73.
Are scenarios the same as use cases?
No: JISC, Scenarios and Use Cases
Alistair Cockburn, Use Case Fundamentals
11b. April 8. Analyzing and Presenting Findings: Quantitative data
Read about variable types; frequency distribtuions; measures of central tendency; and measures of spread. Â Any good basic stats text will cover these. Â Here’s a good online source:
Statistics: Power from Data, from Statistics Canada
12a. April 13. Â Analyzing and Presenting Findings: Quantitative data (cont)
Cool visualizations are fun but often not as informative as they should be for a knowledgeable audience. Furthermore, you need to know about basic types before getting involved in more complex visualizations: you need to know the underlying principles of good data visualization.
Read any good statistics text on basic methods of visualizing data. A good source on presenting quantitative data: University of Reading Statistical Services Center, Informative Presentation of Tables, Graphs, and Statistics.
Statistics: Power from Data, from Statistics Canada:Â Graph types
12b. April 15. Â Presenting Findings
AskTog, 2001: How to Deliver a Report Without Getting Lynched
Short but useful:
Usability Reporting Tips by Chauncey Wilson
Jarrrett, Caroline. Better Reports: How to Communicate the Results of Usability Testing
Analyzing and Reporting Usability Data
Sample usability reports
This page has links to dozens of reports! Â To evaluate them, put yourself in the position of the client: does this tell you what you need to know? Â Too little? Too much? Not the right info? Â http://www.pdfgeni.com/book/usability-report-pdf.html
From Usability.gov — Â http://www.usability.gov/templates/index.html#Usability — their other templates are useful, too.
Test report templates for writing the usability test report.
- Research style usability test report template (DOC – 144KB)
- Short (informal) usability test report template (DOC – 216KB)
- Long (formal) usability test report template (DOC – 536KB)
Good advice on powerpoint presentations: Â Garr Reynolds, Presentation Tips
13a. Tues, April 20: Accessibility (date changed)
Guest speaker: Lucy Greco, University of California, Berkeley Disabled Students’ Program, Assistive Technologies Specialist http://attlc.berkeley.edu
http://webaccess.berkeley.edu/
Just Ask: Integrating Accessibility Throughout Design –“Helps designers and developers create websites, software, hardware, and consumer products that are accessible to people with disabilities, provide a better user experience for all, and realize the additional benefits of accessibility.”
http://www.uiaccess.com/accessucd/ut_report.html
W3C Accessbility
W3C introduction to web acccessibility
National Institute on Aging and the National Library of Medicine, Making Your Web Site Senior Friendly: a Checklist
13b. Thurs. April 22. Presenting Findings
Guest speaker: Liz Goodman
14a. Tues. April 27. UX in the Large Organization
guest speaker : Andrea Moed, senior insights researcher, advertising products group, Yahoo! (and iSchool alum)
Wilson, C. E. 2007. Please listen to me!: or, how can usability practitioners be more persuasive?. interactions 14, 2 (Mar. 2007), 44-ff. DOI= http://doi.acm.org/10.1145/1229863.1229889
14b. April 29: Project presentations
Thermostat group