Syllabus

Last updated 4/12/10  2:30 pm


Regarding links to commercial sites.

1a. Jan. 19. Intro to the course

Please fill out this student info form and bring to first class.

What is User Experience Research? Usability? — What does this course cover? — Course requirements

Assignment 1 This assignment will not be graded; I’ll simply record it as received (or not).

1b. Jan. 21. Intro to User Experience Research

Introductions

Interactive Design and usability — Place in the design process — Why this isn’t just intuitive — Roots in research methods — Adjusting methods to user experience research and the reality of the design world

Methods overview – Goals and methods are intertwined – What do you need to know? What can you know? What resources are available, needed? How much time is available, needed ? – How do you design a project?

Some introductions and defintions:

Nielsen, Usability 101

Usability Professionals’ Association Poster The text on this page is a pretty good overview of a possible design process.

http://www.usability.gov/ Step through much of this site to get an overview.

http://www.usability.gov/methods/index.html Look at this page in particular. Doesn’t exactly match what we’ll be doing, but useful. Click on the links to get a general sense of what these methods cover.

http://www.usabilityfirst.com/methods/index.txl Another overview of common methods — not the same set as usability.gov

http://www.usabilitynet.org/tools/methods.htm Note that this is an interactive chart that lets you specify three different sets of constraints.

http://www.theusabilityteam.com/overview.asp And another overview.

http://www.infodesign.com.au/usabilityresources

2a. Tues, Jan 26. (Updated 1/25) Overview of Methods; if time, more about projects

Assignment 1 Due and discussed in class

User experience research projects:

(1) in professional practice

(21) for this class: What are the requirements? What makes a good project? How do you go about choosing a project? Creating project teams?

Issues in taking on projects (for this course and in professional practice) – Scoping the work – Dealing with clients

Final projects

2b. Jan 28. Expert Evaluation & Available Data: Competitor Analysis; Unobtrusive Measures

Competitive/Competitor Analysis

Some explanations and examples — note that these range from a simple chart to a lengthy report. For your projects, you’re more likely to do a chart:

Unobtrusive measures – Traces of activity

First discussion of measurement issues

3a. Feb 2. Using Available data

slides

Understanding and understanding useful data from existing sources

Example: Who’s using social networking?

Pew Internet and American Life project — read the full reports and look at the survey questions:

How diverse is Facebook? Facebook report Be sure you look at their methodology.

Come to class with at least one surprising or useful fact about who’s using social networking — or one myth debunked. Be ready to show us the data behind your finding.

3b. Feb. 4: Expert Evaluation, cont.: Heuristics

slides

Assignment 2 posted: heuristics There’s not going to be a heuristics assignment.

Muller, Matheson, Page, and Gallup, Participatory Heuristic Evaluation, Interactions, Sep.-Oct., 1998

Nielsen, Jakob. How to conduct a heuristic evaluationand Severity Ratings for Usability Problems

Nielsen’s 10 Usability Heuristics The classics; everyone knows these, and many still use them. Here’s Instone’s piece on how to apply them to the web — again, oldie but goodie.

Seven usability guidelines for websites on mobile devices An example of heuristics for a special situation; in this case, for websites on mobile phones. We’re not saying these are the best; just good examples.

First principles, AskTog

International standards for HCI and usability

Optional, but recommended

4a. Tues., Feb. 9. Intro to Working with Users

slides

Identifying – Classifying – Selecting – Recruiting – Caring for. Ethics. Beginning discussion of sampling.

Courage & Baxter, Understanding Your Users: A Practical Guide to User Requirements Methods, Tools, and Techniques

Hackos and Redish, User and Task Analysis for Interface Design

Skim these — useful resources for ethics of working with users:

Assignment 2 due: heuristics

4b. Thurs, Feb. 11. Usability testing

slides
Rubin, Jeffrey, Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Wiley, 1994 (This book is worth buying — a classic.)

Optional — if and when you conduct usability tests, read the following:

Assignment 2: Usability Test


5a. Tues, Feb 16. No class meeting.

5b. Feb. 18. Proposed project descriptions due.

We’ll spend class time discussing these. If we need to, we’ll take the entire class. If not, we’ll start talking about field work.

Project descriptions

6a. Feb. 23: Discussion

Assignment 2: Usability Test due

Brief reports from people who went to User Experience Friday at Bolt Peters last Friday

Discussion of usability testing assignment

Finish discussion of projects

Assignment 3 posted: field observation

6b. Feb. 25. Guest speaker: Cyd Harrell, Bolt | Peters

She’ll talk about their methods but also the state of the UX research business these days.

Look at their website http://boltpeters.com/

http://boltpeters.com/services/live-remote-research/ — this is an overview of their methods

Look at http://www.rosenfeldmedia.com/books/remote-research/info/table_of_contents/ — TofC for the book they just published — think about whether you have questions, specific issues you want Cyd to address.

http://www.boxesandarrows.com/view/researching-video

http://entrepreneur.venturebeat.com/2010/02/11/bolt-peters-remote-research/

7a . Tues., March 2. Collecting Data from  Users: Field Studies

Slides

We’ll discuss these next two readings as case studies in class. Read these with attention to their methods .

Kirk, David S., Shahram Izadi, Abigail Sellen, Stuart Taylor, Richard Banks, and Otmar Hilliges. 2009. Opening up the family archive. In Proceedings of the 2010 ACM conference on Computer supported cooperative work. Savannah, Georgia, USA: ACM.

A good example of a field study implementing a new technology.  (If you don’t access from campus, you have to use Library Proxy.)

Li, Jane, Toni Robertson, Susan Hansen, Tim Mansfield, and Jesper Kjeldskov. 2008. Multidisciplinary medical team meetings: a field study of collaboration in health care. In Proceedings of the 20th Australasian Conference on Computer-Human Interaction: Designing for Habitus and Habitat. Cairns, Australia: ACM.

Background reading for this and the next several classes:

Blomberg, J., Burrell, M., Guest, G. An ethnographic approach to design. In Jacko J. A., Sears A. (eds.). The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications. Lawrence Erlbaum Associates, Inc.: Mahwah, New Jersey, 2003.   Long, but classic overview of much of what we’re talking about.

David R. Millen. Rapid ethnography: time deepening strategies for HCI field research. Conference proceedings on Designing interactive systems : processes, practices, methods, and techniques. ACM, 2000.

Hugh Beyer and Karen Holtzblatt, Contextual Design, San Francisco; Morgan-Kaufman, 1998. Chapter 1 – Introduction

We are not going to study contextual inquiry/contextual design per se, but it’s a method people talk about.  The term “contextual inquiry” has become a more generic term for field-based observation, rather than a label for Beyer and Holtzblatt’s method.

7b & 8a. Thurs, March 4 & Tues, March 9. Interviewing and Related Interactions with Users

slides

Assignment 3 field observation due March 4.

Assignment 4 posted: interviewing

Who to interview – how to decide what to ask – formulating questions – interview logistics

Conducting an interview

We will practice interviewing skills in class.

Robert S. Weiss, Learning from Strangers – The Art and Method of Qualitative Interview Studies, New York: The Free Press, 1994
Chapter 3 – Preparation for Interviewing
Chapter 4 – Interviewing – Part 1
Chapter 4 – Interviewing – Part 2 – Examples of Interviewing
Chapter 5 – Issues in Interviewing

Example of analyzing and reporting interview data:    Shamma, David A., Elizabeth F. Churchill, Nikhil Bobb, and Matt Fukuda. 2009. Spinning online: a case study of internet broadcasting by DJs. In Proceedings of the fourth international conference on Communities and technologies. University Park, PA, USA: ACM.

8b & 9a. March 11 & 16. Surveys

Slides

Compared to interviews – complementing interviews – survey questions – survey sampling – survey design and implementation for user experience research – Capturing data at time of interaction

Key issues include the process of designing a survey; the design of the survey itself (contents, question wording, question ordering); and survey administration (including sampling).  Paper surveys are increasingly rare in the kinds of domains we’re talking about, but online and paper surveys are fundamentally similar.

Introduction to surveys

There are lots of resources (online and published) on survey research generally. Here are some suggestions.  Most emphasize some aspects more than others — at least skim all of them, so that you can pick up from each what it covers more thoroughly than the others do:

Research Methods Knowledge Base: Survey Research.  You’ll see lots of similarities to interviews, but some important differences.

University of Reading Statistical Services Center, Guidelines for Planning Effective Surveys. This one has less on question construction than the above but more on other surveying issues.

Statnotes, Survey Research.  Especially good on design considerations and item bias.  Also good FAQs.

And usabilty

Questionnaires in Usability Engineering: FAQs (3rd ed)

Sources of questions and examples of questions (and data analysis)

Pew Internet and American Life project publishes their surveys. If they have written a question that works for you, use it!  Also, by seeing how carefully they word questions and answers you get a sense of the kinds of problems in question wording that they had to deal with (which may help you think about your questions).

A recent example of their work –  Online News Survey

Analyzing survey results:

University of Reading Statistical Services Center,  Approaches to the Analysis of Survey Data

Free Survey Software

There are many.  Be sure you check their limits on free or trial use — not just # of questions and respondents, but what sort of data analysis they do, and/or whether you can download data to Excel.

surveymonkey.com

zoomerang.com

polldaddy.com

Assignment 4 due March 16 : Interviewing

Assignment 5 posted : Surveys

9b. March 18. Other Methods of Collecting Data with, by, and about Users

Slides

This discussion will likely run over into the following class.  You need to know about these methods; read these articles for what they will tell you about these methods.

Diary Studies – participants use various media to collect real-time data

Carter, S. and Mankoff, J. 2005. When participants do the capturing: the role of media in diary studies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA, April 02 – 07, 2005). CHI ’05. ACM, New York, NY, 899-908.  Useful for their discussion of using media other than paper, with references to examples.

This one is useful for their method of having people use SMS to record diary entries: Sohn, T., Li, K. A., Griswold, W. G., and Hollan, J. D. 2008. A diary study of mobile information needs. InProceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy, April 05 – 10, 2008). CHI ’08. ACM, New York, NY, 433-442.

It’s quite difficult to find examples of HCI-related diary study data collection instruments.

Example: very elaborate diary study: USPS annual Household Diary Study: Mail Use and Attitudes in FY 2008 — see Appendix C2; Dairy Package.

Example: simple diary study.  Arbitron Radio Listening Diary (and why they use paper)

Experience sampling – Users report their in-the-moment actions in response to some kind of alert.

Sunny Consolvo, Miriam Walker Using the Experience Sampling Method to Evaluate Ubicomp Applications Pervasive Computing April-June 2003 (Vol. 2, No. 2) pp. 24-31.

Cultural probes

This piece is a classic: Gaver, B., Dunne, T., and Pacenti, E. 1999. Design: Cultural probes. interactions 6, 1 (Jan. 1999), 21-29.


March 23 & 25. Spring Break

10a. March 30.  Focus Groups

Slides

Many resources online.

10b. April 1: Analyzing results from field studies: qualitative data

Affinity Diagrams.  Read http://www.usabilitybok.org/methods/affinity-diagram and then skim through these — all short; repetitive, but often one will add something that the others didn’t cover:

http://www.balancedscorecard.org/Portals/0/PDF/affinity.pdf

http://www.usabilitynet.org/tools/affinity.htm

http://www.infodesign.com.au/usabilityresources/affinitydiagramming

http://www.hiser.com.au/articles/dtech_affinity_diagrams.html

http://www.mindtools.com/pages/article/newTMC_86.htm

http://www.usabilitybok.org/methods/affinity-diagram

http://www.infodesign.com.au/ftp/AffinityDiagramming.pdf

http://www.uie.com/articles/kj_technique/

11a. April 6. Personas and Scenarios; Start Quantitative Data

Good, very short intro to scenarios: What is a scenario? from Information & Design.

Good intro to scenarios in HCI: The Blind Men and the Elephant: Views of Scenario-Based System Design, Interactions, Nov.-Dec. 2004

Good typology of scenarios: Cooper, A. (1999). The inmates are running the asylum. Indianapolis: Sams. ch 11: Designing for people.  userid & pwd: is214

Pruitt, John and Grudin, Jonathan. Personas: Practice and Theory. 2002.

Action figure personas (really!): Nieters, J. E., Ivaturi, S., and Ahmed, I. 2007. Making personas memorable. In CHI ’07 Extended Abstracts on Human Factors in Computing Systems (San Jose, CA, USA, April 28 – May 03, 2007). CHI ’07. ACM, New York, NY, 1817-1824.

Personas pro and con:

Pro: Norman, D. A. 2008. THE WAY I SEE IT
Filling much-needed holes. interactions 15, 1 (Jan. 2008), 70-71.

Con: Portigal, S. 2008. TRUE TALES
Persona non grata. interactions 15, 1 (Jan. 2008), 72-73.

Are scenarios the same as use cases?

No: JISC, Scenarios and Use Cases

Alistair Cockburn, Use Case Fundamentals

11b. April 8. Analyzing and Presenting Findings: Quantitative data

Read about variable types; frequency distribtuions; measures of central tendency; and measures of spread.  Any good basic stats text will cover these.  Here’s a good online source:

Statistics: Power from Data, from Statistics Canada

12a. April 13.  Analyzing and Presenting Findings: Quantitative data (cont)

Cool visualizations are fun but often not as informative as they should be for a knowledgeable audience. Furthermore, you need to know about basic types before getting involved in more complex visualizations: you need to know the underlying principles of good data visualization.

Read any good statistics text on basic methods of visualizing data. A good source on presenting quantitative data: University of Reading Statistical Services Center, Informative Presentation of Tables, Graphs, and Statistics.

Statistics: Power from Data, from Statistics Canada: Graph types

12b. April 15.  Presenting Findings

AskTog, 2001: How to Deliver a Report Without Getting Lynched

Short but useful:

Usability Reporting Tips by Chauncey Wilson

Writing usability reports

Jarrrett, Caroline. Better Reports: How to Communicate the Results of Usability Testing

Analyzing and Reporting Usability Data

Sample usability reports

This page has links to dozens of reports!  To evaluate them, put yourself in the position of the client: does this tell you what you need to know?  Too little? Too much? Not the right info?  http://www.pdfgeni.com/book/usability-report-pdf.html

From Usability.gov —  http://www.usability.gov/templates/index.html#Usability — their other templates are useful, too.

Test report templates for writing the usability test report.

Good advice on powerpoint presentations:  Garr Reynolds, Presentation Tips

13a. Tues, April 20: Accessibility (date changed)

Guest speaker: Lucy Greco, University of California, Berkeley Disabled Students’ Program, Assistive Technologies Specialist http://attlc.berkeley.edu

http://webaccess.berkeley.edu/

Just Ask: Integrating Accessibility Throughout Design –“Helps designers and developers create websites, software, hardware, and consumer products that are accessible to people with disabilities, provide a better user experience for all, and realize the additional benefits of accessibility.”

http://www.uiaccess.com/accessucd/ut_report.html

W3C Accessbility

W3C introduction to web acccessibility

W3C Quick tips

WAI guidelines and techniques

Section 508 standards

National Institute on Aging and the National Library of Medicine, Making Your Web Site Senior Friendly: a Checklist

13b. Thurs. April 22. Presenting Findings

Guest speaker: Liz Goodman


14a. Tues. April 27. UX in the Large Organization

guest speaker : Andrea Moed, senior insights researcher, advertising products group, Yahoo! (and iSchool alum)

Wilson, C. E. 2007. Please listen to me!: or, how can usability practitioners be more persuasive?. interactions 14, 2 (Mar. 2007), 44-ff. DOI= http://doi.acm.org/10.1145/1229863.1229889

14b. April 29: Project presentations

Thermostat group

15.  May 4 & 6 – Project presentations

Final papers due: Tues. May 11, 5 pm