How to Use This Site

To avoid confusion, this is what I suggest:

1. Start with the syllabus, and follow links from the table cell for the class you’re interested in.

(Those pages are dated “posts,” not “pages” (in WordPress’ terminology) because WordPress allows categories to be attached to posts,  but not pages.)

2. If you’re looking for a specific topic, use the categories along the right of the home screen.

3. To be updated on changes, subscribe to the RSS feed  (at the bottom of the page on the right).

About Readings and This Site

In case you haven’t figured this out already, I’m continually updating these pages. Because this field is continually developing, I update the readings etc. each time I teach this course, so I’m continually  working on this syllabus.  Often when I’m looking for one thing I find resources of interest for other topics for this class.  And new resources are always appearing.

Generally, for any topic, read Kuniavsky first, then whatever resources are listed as readings.  Because of a lot of this course is about methods and tools, and people in this field present their work in a variety of ways, not all the important resources are readings; e.g., some are videos.   Depending on your project (and future activities), you’ll need more depth in some areas than what we are covering in class and the added resources will be useful.   So these pages are resources for the class sessions on a topic, but also for your projects and for your future work.    (These pages will stay up indefinitely.)

I use the “categories” to indicate what topic a page applies to, so for any topic, always check what’s listed under the relevant categories.  For me, putting the resources up and linking them to specific days are two different activities, so there may be resources that aren’t currently linked on the syllabus.

Usability Testing

See also Remote Usability Testing; Mobile Usability Testing

Readings

updated 9/20

Kuniavsky, ch. 10

Rubin, Jeffrey, Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Wiley, 1994

(This book is worth buying — a classic — get the new edition, 2008.)

New edition has sample forms online — note in particular:

Chapter 05: Test Plans and Designs
A sample test plan including sections for overall objectives, research questions, location and setup, recruiting participants, and methodology.
17.41 KB HTTPFTP
Chapter 08: Session Forms
Pre-Test Forms: Background Questionnaire
9.69 KB HTTPFTP
Chapter 08: Session Forms
Post-Test Forms: Closing Likert Ratings
61.39 KB HTTPFTP
Chapter 08: Session Forms
Post-Test Forms: Closing Semantics Deferentials
8.07 KB HTTPFTP
Chapter 08: Session Forms
Session Script: Example A Session Script
12.78 KB HTTPFTP
Chapter 08: Session Forms
Pre-Test Forms: Guidelines for Observers
7.56 KB HTTPFTP

Joseph S. Dumas and Beth A. LoringModerating Usability Tests: Principles and Practices for Interacting, Elsevier, 2008. The table of contents for this book has an excellent set of rules for interacting with participants in tests, which apply pretty well to interviews, too.  The table of contents has other good advice, too.  Available as an e-book and as a Kindle book.

Nielsen/Norman group User Testing workshop: list of topics tells a lot about what you need to know.

Optional from Rubin— but highly recommended:

From Usability.gov: Good templates for a variety of documents for usability testing: test plan, tester scripts, consent forms, note-taker’s guide.  As part of a page of usability templates of all kinds.

Reporting

Kuniavsky, ch. 17

Templates from usability.gov — scroll down to Reports.

Common Industry Format — is supposed to be a standard but no one uses it.

Tools

Morae: Not cheap but widely used

Ethnio

Report comparing available tools: Robin Good, Website Usability Testing: Guide To The Best Professional Usability Testing Tools And Services.  I don’t know how good this assessment is — I’m suspicious looks OK — may be most valuable for the list of tools for which you should seek other reviews. None are cheap, but some may have free trials.

Usabilla: micro usability tests

A list and description of usability testing tools: 24 Usability Testing Tools from W Craig Tomlin — don’t know the author but this list is useful.

Examples

Really good example — Wikipedia study by Bolt | Peters.  They use the Wikipedia format for report to/about Wikipedia. Includes links to highlight videos.

Usability Measurement


Quantitative approaches to usability require that you define your measures and measurement and analysis methods.

The Table of Contents of Tullis & Albert, Measuring User Experience, is helpful for getting a sense of the kinds of measures and measurement issues relevant to UX.

The web page accompanying  Tullis & Albert, Measuring User Experience, links to a wealth of resources.

These PPT slides by Tullis provide an excellent overview of measurement issues, especially statistical tests and reporting. Too many people in user experience research don’t know statistics!

Good example: Wikimedia Metrics

Ethics

Working with Users

This may seem like a lot of reading, but some of it is redundant.  However, each reading adds something others don’t. Given the importance of working with users for this kind of research, there’s a lot that you need to consider.

Kuniavsky, ch. 7

For pwd protected readings:
user is214
pwd is214

Courage & Baxter, Understanding Your Users: A Practical Guide to User Requirements Methods, Tools, and Techniques

Hackos and Redish, User and Task Analysis for Interface Design

    Evaluation by Experts: Heuristics, Guidelines, Walkthroughs

    Heuristics

    Heuristic Evaluation from the Usability Body of Knowledge

    What You Really Get From a Heuristic EvaluationDana Chisnell, UX Magazine, 2010

    Jakob Nielsen’s 10 heuristics, http://www.useit.com/papers/heuristic/heuristic_list.html. These are the classics everyone immediately thinks of when you talk about heuristics.

    Mozilla heuristics:  Quantifying Usability, Alex Faaborg in UX Magazine:

    These are the UX principles currently being used by developers working onFirefox and other projects in the Mozilla community. We are currently utilizing Bugzilla’s keyword functionality, similar to how current bugs can be flagged as violating implementation level heuristics, like data loss. These principles can be added to any bug tracking that allows bugs to be tagged.

    Guidelines

    Guidelines are used like checklists as, well, guidelines for design. They can be used for evaluation by assessing a product against relevant guidelines.  The best approach is to find or develop guidelines specific to a product domain; the more specific the better.

    Guidelines can be developed by looking at good examples — for example, e-commerce guidelines could be developed by examining sites like Amazon.

    Sample Guidelines

    Apple’s iPad User Experience Guidelines

    UX Magazine’s summary of iPad User Experience Guidelines

    Blackberry Smartphones UI Guidelines

    Susan Weinschenk, The Psychologist’s View of UX Design, UX Magazine May 19th 2010. Not intended as guidelines, but they work.

    Usability.gov research-baseed guidelines for web design.

    Good: these are based on large-scale research projects.   A PDF manual of the guidelines can be downloaded from this site.  Each guideline is evaluated for how well it is supported by research data; can be highly valuable for convincing clients of their legitimacy.

    Not so good: the manual violates what I consider the most basic guideline for such material: no date!  Most recent material referenced: 2006.

    Web Accessibility Initiative  (added 9/6/10)

    Guidelines for various kinds of content, tools, and applications

    Walkthroughs

    Pluralistic Usability Walkthroughs

    Introduction (readings)

    Introduction to UX/UX Research


    (1) Kuniavsky, ch. 3

    (2) Troll through the webpages below for overviews of each topic.

    User Experience

    Principles for Usable Design from Usability Body of Knowledge

    Nielsen/Norman Group User Experience

    Nielsen, Usability 101

    A fairly good collection of definitions  of UX  and UX design from a consulting firm

    Usability Professionals’ Association Poster The text on this page is a pretty good overview of a possible design process.

    Top Ten Myths About Usability, Tom Tullis – very short PPT but true; links to PDF of all of his slides for this presentation.  His book is Measuring the User Experience so this presentation and his website are useful for getting a sense of quantitative approaches to user experience.

    User Experience Research

    http://adaptivepath.com/services/ A list of services and products from a local UX consulting firm.

    User experience research methods

    Good overview based on the major dimensions on which methods can be categorized.

    http://www.usability.gov/ Step through much of this site to get an overview.

    http://www.usability.gov/methods/index.html Look at this page in particular. Doesn’t exactly match what we’ll be doing, but useful. Click on the links to get a general sense of what these methods cover.

    http://www.usabilityfirst.com/methods/index.txl Another overview of common methods — not the same set asusability.gov

    http://www.usabilitynet.org/tools/methods.htm Note that this is an interactive chart that lets you specify three different sets of constraints.

    http://www.theusabilityteam.com/overview.asp And another overview.

    http://www.infodesign.com.au/usabilityresources

    Syllabus is going up

    See link under “pages.”

    Textbook!

    Mike Kuniavsky, Observing the User Experience: A Practitioner’s Guide to User Research, Morgan Kaufmann. Please buy. Not in ASUC bookstore.