August 27, 2010

Usability Testing

See also Remote Usability Testing; Mobile Usability Testing

Readings

updated 9/20

Kuniavsky, ch. 10

Rubin, Jeffrey, Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Wiley, 1994

(This book is worth buying — a classic — get the new edition, 2008.)

New edition has sample forms online — note in particular:

Chapter 05: Test Plans and Designs
A sample test plan including sections for overall objectives, research questions, location and setup, recruiting participants, and methodology.		 17.41 KB HTTP | FTP
Chapter 08: Session Forms
Pre-Test Forms: Background Questionnaire		 9.69 KB HTTP | FTP
Chapter 08: Session Forms
Post-Test Forms: Closing Likert Ratings		 61.39 KB HTTP | FTP
Chapter 08: Session Forms
Post-Test Forms: Closing Semantics Deferentials		 8.07 KB HTTP | FTP
Chapter 08: Session Forms
Session Script: Example A Session Script		 12.78 KB HTTP | FTP
Chapter 08: Session Forms
Pre-Test Forms: Guidelines for Observers		 7.56 KB HTTP | FTP

Joseph S. Dumas and Beth A. Loring, Moderating Usability Tests: Principles and Practices for Interacting, Elsevier, 2008. The table of contents for this book has an excellent set of rules for interacting with participants in tests, which apply pretty well to interviews, too.  The table of contents has other good advice, too.  Available as an e-book and as a Kindle book.

Nielsen/Norman group User Testing workshop: list of topics tells a lot about what you need to know.

Optional from Rubin— but highly recommended:

From Usability.gov: Good templates for a variety of documents for usability testing: test plan, tester scripts, consent forms, note-taker’s guide.  As part of a page of usability templates of all kinds.

Reporting

Kuniavsky, ch. 17

Templates from usability.gov — scroll down to Reports.

Common Industry Format — is supposed to be a standard but no one uses it.

Tools

Morae: Not cheap but widely used

Ethnio

Report comparing available tools: Robin Good, Website Usability Testing: Guide To The Best Professional Usability Testing Tools And Services.  I don’t know how good this assessment is — I’m suspicious looks OK — may be most valuable for the list of tools for which you should seek other reviews. None are cheap, but some may have free trials.

Usabilla: micro usability tests

A list and description of usability testing tools: 24 Usability Testing Tools from W Craig Tomlin — don’t know the author but this list is useful.

Examples

Really good example — Wikipedia study by Bolt | Peters.  They use the Wikipedia format for report to/about Wikipedia. Includes links to highlight videos.

* Filed by Nancy Van House at 11:30 am under Methods,Readings,Usability Testing
Comments Off on Usability Testing

Usability Measurement


Quantitative approaches to usability require that you define your measures and measurement and analysis methods.

The Table of Contents of Tullis & Albert, Measuring User Experience, is helpful for getting a sense of the kinds of measures and measurement issues relevant to UX.

The web page accompanying  Tullis & Albert, Measuring User Experience, links to a wealth of resources.

These PPT slides by Tullis provide an excellent overview of measurement issues, especially statistical tests and reporting. Too many people in user experience research don’t know statistics!

Good example: Wikimedia Metrics

* Filed by Nancy Van House at 11:15 am under Measurement,Readings
Comments Off on Usability Measurement

Card Sorting

Updated 10/15/10

Not a fancy method but used a lot — some say that the post-it is the primary tool of UX research.

Kuniavsky p. 192-201

Mostly a list of tools from Tom Tullis

Card Sorting from Usability Body of Knowledge: good overview, extensive references

Affinity Diagramming – short and simple


* Filed by Nancy Van House at 11:03 am under Card Sorting,Methods
Comments Off on Card Sorting

Ethics

* Filed by Nancy Van House at 10:30 am under Ethics,Readings,Users
Comments Off on Ethics

Working with Users

This may seem like a lot of reading, but some of it is redundant.  However, each reading adds something others don’t. Given the importance of working with users for this kind of research, there’s a lot that you need to consider.

Kuniavsky, ch. 7

For pwd protected readings:
user is214
pwd is214

Courage & Baxter, Understanding Your Users: A Practical Guide to User Requirements Methods, Tools, and Techniques

Hackos and Redish, User and Task Analysis for Interface Design

    * Filed by Nancy Van House at 10:18 am under Readings,Users
    Comments Off on Working with Users

    Evaluation by Experts: Heuristics, Guidelines, Walkthroughs

    Heuristics

    Heuristic Evaluation from the Usability Body of Knowledge

    What You Really Get From a Heuristic Evaluation, Dana Chisnell, UX Magazine, 2010

    Jakob Nielsen’s 10 heuristics, http://www.useit.com/papers/heuristic/heuristic_list.html. These are the classics everyone immediately thinks of when you talk about heuristics.

    Mozilla heuristics:  Quantifying Usability, Alex Faaborg in UX Magazine:

    These are the UX principles currently being used by developers working onFirefox and other projects in the Mozilla community. We are currently utilizing Bugzilla’s keyword functionality, similar to how current bugs can be flagged as violating implementation level heuristics, like data loss. These principles can be added to any bug tracking that allows bugs to be tagged.

    Guidelines

    Guidelines are used like checklists as, well, guidelines for design. They can be used for evaluation by assessing a product against relevant guidelines.  The best approach is to find or develop guidelines specific to a product domain; the more specific the better.

    Guidelines can be developed by looking at good examples — for example, e-commerce guidelines could be developed by examining sites like Amazon.

    Sample Guidelines

    Apple’s iPad User Experience Guidelines

    UX Magazine’s summary of iPad User Experience Guidelines

    Blackberry Smartphones UI Guidelines

    Susan Weinschenk, The Psychologist’s View of UX Design, UX Magazine May 19th 2010. Not intended as guidelines, but they work.

    Usability.gov research-baseed guidelines for web design.

    Good: these are based on large-scale research projects.   A PDF manual of the guidelines can be downloaded from this site.  Each guideline is evaluated for how well it is supported by research data; can be highly valuable for convincing clients of their legitimacy.

    Not so good: the manual violates what I consider the most basic guideline for such material: no date!  Most recent material referenced: 2006.

    Web Accessibility Initiative  (added 9/6/10)

    Guidelines for various kinds of content, tools, and applications

    Walkthroughs

    Pluralistic Usability Walkthroughs

    * Filed by Nancy Van House at 10:01 am under accessibility,Guidelines,heuristics,Methods,Readings
    Comments Off on Evaluation by Experts: Heuristics, Guidelines, Walkthroughs

    Introduction (readings)

    Introduction to UX/UX Research

    (1) Kuniavsky, ch. 3

    (2) Troll through the webpages below for overviews of each topic.

    User Experience

    Principles for Usable Design from Usability Body of Knowledge

    Nielsen/Norman Group User Experience

    Nielsen, Usability 101

    A fairly good collection of definitions  of UX  and UX design from a consulting firm

    Usability Professionals’ Association Poster The text on this page is a pretty good overview of a possible design process.

    Top Ten Myths About Usability, Tom Tullis – very short PPT but true; links to PDF of all of his slides for this presentation.  His book is Measuring the User Experience so this presentation and his website are useful for getting a sense of quantitative approaches to user experience.

    User Experience Research

    http://adaptivepath.com/services/ A list of services and products from a local UX consulting firm.

    User experience research methods

    Good overview based on the major dimensions on which methods can be categorized.

    http://www.usability.gov/ Step through much of this site to get an overview.

    http://www.usability.gov/methods/index.html Look at this page in particular. Doesn’t exactly match what we’ll be doing, but useful. Click on the links to get a general sense of what these methods cover.

    http://www.usabilityfirst.com/methods/index.txl Another overview of common methods — not the same set asusability.gov

    http://www.usabilitynet.org/tools/methods.htm Note that this is an interactive chart that lets you specify three different sets of constraints.

    http://www.theusabilityteam.com/overview.asp And another overview.

    http://www.infodesign.com.au/usabilityresources

    * Filed by Nancy Van House at 9:59 am under Introductory,Readings
    Comments Off on Introduction (readings)

    August 16, 2010

    Student Information Form

    Please fill out, print, and bring to first class.

    Name:

    iSchool student?

    ___Class of 2011

    ___Class of 2012

    ___PhD

    Other: Dept/major, degree objective, expected year:

    Relevant experience: work and/or education:

    Why are you taking this class?

    Anything else you think it would be useful for me to know?

    * Filed by Nancy Van House at 5:47 pm under logistics
    Comments Off on Student Information Form

    Syllabus is going up

    See link under “pages.”

    * Filed by Nancy Van House at 2:28 pm under Readings
    Comments Off on Syllabus is going up

    Room changed

    From 205 South Hall to 110 South Hall.

    * Filed by Nancy Van House at 11:09 am under logistics
    Comments Off on Room changed

    Next Page »