Choosing the right UX method or tool can be confusing. I recently attended a UPA webinar on doing this, and found the information pretty useful. The speaker,Bill Albert of Bentley University was really good, and easy to follow. He is the author of Measuring the User Experience. Thought I would share what I learned.
Bill pointed out from the beginning: Because most of us only have time to focus on a limited number of research and evaluation methods, it is important to choose the right methods for results from the start of a project. UX methods are dynamic, and therefore difficult to categorize and strictly define.
The main choice to make:
1) Do we need a Qualitative or Quantitative technique?
Qualitative focuses on “why”-and relies on observation, description, and insight.
Quantitative answers the “what” question- it should happen when you are almost done with your project, and utilizes a larger sample size.
2) Decide if you are going to focus on Behavior or Preferences.
Behavior- focuses on actual behavior. How learnable is the product? If you choose this, use usability lab studies, card sorting, or competive benchmarking.
Preferences-what do users like and dislike? This is highly subjective. What features/functions do they care most about? If you are using this method, use an advisory board, focus group, or in-depth interviews.
Best Qualitative methods:
1)Usability lab testing: single best method for improving the usability of a design during the design process
- Easy to administer- in a lab,or a remote test, moderated or unmoderated
- Observation is highly impactful
- Difficult to prioritize issues
- Not ecologically valid
- Great for Problem detection
2) Expert evaluation
- Going through at interaction level, looking at best practices, quick and cheap
- No participants
- Experienced evaluator can reliably identify a significant number of usability issues
- You have to be able to play the role of the user
- Biases can creep in
- Temptation to evaluate from a requirements perspective-
3) Field research
- Observing product in field
- May use surveys
- Most ecologically valid
- Tremendous source for innovation
- Tap into different set of users
- Different population than those that are willing to come to lab
- Not well suited to evaluate a specific set of tasks
- What do they actually use the product for?
4) Focus groups
- Focuses on customer attitudes, preferences, and opinions about a product
- Short amount of time
- Allows for free flow exchange of information
- Cannot access how product design will actuality be used
- Can be dominated by a single participant
- Strong bias to what others want to hear, especially the moderator (social desirability bias)
Common quantitative methods:
- Technology that captures eye movements
- Can be used in the lab or field
- Best when noticeability of key feature is critical ie: are people seeing x
- Easy to set up
- Analysis may be tedious with video
- Most people can be calibrated
2) Online surveys-you can get into trouble here! Easy to jump to conclusions with a poor survey design!
- Easy to set up and administer
- Low cost
- Technology is advancing
- Focused on opinions only
- 10% of participants are fraudulent
3) Unmoderated usability testing
- Participants use a web application that guides them through a usability study
- Large sample size
- Heavily focused on metrics
- Collect reliable metrics on all facets of usability
- Data collection is quick, easy
- Cannot use if you are n0t ready to send out prototype
4) Card sort-most ignored, but valuable!
- Use a web application that allows them to categorize cards into groups
- Tools: WebSort Note: I recently used this as a user with an Educause survey, where I was asked to sort the different parts of the Educause website into meaningful categories. Cool!
- Need 30 people to get reliable data
- Validates information architecture
- Complicated statistics are easily managed and interpreted through compelling visualizations
- Does not work with multiple levels
- Difficult to understand mental motivations
So what if you have no time!?
Use a Usabilty test discount method– just grab some people in your organization! This is a valid expert review, and requires no budget!
I have to say, having just finished a web usability project, I did only usability testing, without any tracking software, and a survey. I REALLY want to do a cardsort and eye-tracking for the next time! I am still looking for a cloud ux software that will allow a user to log in, and do our usability test from anywhere, on any computer. That might work with Loop11, and I will be trying it this week.
Most of all, I love when a speaker understands that sometimes you do not have a lot of time or money to do a project. It is great to be able to spend a semester, or even a year, working on something, but sometimes you only have two weeks, or no money. I believe UX always matters, and you can still put your UX hat on and pull up your UX bootstraps and get to work even if you have no time or money!