Last week (was it really just last week?), I did my first usability test and I thought it went well enough, but there are of course improvements needed. I looked up some resources (which I will put up on a later date), but while there is a general outline, no resource can give you specifics on how to conduct a usability test for a particular site.
Methodology
- 5 participants, 1-2 from each user group
- Each participant was given the choice of using a PC or MAC.
- Each participant was given a scenario of working on assignments by themselves without facilitators to help with the task itself.
- Participants were given 5 tasks to do, presented one at a time.
- Participants were asked to voice their thoughts and were asked questions about their process during a task, after a task, and/or after all tasks were completed.
- Each session was recorded using video, audio, and screencapture programs.
Results Analysis
Results were compiled for completion rate, but no other metrics were found useful. For example, time completion did not work in this case since users were asked to voice their thoughts and some did so very thoroughly, while others did very little.
Most of the analysis then was drawing conclusions based on behavioural trends and repeated comments made by users.
Results
The results might have been as expected. Users tended to be either novice or expert users, which may seem fairly obvious, and 1 of 2 types:
- selective user: tends to look over things carefully, choosing that which seems to best fit what he/she wants. Unlikely to click on unfamiliar things.
- explorative user: tends to click on the first link that looks like it might be what they are looking for. Does not mind making mistakes. More likely to click on unfamiliar things.
Recommendations were made about the site in an attempt to make the site user-friendly to both types of users, and to ensure both types navigate the site as it was designed.
A number of recommendations were also made revolving around content, as there were numerous content issues and content is not taken care of by the developers (which includes me).
Reflections & Improvements
Overall, I thought the sessions went fairly well. There were a couple of improvements that we implemented in the middle of the study. Although in a more academic-based research study, this might be considered taboo, we thought it would produce more useful results.
Some improvements we made:
- printed copy of tasks
- added to script that task completion is user determined (not determined by facilitator)
- made sure to clear browser cache for every session (browsers can be set to do so automatically of course)
- minor rewording of tasks to make examples as unambiguous as possible
For the next usability test, further improvements can be made:
- more context for scenario to give participants appropriate perspective
I think it is also very valuable to have a second facilitator since each facilitator tends to catch/see and focus on different aspects of the user experience, so each will contribute to the questioning of the participant.
Conclusion
The usability test was very valuable in seeing whether the design and organization worked for our users. It also helped to identify various problems and what’s better, how we might improve them (as some tasks were purposefully chosen because they might be problematic elements on the site). Some improvements of the site will depend on others, but hopefully, the results of the study will convince them that the improvements need to be made.
