Steve Krug: You’re NOT Doing Usability Testing? Are You Nuts?

The University of Toronto iSchool was lucky enough to get Steve Krug of Don’t Make Me Think fame. If you haven’t read it and you’re at all interested in usability, I would definitely recommend it. Anyway, here are my notes from today’s talk.


Convince you that usability testing is the most valuable thing you can do to ensure that what you’re building works.

12 Years Ago – The Debate

Realized seeing the same story over and over again. He first started by turning it into a comic. Basically the same debates happened over and over, frequently without decisions being made. It’s difficult to find a middle ground, because each person comes with their own perspective. Partly, there’s no hard and fast answer, because it depends on the context.

Usability testing eliminated most debates. Seeing users use what they designed changed their perspective and start talking from the user perspective rather than personal biases.

Keeping It Simple

Traditionally, usability is done in a lab with the minimum 8 users in a day by an outsider, ending in an expensive report which may not be convincing to the team.

Rocket Surgery Made Easy written more for people intending to actually do usability testing.

Just do it with:

  • 3 users per round
  • set up a monitor in another room to watch
  • record screencast
  • no stats, no faux validity, but produces meaningful insight
  • no big report, just debrief over lunch, report in 1-2 page email

Live Demo

Use script, which can be downloaded from Krug’s website. Instead of a website, he did a live demo on the Clear mobile app. Interestingly, the participant could not complete the task, which was interesting since the app is highly praised.

RSME: The 22-minute version

6 Maxims

  • A morning a month is all we ask.
  • Start earlier than you think makes sense. – Can start before you start by testing a competitor’s or old version, or even a sketch on a napkin.
  • Recruit loosely and grade on a curve. – Take just about anyone, and judge whether everyone would have that problem or if it’s just because not from target audience.
  • Make it a spectator sport.
  • Focus ruthlessly on the few, most serious problems. – Right away write down the top 3 problems to take into debrief.
  • When fixing problems, always do the least you can do. – The smallest change might do to solve or help mitigate the problem. Tweak, don’t redesign.


  • James Chudley CX Partners wrote on the approach to doing mobile app testing that needs context.
  • Tools for Remote Testing – Goto Meeting, WebEx and need high quality audio
  • Analytics good to help know what to test
  • Do usability more frequently rather than testing a lot of people, because you quickly hit diminishing returns
  • Task specification – allow user to have the choices of content

Author: Cynthia

Technologist, Librarian, Metadata and Technical Services expert, Educator, Mentor, Web Developer, UXer, Accessibility Advocate, Documentarian

One thought on “Steve Krug: You’re NOT Doing Usability Testing? Are You Nuts?”

Leave a Comment

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: