Understanding user needs is critical to the success of any product, and usability testing is one important tool that helps us build websites, programs, and apps that users love. But not all usability tests are created equal. I recently had the opportunity to conduct a contextual usability test with Wits, a Minnesota Public Radio show, and was reminded of the quality of information you can get by observing users in a real situation.
The team reviewing Tweetwally setup at the event.
Clockwork is in the process of implementing new Pro features for Tweetwally, one of our products. Wits agreed to use the beta version at their season premiere. The show is famously Twitter-friendly during the live tapings. They encourage tweeting at the event, and display all Wits-specific tweets (hashtag #wits) on a tweetwall at various points throughout the show.
Wits was particularly interested in Tweetwally Pro’s new moderation feature because it would allow them to approve tweets before display on the tweetwall. Not only would Wits be using a feature they were excited about and invested in, we could watch them use it in a high-stakes, real environment. This is the ultimate scenario in usability testing for a few reasons.
It removes the need for hypotheticals.
We didn’t create scenarios and ask questions like, “Imagine you want to . . .” The user didn’t have to imagine anything. She had a job to do and knew exactly what she needed from the tool.
Important use cases quickly rise to the top.
By watching her in a real environment, we saw first-hand all of the situational challenges and demands—the real use cases. We were able to quickly assess what was working well and what wasn’t, as well as prioritize enhancements based on real needs.
User behavior is more natural.
Much of what we learned by seeing the tool in action at the event may not have been discovered through testing in a lab environment (usually an office with monitoring equipment). While doing any usability testing is usually better than doing none, testing in a lab can make the participants nervous or disconnected from how they would use a site or application in real life. While it’s difficult to completely remove these variables, testing in a real environment makes it a lot easier.
Contextual observation isn’t always an option, in which case using a lab for testing may be the next best thing. But if you see an opportunity, take it. You won’t regret it. There’s simply no replacement for the insights and efficiency gained by watching and talking with users on their turf.
The real-time, real-device testing scene.
