We’ve all been there, we start designs for a new project and have the best intentions to test our designs with users. However when priorities are defined, “shipping it” gets in the way and our good intentions are blown away. The good news for Agile teams is that there is a workflow for making usability testing part of your design process. In this post, I will share 10 best practices to help you get there:
- Plan like a boss
- Make usability testing a story
- Test little, but test often
- Find participants: soon!
- Create a script, and stick to it!
- Run a pilot test
- Embrace templates!
- Invite your team to the testing sessions
- Be specific during the debriefing
- Go with the low hanging fruit
Approach usability testing like a pro by creating a simple, yet informative, Usability Testing Plan so you can share with your team and get a buy-in for doing testing as part of your design work. “Wait, do I have to sell this idea to my team?” The answer is YES! Part of the Agile spirit is working as a team, and because most people don’t see the value of it yet it’s important to make the case and explain why, what, and how you will test.
The why - If your team is new to usability testing, brief them on the why is usability testing so important. The point is to make everybody comfortable with the idea.
The what - Think about the current needs of your project. This will
help you evaluate what you should be testing and in which priority.
Remember at this stage you want to keep it very high level.
The how - Will you use prototypes or live code to test? Will you
conduct tests in a lab or remotely? How often will you test? Sorting this will help your plan take shape and anticipate possible questions from your team.
I find useful to also include an illustration of the testing cycle as this helps understand better what’s involved in conducting the usability test and how it will play out during the sprint.
Usability testing should be added as a design story in the project backlog. Actually, it should be multiple stories! Because the idea is to test little but test often (more on that in number 3), adding multiple stories in the backlog for usability testing will help the team prioritize and plan accordingly.
Once the story is in the backlog, you can break it down into multiple tasks, like:
- Defining the tasks to be tested
- Scheduling participants
- Preparing the test
- Conducting the test
- Preparing and presenting findings
- Creating stories based on findings*
*This one is key as it will ensure findings are actionable.
When usability testing is a story, it will make your tasks visible to the Agile team during the sprint. It will also help reporting on your progress during scrum and is a great way to internally socialize usability testing.
The idea here is that because you will be testing often, you can focus on testing just a few things every time. Identify the most important tasks to be tested (an average of 5 tasks is a good number). This will help you keep the tests short and doable during a sprint.
Include these tasks in a “mini-plan” for the specific test in addition to:
Creating a “mini-plan” will help you develop the test as well as writing a the final report or findings summary. I especially find it useful to connect each task to a goal, as I can then evaluate if the goals were met and quickly summarize the outcome in debriefing sessions.
Since usability testing depends on having participants, be sure to find and schedule participants as soon as possible, even if details about the test are still being worked out. But where do you start? Here are a few ideas:
Ask the product owner or stakeholders on your project. Many of them know actual users that would be happy to participate.
Recruit people via HR department. They have access to all of the company’s staff and can help you recruit employees and new hires that are not familiar with your project.
Use a remote testing tool. If conducting remotely you can try online applications like ethnio or usertesting.com. Ethnio works for finding participants, and usertesting for finding and conducting the test. There is a wealth of user testing apps that can get you what you need.
Find them on the streets! If this is an option for you, you might be able to find participants just about anywhere. You may meet very interesting people on the way.
When booking participants:
Schedule the tests consecutively but allow some time in between for quick breaks. This will help you focus and dedicate an entire morning to just testing vs jumping back and forth between tests and other tasks.
If you are going to record the session, give participants a heads up. Some people might not feel comfortable with it, and it’s good to let them know prior to the test. You might even have them sign a form.
If conducting the test remotely, make sure participants will be able to access the app or prototypes to be tested.
Prior to conducting the test, write down exactly what you plan to say during the test, practice it, and then stick to it once the test starts. This works well because it helps you keep focused and on time. Usability testing is about crafting tasks that actual users would perform in real life using your design or app, and then observing and listening, so the less talk, the better.
Include in your script:
Notes to yourself. Write down tasks that you should not forget to do or mention, like starting the recording or even noting that the session will be recorded.
An introduction to the test. Explain how the test works and how long you estimate it will take. Also state how many tasks there are and prompt participants to tell you when they think they are done with each task. This will help keep the test moving.
Information about each task. Provide the scenario for each task to encourage the user to engage with the interface as they would do in a real-life scenario. Clearly denote any actions that need to be taken as part of each task.
Closing Remarks. Use the end of the test to ask the participant clarification on any actions that were not straightforward. Many people have a hard time voicing what’s in their head. Also, allow time for them to ask questions and then thank them for their time.
At Bitovi we use this script template. Feel free to snag it and shape it to your needs.
Before conducting the actual test: test the test! For this, find a “neutral” participant with whom you can share your design but who doesn’t know much about it (for example this could be a developer from another team). Then, conduct a pilot test to simulate how the actual test will work.
Use the pilot test to:
Verify prototype functionality. The last thing that you want is for a participant to get stuck in a flow because you missed to include an interaction in the prototype.
Validate your script. Practice using your script, paying special attention to whether or not the user understands the tasks and then adjust accordingly.
Time the test. Use the pilot to time how long the test session will go, from start to end. Doing this will allow you to adjust your script and make sure tasks are doable within the proposed timeframe.
Since conducting a usability study has many repetitive tasks, having a few templates that you can use will make the process smoother.
These templates are the ones that I find the most useful:
As you use these you will find ways to improve them, so be sure to reach out to @adrifolio for any suggestions!
Inviting your team to be observers during the testing sessions is a great way to promote a user-centric culture. As an observer, everyone can learn more about the users and have empathy for their needs.
Ask observers to take notes during the test and to send you the 3 most important learnings they noticed. Compare with your notes and include conclusions as part of the debriefing meeting.
Not everyone will be able to attend all of the sessions, so it’s good to keep the team updated about what’s happening during the sessions by reporting any changes on the schedule or location, as well as sharing videos of the testing sessions, when available.
To conduct an efficient debriefing meeting, present your findings describing not only what is not working, but also the reasons why this is the case.
A good way to uncover these reasons is to observe the flow each of the participants took to accomplish their task and then note any pain points. For example. In the following flow is illustrated two different paths to accomplish the task of:
- Scenario: Imagine you want to buy a blue chair and you want to ship it to the zip code 12004.
- Task: Go to [url] and add to the cart a blue chair.
Then you can divide these pain points into:
- Issues that hinder the experience (in yellow)
- Issues that halt the experience (in red)
Visualizing each of these flows side by side will help to paint the picture of where the issues are and find the specific causes. Then, during debriefing you can share your findings with the team along with the flow analysis and provide more insight into the conclusions of the study .
While a usability study can surface many issues (even some that weren't part of your specific study) it’s good to step back and ask yourself: “What is the minimum that can be done to improve the UX?” The idea is to focus on the small items that can be done within the next sprint, even if these don’t solve bigger issues.
Finally, be sure to convert all of the findings into stories and add them to the backlog. Usability studies can’t create a real impact if findings are not documented and tackled during the following sprint cycles.
Here are some resources to bookmark. Be sure any other resources you dig in!