How I sold a monthly usability testing program to my client
Starting small, gaining momentum and getting the buy-in for continuous research
The best organizations out there that have anything to do with tech and digital products (and beyond) run regular user research. They have a system in place where their product and its features are regularly exposed to (potential) users for early and continuous feedback.
If you are a designer and you don’t work in a company that implements such a program, I know how you feel. You might be thinking: “It sounds great, but my boss is never going to agree”. Your small team is always rushing to build stuff and you can barely squeeze in some usability testing here and there. Maybe you just test with colleagues or friends and family. And it’s all very sporadic and time consuming.
With this article, I’m sharing the story of how I managed to go from this scenario to selling a monthly usability testing program to my client, walking you through the steps I took on my way.
What is usability testing?
Let’s start by defining what usability testing actually is.
“Usability testing refers to evaluating a product or service by testing it with representative users. Typically, during a test, participants will try to complete typical tasks while observers watch, listen and takes notes. The goal is to identify any usability problems, collect qualitative and quantitative data and determine the participant’s satisfaction with the product.” Source: usability.gov
What are its benefits?
- Testing a solution and iterating on it before spending time building it saves effort and money
- Improved usability means improved business metrics (e.g. better conversion)
- By speaking with end users, you’ll find new and surprising opportunities for improvement
- Usability testing is a powerful eye-opener. Seeing users actually failing to understand your product has the potential to change even the most stubborn of minds and introduce a more user-centric approach
Our starting point: sporadic usability testing
I’m a product designer at an agency, and I currently lead a small design team working on a long term commitment with one of our clients. As I joined the project, we weren’t running usability testing very often, and I made it a priority for myself to find ways to improve that.
At the same time, I knew from past experiences that I couldn’t just walk in and say: “Hey, let’s spend a bunch of money every month to recruit participants and do usability testing!”. I needed to first build a culture around testing early and often, and find ways to start small and show the value of usability testing to my client and team. In other words, I needed to lay down a solid foundation of cheap, quick wins to get people on board. It’s relatively easy to get the buy-in for just one round of testing, but getting your stakeholders to say yes to regular research is a whole other story.
1. Empathize with your stakeholders
Before we start thinking about solutions, we should try and understand why it’s so hard to get people on board to begin with. As designers, the value of research is crystal clear to us, and it might be difficult to understand how somebody could just not get it.
Here we can make use of one of our most valuable design tools: Empathy!
If you put yourself in your stakeholders’ shoes, it’s understandable that they could be skeptical about your plan. They might not know its value or how much time and money you want them to spend in order to unlock that value.
The truth is, it’s hardly ever really a problem of time and the money spent in usability testing is unlikely to make a dent in the budget. People mention time and money because that’s what they know and can measure, but the obstacle is hiding behind that. The real problem is mostly people not knowing the value and the return of investment (ROI) of research.
That’s why the first step is to empathize with your stakeholders, understand where their skepticism is coming from and then think of ways to show them that research will improve their product and it’s not as costly and time consuming as they think.
In fact, usability testing is going to save money by identifying issues early, when fixing them is still cheap. Fixing usability issues can also make money, by positively impacting some of your KPIs, a typical example being increasing revenue by improving your conversion rate. Learn more about the ROI of UX here.
2. Start small and build a foundation of quick wins
I found that a simple way to remove the money and time obstacles and still show the value of research to your stakeholders is to start small. Find a cheap and quick way to run some usability testing often: it will make it much harder for a stakeholder to say no.
So that’s exactly what I did: as we were working on new features, I casually proposed to run lightweight testing. I would say things like: “I think we should test if our design works. Why don’t we run a lightweight usability test? We can just put together a quick prototype and test it with 5 people around us to see if we find any issues”.
Luckily, in our case we had quite a few people to test with as I work at an agency and many of my colleagues are not familiar with my project. So we set up tests and recruited participants among our colleagues. We didn’t need any budget and the time investment was very limited: only around a week for a designer to work on a prototype and script, and about a day for the actual testing.
One thing though: the fact that you are running cheap and quick tests, doesn’t mean that your outcome can also look cheap. Make sure you treat your tests in the most professional way possible, from planning to reporting the results.
For each test, we did the following:
- Ideated and agreed on research questions with our stakeholders
- Wrote a test plan and script, which we’ve also shared with stakeholders for reviewing and approval
- Piloted the test before running it
- Found 5 or 6 participants among our colleagues
- Invited stakeholders to the sessions as observers
- Ran and recorded the sessions professionally
- Planned and ran a debrief meeting, involving stakeholders, at the end of each round to discuss and summarize the outcome
Side note: unless your stakeholders are already familiar with usability testing, sooner or later you’ll get some variation of the following question: “Are we sure that testing with only 5 people is enough?”. At this point I’ll say something about the qualitative nature of this kind of research and I’ll be sure to end the conversation by sending a link to this article by Jakob Nielsen.
3. Involve stakeholders along the process
“Tell me and I’ll forget; Show me and I may remember; Involve me and I’ll understand.” Chinese Proverb
One point that I briefly touched upon in the previous paragraphs is actually a crucial one: inviting stakeholders along, from the beginning to the end of the process.
Stakeholders need to be involved in the test planning, so that they have a say when it comes to research questions and plan, and they feel more invested. This also means they will trust the process more, since they took part in it.
Next, you need to make sure stakeholders come to at least a few of the sessions you have scheduled. It’s a known fact that usability testing can be an eye-opener. When you see users struggle understanding your product, it can be a revelation. You might finally realize you are not the user!
When it comes to convincing stakeholders about the value of usability testing, Steve Krug adopts this approach:
“The tactic that I think works best is getting management to observe even one user test. […] In my experience, executives often become fascinated and stay longer than they’d planned, because it’s the first time they’ve seen their site in action and it’s often not nearly as pretty a picture as they’d imagined.” Steve Krug, Don’t Make Me Think
If your stakeholders can’t participate to any of the live sessions, you can always send them a video reel with some highlights. However be mindful that they might end up not watching it, and in my experience you’ll get better engagement if they are right there when it’s happening.
By inviting stakeholders to take part in the process, you are building awareness and finding allies. Let’s face it, at the end of the day your client or boss is the one with the budget and the power to make more rigorous research happen, so they have to start trusting the process and there’s no better way to do that than to experience it first hand.
4. Document, share and act upon the outcome fast
Once the test round is over, you might think your work is done. Well, not quite. What comes next might be just as important, especially if you are trying to demonstrate the value of what you are doing.
First of all, make sure you document and share your findings. After running a debrief meeting to summarize the outcome, write a report and share it with the broader team. Do a readout, post it on Slack or do whatever is needed to make sure the information reaches many people in the organization.
To that end, the outcome of your research might become a piece of knowledge about your product and users that can inform future design decisions, too. So make sure you build a repository of test reports and people know where it is. Bringing up past research and using it to solve new problems shows once again the value of what you are doing here. You are building knowledge that can be referred to time and time again.
More importantly, after the research something has to happen. Documentation without action is nothing. If no action is taken after the tests, then what was the point of doing them in the first place? You need to get to the point where you can show a good story to your stakeholders, and to do that you need a cause and effect relationship between your research and what happens next.
That’s why it’s your responsibility to take ownership and keep the momentum: right after you wrote and shared your report, plan a session with your stakeholders and team to think about action points. What can we already do based on the outcome of this test? Can we already create user stories for some redesign or fixes? Make it happen! People have to see that your test generated insights that fueled improvements that were actually implemented fast.
5. When the time is ripe, pitch a continuous research program
Eventually, if you are as lucky as I was, your client or boss will be pretty excited about usability testing and you’ll see that they start mentioning it to others and bringing up ideas on what we could test next. Testing will be a standard practice in your team by now, and it will be seen as a necessary step to validate your designs.
You will also find that it becomes less and less feasible to keep running it low budget as you did so far. For instance, you’ll see that recruiting internal participants is not scalable. You’ll run out of people to ask to and you’ll start getting the same participants over and over again. This leads to a biased, homogenous sample, which is not reflecting the spectrum of your real or potential customers.
At this point in my journey, I decided it was worth spending some time to craft a good old slide deck and pitch my idea: a monthly testing program with participants recruited through an agency.
So I looked back at the last few months and put together a story starting from a description of how we did usability testing at that point and the issues it caused. I also included some benchmarking, where I showed examples from well known companies which are running weekly or biweekly testing. I knew that a weekly cadence was not feasible for us and maybe even counter productive, as we don’t have a dedicated research team yet. Nonetheless, I decided to present this and then proposed a monthly research program for our team. Here, I made use of a psychology principle called anchoring, which is commonly known in behavioral economics and used in negotiations. After setting the frame of reference (or anchor) to a weekly cadence, monthly research doesn’t sound like a lot of work.
I then pitched my idea in a 1 to 1 chat to my main ally: a key stakeholder who was involved in all the previous usability testing rounds and was pretty engaged. He also had the budget and authority to make this idea happen. You want to hit the right combination here: someone who is excited enough about testing and has enough power to promote it within the organization.
The pitch was a success beyond my expectations. The client couldn’t wait to get started with the program and we immediately began looking into how to secure the budget to recruit people every month.
6. After the yes: don’t just sit and wait for things to happen
You made it! You got the green lights for your continuous testing program. Can you now sit back, relax and wait for it to happen? Once again, not really. The truth is there’s still a long way to go to implement your program and make it part of the ways of working and culture of your organization or team. And don’t expect it to be anybody else’s priority. Nobody will keep pushing for this to happen if you don’t.
So do whatever it takes to start as soon as possible with the first rounds of testing. In our case, we got the budget to run a three months proof of concept (three rounds of testing), after which the client evaluated the results and secured the needed budget for ongoing research.
You also have to make sure that running monthly tests becomes a smooth process and doesn’t take too much of your time every month. So what we did when setting up our proof of concept period included:
- Finding an agency to recruit participants
- Choosing tools to perform, record and store our sessions
- Writing templates for test plans, scripts, reports, consents and other documents
- Creating a repository for test reports
- Setting up a research backlog and a kanban board (we used Jira)
- Scheduling recurrent meetings with the people who were going to be involved in the program
- Educating other designers in the team and involving them in the process to make sure they got some experience with it and we could build those skills across the team
And if you get a no, don’t be too hard on yourself. This is a success story, but for every success there are many failures. I can’t tell you how many times I tried this in the past and failed. So no, you are not a bad designer. Just let this one go, learn from the experience and try again. Trust me, it’s worth it.
Did you or your team try something similar? How did it go? I’m interested in your thoughts and feedback, so please feel free to leave a comment!
Further readings
I’m a Product Designer currently crafting meaningful digital products and experiences at Accenture Interactive Amsterdam, where I apply user-centered design principles and methodologies to make technology human. If you want to get in touch, you’ll find me on LinkedIn.
