UX Research — Surveys
Getting past the three research hurdles
Many businesses often have a lot of ideas on what products and services would be enticing to their clients. But without context and feedback from the end user (aka their clients), businesses may be blind to what their clients truly need. Surveys are one of the simplest ways to gather early and quick feedback. Even with its simplicity, it is sometimes difficult to do in the context of a bigger organization or with skeptical members of top management. In this article, I share three hurdles I faced when I conducted my research and some tips and resources that may become helpful to you.
Hurdle One: Handling Skepticism
There is a general skepticism about how valuable user research is. In my own experience as a UX Consultant, even in industries where top management is willing to pay for the service, there will be members of the team who are not convinced that it is worth the effort. Skepticism with regard to the value of user research is not a new topic. In fact, there are so many references you can look into on how to win stakeholders over.
- Christopher Nash from Dropbox Design wrote a Design Research article specifically on this topic.
- This Quora thread has a good range of responses from people in the UX field. Read: How do you handle it when people are sceptical of the value of usability research?
I’d like to share my own experience. I vividly remember sitting in a room where one of the Department Heads was skeptical about the whole initiative. “It’s just a survey, why must we pay someone to do it? We’re deciding what to launch, why run a survey anyways?”
I explained that the survey was to gather feedback on the products they planned on offering. Since they had a variety of options to offer but were road-testing just a few to see how viable business is, it was worth getting early feedback to know if they were headed in the right direction.
Despite the facts and statistics, the other members of the management team also wanted more assurance. They felt that they could not gather certain feedback or clarity because they did not have the time to conduct the research, thus they felt it best to hire someone who could.
The survey questions I prepared aimed to understand their profile, context and interest levels with the different products they had. I also offered my research as an opportunity for the business team to ask questions that they still have left unanswered. Upon giving them the chance, they shared what they wanted to know and that provided me an opportunity to share how we can make the insight gathered more robust if we try to learn more about their habits, understand other contexts that affect how they would eventually use the product/service.
People within the team became curious as to what sort of insights could be generated by the research. The hesitancy and skepticism faded because the planning stage was transparent, collaborative and open to everyone’s input. Because they gave their input on the questions, the ownership for the project became shared. I learned that sometimes, the skepticism is not just on the value of your research but on what your role is in relation to their job. It was no longer about an external consultant coming in and telling them that they were doing things wrong. It changed to an impression that someone else was coming in to help them learn more about their users.
Hurdle Two: Conducting the Survey
When surveys are well done, you can get a lot of valuable information. I have listed some great references to help you craft your survey:
- Nielsen Norman Group’s 28 tips for Creating Great Qualitative Surveys
- UX Mastery’s tips on Creating Effective Surveys in their article Better User Research through Surveys
- Sarah Doody shares a tip about using first person questions to get better survey insights in this article.
As suggested by NNg in the article above, it’s best to use a platform that’s already working. Most of these platforms (SurveyMonkey, Typeform etc.) allow you to set up logic for surveys. This helps you distill insights better and prevent clashing answers.
But there are other things that affect how people answer your survey. Literacy affects survey validity. In an instance when the business team explained that a lot of their potential users were not literate in English and not as tech-savvy, I decided to conduct the survey in a more approachable way.
Instead of an email link from someone they did not know, I asked for a coffee break with the respondents. I asked that a contact person from the business would make the introduction but at the venue, I took over. I introduced myself, told them who I was and explained that I was asking them to participate in a survey that should not take more than 15 minutes of their time. I gave assurance that the survey was not in any way going to affect their performance appraisals.
Instantly, some of the people in the room laughed. “We get so many feedback surveys that we just give the highest rating and finish it.” I thought so too. I handed printed surveys with translations to everyone and I asked them to let me know if they did not understand any of the terms presented and if any of the options in the survey did not make sense to them.
Being smart with how you conduct research is not just with how you use your resources or plan your surveys. It is also about how empathetic you are to your respondents’ context. It is not just a matter of asking the right questions but also asking them in a format that your respondents can access. It is asking them in place where they feel safe, in words that they can understand and using tools that they can use with ease.
Ensure that the words that you choose can be understood. More importantly, provide opportunity for comments or clarification. In the same project, it was helpful for me to be present during the data gathering because the respondents had a chance to clarify certain words that they hadn’t encountered before directly instead of guessing through their survey.
Hurdle Three: Sharing the Insights
Once you finish gathering your insights, you will have to share what you have learned. Sharing insight is part distilling and part visually communicating. Post data-gathering, you will have two types of insights: those that are directly related to the questions you asked and those that are indicative of other considerations affecting your users.
Presenting the direct insights must be summarized in a way that is understandable by anyone who would look at your deck. Depending on the information you can summarize this into quotes, tables or data visualizations like pie-charts, bar graphs and word clouds.
Check out resources such as:
- DataViz Project
- Nancy Duarte’s videos and books for inspiration.
Equally or more important for people who will read your research, would be the insights unearthed that show relationships and factors affecting the end user’s context of use.
Do not filter this information even if you feel that some of them manifest stereotypes. As a researcher, it important to share what you find out and enable teams that will use this research to design experiences and products with these contexts in mind.
As much as I can, I also create one-page summaries that can easily be attached in an email or printed to distribute in a meeting so that the insights can be shared.
End note: Getting feedback from your end users is a worthwhile initiative. In my experience, I experienced a few hurdles but ultimately, the client was happy with the insights and saw the value of doing research. Some surveys may validate, others may change your course, while some present new opportunities. In my case, certain insights prompted my client to pivot their strategy because what they had assumed to be true about their users turned out to be false. They prioritized other ideas and have done well.
In fact, there are countless references about how UX Research is worth doing not just because of ROI but also because of cost-savings. See some references below:
