How to start a User Research Repository
Anyone who has read my Medium articles knows: I love everything that is systemic. Why? Because it’s the best way to build something better and bigger while understanding the underlying relations between elements.
So, after Atomic Design & Design Systems, it was more than logical for me to fall into Atomic Research & User Research Repositories :)
In this article, I’ll explain why I think this is the next “big thing” for the design world and detail how we started this journey at OpenClassrooms.
First, what is a User Research Repository?
A User Research Repository is a place where you can find existing user research insights and create new ones. This allows different profiles within the company (and not only the UX team) to access different parts of the research results, use them for their own projects, and make clear-eyed strategic decisions based on them.
A user research repository is a knowledge base that stores and organizes user data, insights, and research to help teams effectively analyze information and collaborate to improve the product experience.
And it’s not something new. In “classic” research domains, researchers also share their publications in repositories (for example, the RMIT Research Repository) and regularly build new knowledge upon others’ findings. That’s how they can always push the research further and discover every day more and more about science, space, human evolution, etc…
Why did we built that repository?
When I arrived at OpenClassrooms, I discovered that the knowledge around users was not really shared or re-used after the research experiment was over. It was hard to deeply understand who our users were and what their main needs were. And even harder to collaborate with other teams around user research.
When we structured the practice, we made several observations about what wasn’t working:
- User research conducted by product designers often stayed at a squad level and was rarely reused for another subject. Consult previous research was hard unless we knew the person who worked on it. And even then, we often landed on a super-complete (but not easy to read) analysis of which we only needed a small part.
- In addition, we discovered that other departments were also conducting their own research (e.g. sending out surveys). And again, it was a hassle to find who worked on what or to retrieve the analysis. In the worst-case scenario, there was no analysis at all and we only had access to the Google Form’s raw answers.
We concluded that all this precious material collected during the user research was under-exploited. What if we could centralize everyone’s research results and go deeper into our knowledge, instead of keeping the results for ourselves and asking the same questions to our users again and again?
That’s why my team and I built our User Research Repository.
Where did we start?
It was in 2021 when we first talked about having a shared repository. At that time, we were only 10 on the team and I had just offered one of my Product Designers (the talented Mathieu Klein) to become our first full-time User Researcher.
If you want to know more about user research skills you can check this article.
#1. Define our vision
Creating a new job meant structuring the practice and defining what we wanted to achieve through our user research practice.
I realized that, in previous years, OpenClassrooms’ user research practice focused on testing designs and defining whether we were “building the thing right”. That is usually called “evaluative” research.
However, user research should also be “generative”, meaning helping the company to “build the right thing”:
We want user research to help collaborators to make the right decisions for our product, ensuring that the voice of our users is known and heard.
Our vision: “User research should empower a strategic, user-centric vision and help collaborators taking data-driven decisions by providing impactful and reliable insights.”
To achieve this vision, we worked on several axes:
- Evangelize user research within the company.
- Spread our methodologies.
- Create a user panel to aid recruitment.
- Centralize our research results to be more efficient and be sure we communicate every insight at the right level.
This last point was the trigger of our repository.
#2. Better understand Atomic Research
Of course, when I heard about “Atomic Research” I immediately thought: “This is for me!” :)
But I have to admit, the metaphor is not as clear and pertinent as the atomic design. I don’t think that atoms, molecules, and organisms really make sense in user research are we don’t manipulate “bulding blocs” but data that can have several dimensions. What's interesting though, is that it gives a concrete model to start with and it was a great help in structuring our repository:
Some people may ask: can you really start a repository without having this model in mind? Yes, you can. But I would say that it’s the same as starting a Design system without knowing anything about atomic design…
Let’s see no w the different components of Atomic Research:
- Research Experiments: A research activity is undertaken to answer some questions we have regarding our users. Research can be conducted by any method that helps collect user feedback, such as interviews, user tests, surveys, and so on.
Example: we noticed that some of our students were late in their training. We then run a series of interviews to understand why. - Facts: During the research experiment, we collect “facts”. These are unquestionable statements, something the user says or does. It can be a quote, a video excerpt, a pourcentage in a survey, etc.
Example: A quote from a student reads, “Project 5 was very hard. I felt stuck and I lost time.” - Insights: Insights are what we’ve learned, the analyzed result of the different facts we’ve collected. Insights are immediately usable to take action on the product. In our repository, they look like small articles with the main lessons and evidence (check our template, later in the article).
Example: I collect a lot of different facts around this same idea. So the insight could be, “The difficulty of Project 5 is underestimated and the time planned to do it is not realistic.” - Conclusions & opportunities: The sum of those different insights will strengthen our convictions. We will then be able to make recommendations to improve the existing experience or suggest something new.
In main companies, the deliverable of user research is frequently the final report with all the conclusions and recommendations of a specific experiment. The goal of atomic research is to understand that every piece of research should be accessible and reusable. So it’s not just about delivering the complete analysis report anymore but clearly extracting the facts and insights, and storing them so everyone can consult and reuse them. Thanks to the repository, you can start searching from wherever you want (research experiment, facts, insights, or conclusions) but still keep the link to the other elements.
To finish with my example: the insight on “Project 5” could be directly reused by our Learning team. They will have access to this very specific learning with all the linked facts without needing to read all the other conclusions of our research experiment. They will be able to push the research further and build their own conclusions: “Is it normal that this project is so hard?”, “What could we do to support students during this specific project?”, etc.
If you want to know more about Atomic Research, I encourage you to follow Daniel Pidcock who produce a lot of interesting articles and talks on this subject.
#3. Define who will use it and how
The other thing that was important for us was to define the main users of this repository. Here are the users we defined:
- Owners (e.g. User Researchers)
Owners are the first contributors to the repository. They structure the repository, defining the guidelines and process. They are in charge of maintenance and help choose which source of user feedback should be included. - Recurrent contributors (e.g. Product & Content Designers, PM,…)
Recurrent contributors constantly feed the repository with all the facts collected during their research and produce insights when they find something that is worth sharing. - Facts consumers & occasional contributors (e.g. Product Marketing)
These users don’t directly upload facts in the repository but they want to find insights to reuse them in their daily decision-making processes. They can sometimes create their own insights, based on existing facts. - Insights and research results consumers (everyone in the company)
These users only consume the final results of research (insights and full analysis). We don’t give them access to the facts because we’d rather they focus on high-confidence, analyzed results. For now, they’re not allowed to create insights… At least, not in this first version of our repository. :)
What are the structural elements of our repository?
The tool
The tool is important, but it should not be an obstacle to starting. You can start building and storing your research results following Atomic Research principles with free version of tools like AirTable, Notion, or Coda. You can even start with a Google Sheet containing links to your research elements, as it will help you test the method and refine your needs.
On our side, we were lucky to have the opportunity to test EnjoyHQ (recently acquired by UserZoom) for free for one year! It was perfect to test, learn, and then decide whether we wanted to keep this tool. We did a benchmark on the other tools (Dovetail, Airtable, EnjoyHQ, Condens, and Aurelius). Our main criteria were: The capacity to centralize insights, find them easily using keywords, analyze insights, and communicate them in a legible way. We also had criteria around the security of the data and the pricing:
We decided to stay on EnjoyHQ, mostly because of its powerful way of automatically integrating different data sources (e.g. NPS, AppStore comments, Slack Channels, Zendesk tickets). It also allows us to create rules to automate tagging.
Taxonomy and properties
Taxonomy is the heart of the repository, as it’s what allows people to easily find the facts or insights they’re searching for.
The best way to define the taxonomy is to collaborate with other teams to define common rules, especially with the Data team as they might already have a taxonomy you can start with.
On our side, we defined 5 main types of properties:
- User type (e.g. visitors, candidates, students, mentors, alumni)
- Language and country (e.g. French, US English, UK English)
- User journey moment (e.g. discovery, admission, training, diploma)
- Domain (e.g. marketing, learning, admin, student support)
- Type of feedback (e.g. bug, missing feature, new idea)
- Research methodology used (e.g. survey, interview, user test, secondary research, white papers)
Depending on feedback sources, some facts are automatically well-labeled. For example, every fact that comes from our Student NPS will have the user type “Student” and the language “French”.
As for the tags, everyone can create their own, as they can be linked to a very specific topic (e.g. “dark mode”, “dashboard”). When we see that a tag is frequently used, we can transform it into a property. EnjoyHQ also allows us to merge similar tags so we can easily clean the duplicates (yippee!).
Types of insights
Remember that our insights are what we’ve learned during a research activity. And as I said previously, our main objective is to impact the roadmap and the strategy. It was then really important for us that collaborators can immediately understand the potential impact and the maturity of each insight we deliver.
That’s why we created 3 types of insights:
- Organization insights:
The highest level of insight, as they are of interest to and be used by the whole company. These insights are “macro”, as they can reflect human behavior or market trends. Organization insights are often produced by User Researchers and issued from generative research.
Example: “Large companies need to transform their workforce” - Tribe/department insights:
These insights can feed different squads and sometimes different departments. They are more broadly communicated and will surely be reused by other teams but not as transversal as Organization types.
Example: “Delay in jury create a bad off-boarding experience” - Squad insights:
These insights are specific to the squad’s scope. Squad insights have a short lifetime, as they can quickly be solved. They’re produced by Product Designers and mainly communicated within the squad.
Example: “Filling motivation questions is tough for applicants.”
Trust levels
We also define the “trust level” of each insight we produce. As we tend to do continuous research, we have some insights that are mature and others that are just assumptions. Even if an insight is still “draft”, we want it to appear in the repository so other people can see that someone is working on this subject and can add elements that will complete the research.
We want to be very honest and make sure everyone knows the level of confidence they can have in our insights.
Every Insight has a trust level tag, from “draft” to “reliable”:
Mathieu Klein, our Lead User Researcher, defines the calculation on different criteria:
- Creation date
- Number of facts that supports this insight: Are we talking about 5 people? 5000 people?)
- Number of sources: Do all the facts come from the same raw material of interviews? Or do we have similar facts that come from one interview, a survey, and a comment on the Apple Store?
- Number of contradictory facts: Do we have facts that invalidate this insight? How many?
- Quality of the feedback source: Is this primary research? Secondary? Who did this research and in which context?
How do we use our repository on daily basis?
We have different ways of using our repository. Basically, the process is very similar to the one we use with our Design System: Search first and then update existing content or create new content if needed.
#1. Search for existing facts or insights — Data gathering
When we launch a new research project, after defining our research questions, our first reflex is to search our repository to see if we find answers to our questions.
Most of the time, we find a lot in the repository. To give you an idea, we have 22,000 documents in EnjoyHQ and more than 70 insights!
#2. Collect new facts and upload them into the repository
If we don’t find all the answers to our research questions, we launch a research experiment (remember, it could be a user test, interview, survey, etc.). We upload the experiment into the repository and tag the essential parts to create new facts.
#3 Update insights or create new ones
After analyzing facts, we have 2 possibilities:
1. What we learned reinforces or contradicts an existing insight: We can update it (continuous research).
2. We learned something new: We can create a new insight.
#4. Communicate our insights
Finally, when one insight has reached a good level of trust, we communicate it to the right people, depending on its type (Squad, Department, or Organization). On EnjoyHQ, insights look like small articles (that’s what they call “stories”), so it’s easy to read. We have a template, so we make sure all stories are similar, no matter who worked on them:
How do we maintain our repository?
Collaborative effort
The power of the repository lies in its data: The more we use it and feed it, the more powerful it becomes. Our User Research team is still very small, so we need to train as many people as possible.
We create documentation in Notion on how to use the tool, upload facts, and create insights.
I also add the delivery of insights as a Goal for my team: By the end of the year, each Product Designer needs to create at least one “Story” on EnjoyHQ. It encourages everyone to read the documentation, test the tool, ask for help if needed, and give feedback so we can improve the process.
Knowing other’s projects
We encourage Product Designers to share their research projects in what we call “Design Sharing”. It helps everyone to have a good knowledge of others’ scope and projects so they can improve somebody else’s insight if they find elements during their own research.
Avoid obsolete insights
It’s been one year since we started using our repository and for now, we don’t delete old insights as the volume is still manageable. We rely on EnjoyHQ’s filter to make sure we select a time frame and avoid the oldest. When we would like to do some cleaning, we will filter insights by type and, for example, delete the “Squads” types that might not be relevant anymore as they will have been resolved.
Useful tips we discovered on the way
You can’t plan everything right from the beginning of your journey. You’ll need to start small with a few people to define your process and then open the repository to other teams.
Our first insights were far from perfect. And it was OK. We needed to deeply understand the difference between our classic research experiment results (most of the time, a long Google Doc containing multiple insights) and this new “Story” format.
Few properties
At first, we created a lot of properties, thinking they could be useful. If you say “could”, then don’t! Some of our original properties haven’t been used in the year, so we now have to delete them. It’s better to start with properties you use every day to filter your repository and add new properties only when you need to.
The right amount of automated feedback sources
Your repository will unleash its power when you will have a significant quantity of facts and insights. However, it is easy to become overwhelmed by the quantity, which causes a lot of noise in the repository. So at first, integrate 1 or 2 automated feedback sources (e.g. NPS, Zendesk tickets). Then, for every new source, ask yourself: Is this source relevant and “clean”? Will I be able to do something with the facts that are uploaded into the repository?
Quickly start mapping your feedback sources
Something I would have liked to do sooner was to map all the feedback sources within the company (e.g. Chatbot on discovery pages, Satisfaction surveys after a mentoring session, Taking specific feedback during a call with Customer Success…)?
We’re currently doing this work and it will help us having an efficient continuous research process and better define the sources we want (or won’t) directly upload in our repository. It enable us to understand which team is responsible of what and how we can collaborate better with them to make sure we don’t duplicate the questions.
Our next challenges
Even if I’m very proud of where we are today, we still have a lot of challenges to overcome:
- Continue to train the teams (especially the Product Designers and Product Managers) on insight production. Our next goal is to have a continuous research process in place for every squad.
- Improve the discoverability of our insights on EnjoyHQ. The tool is not always ergonomic, so we have to find more powerful ways of searching the content and finding relevant facts or insights (e.g. better navigation).
- Continue to map the user feedback sources and collaborate more and more with other departments. This will allow us to build a common vision and knowledge of our users’ major pain points or needs during their journey with us.
Our company already has a strong “human” DNA, thanks to Mathieu Nebra who created the OpenClassrooms precursor “le site du Zero” when he was 13 years old with the deep desire to help people and Make Education Accessible to everyone. Our User-centric approach has become increasingly stronger over the years and it’s now the perfect moment for us, thanks to our repository, to go to the next level!
