Net Promoter Score (or Not a Perfect Solution)
If you run a quick Google search for “NPS,” you’ll find a plethora of articles advocating for the use of it in pretty much every company. Marketologists love it; senior leadership is over the moon about it. But is it really that useful?
Disclaimer: this article is based on my personal opinion and experience and backed by a few studies (and common sense). I see some benefit in NPS when it comes to showcasing how great you are as a company to investors, but, as you’ll see below, not much beyond that.
NPS is a highly popular metric to assess what stakeholders like to call “brand health”. A lot of times, it’s confusingly referred to as a “customer satisfaction” survey or even “customer experience” survey.
“Customer experience” in recent years has generally become a buzzword, devoid of any real meaning. Every snippet of information you get from a customer is classified as “experience” and every useless metric is believed to be measuring it.
NPS in partuclar is a simple one-question survey measuring the likelihood of users or customers to recommend the service or product to their friends or colleagues. It’s based on an eleven point Likert scale and divides customer responses in the following way:
As you can see, unlike traditional scales, NPS is not attributing positives and negatives equally. In fact, anyone below 7 is considered to be a risk (a risk of abandoning the product/service usually). Only those who select a satisfaction of 9 or 10 are considered to be “positives”, while 8 and 7 become neutrals. Regardless of the fancy terminology (yes, “detractors” does indeed sound much fancier than the normal “dissatisfied customer”), NPS is more or less your usual “To what extent are you satisfied or dissatisfied” but with a more elaborate calculation behind it.
Know Your Customer or Know Your Industry?
So what’s really the problem here? I’ll start by making a point about industries. NPS is currently used by all types of businesses, covering all types of services and products. But are all industries equal? Do they all enjoy the same level of satisfaction? Simple common sense would tell us no; this can’t be the case.
First and foremost, each industry has a benchmark of expectations from its customers. Expectations change as the industry evolves, and they tend to only rise with time. Think about your experience with a bank. Twenty years ago, face-to-face (or in-branch) customer experience was one of the most important metrics to assess satisfaction with a banking brand. It is not so anymore. App and online experiences now play a much larger role (just look at the popularity of challenger banks!), especially among younger audiences. But do you expect the same things from your bank as from, say, a car rental company? I highly doubt it.
Academics have come to the same conclusion with Uppsala Universitet study telling us that:
“…the loyalty behaviour could be manifested due to multiple industry-specific factors which are irrespective of company’s internal performance and satisfaction”
Let’s look at a particular example. Studies have been conducted to assess an average level of satisfaction across various industries in the US. Restaurant industry, for instance, is enjoying a particularly high average level of satifaction of 80, while Internet Service Providers can only boast a humble number of 64. That’s a 16-point difference between the two largely popular industries. Try explaining to your potential investor how your low (by NPS standards) score is just an average across the industry.
In reality, the explanation of the discrepancy is rather logical and doesn’t require going through heaps of academic data. Your level of satisfaction with a product/service can be roughly coded as:
(Initial expectations + Complexity of industry + Already existing experience with the industry)/ How well expectations were met = Actual satisfaction
I’m greatly simplifying the actual model to make a point here as satisfaction is also embedded into cultural differences (have you heard of the French being famous for complaining about literally anything?), personal experience, the mood at the point of filling out the satisfaction survey, and other tons of factors, all of which are impossible to list here. For now, let’s just focus on the ones I highlighted in the formula.
Are you the type of person that can’t wait to speak to your insurance provider about the claim you’ve made a month ago? Yeah, I didn’t think so. Certain industries, like insurance, are infamous among even their own customers. In most cases, the industry is viewed as a necessary evil and 90% of your communication with the insurance provider would be during unpleasant or even plainly stressful times for you as a person, let alone the fact that the industry as a whole could be (rightfully so) viewed as predatorial. How likely are you then to have positive image of your insurance provider even if they are doing their job reasonably well?
Our brains are wired to consider multiple different sources of information and we rarely, if ever, base our opinion solely on one factor (even when we believe that to be the case). Despite the fact that judging one particular company on the basis of the whole industry isn’t always fair, it is exactly what’s happening.
When diversity is not just a buzzword
Cultural differences — we hear so much about these that it’s surprising to see that NPS has not yet been affected by any re-evaluation within that field. Despite forming one economic and political union, even countries within the EU differ greatly when it comes to how they express their satisfaction or dissatisfaction.
A 49 country study published in 2022 looks at the differences in positive and negative societal emotional environment by country. The chart below shows exactly how big those differences could be. For example, United Kingdom scores low on both positives and negatives, while Ghana remains the most positive out of the sample.
What it means for NPS is that whenever we’re trying to run cross-country comparison of what has been promised to us as a universal score, we are comparing apples to pears. What can be a particualrly damning score for Ghana, might end up being a rather positive one for South Korea. This presents a rather peculiar problem specifically for international companies, where the NPS score might differ greatly across different markets, which is hard to explain to a layperson who might not be interested in methodological concerns around NPS.
What’s behind the number?
In the course of my career both as a market/consumer and as a UX researcher, I’ve heard this phrase and variations of it at least a billion times:
“I’m not giving it a rating of 10, because there is always room for improvement”
I’m not going to argue whether it’s a universal feeling as we’ve seen that cultural aspect comes into play, but what I know for sure is that participants in the UK are quite unlikely to give the highest rating even when perfectly satisfied. In classic Likert scales where each number usually has a verbal association (e.g. 10=very satisfied) and positive and negative values get netted, it’s not usually a problem. It is a problem, though, when it comes to the NPS.
An average person doesn’t think of 8 or 7 being a bad (or “passive” as NPS says) rating. It leaves something to improve on but it’s not necessarily associated with major problems. And how about rating of 5? Traditionally, most people believe five to be a neutral option where you are not experiencing any issues but you are also not over the moon about the product or service. It doesn’t mean you’re about to switch your loyalties but might simply mean you’ve not been “wowed” by the experience with a company.
The meaning assigned to each NPS value seems almost arbitrary to a layperson. It’s also not something that is explained to potential participants ahead of time, they are just presented with a rather standard scale and asked to provide an answer. In fact, a lot of studies looked at how well NPS score transfers into financial performance or other company metrics. Unfortunately for NPS fans, the correlation is proven to be pretty low. For example, Cambridge University study shows that “…NPS measurement does not necessarily correspond to actual behavior.”
Why not, though? People are telling us they would definitely, 100%, absolutely recommend our company to others; why wouldn’t that translate into higher revenue, customer loyalty or subscriptions? Well, for once, most companies offer more than one standalone type of product or service. Personally, I am a big fan of one particular skincare brand. Almost all my skincare products are from them. However, there is one very particular product that I bought a while ago and absolutely hated. Did I start to hate the brand or became a “detractor” in NPS language? No, I’m still a big fan. But I also absolutely discouraged my friends from buying that one product I disliked. So, what we have here is that overall I’m a promoter but my behaviour is contradictory — I’m both recommending and discouraging people from buying from the brand, which in NPS world is an impossibility.
It gets even more complex when we think about demographics. My likelihood of recommending something is not based solely on me liking what the brand has to offer. It also needs to match what I know about the person I’m recommending a product to. For example, I’m very happy with my choice of transitioning from Apple products to Samsung, but I’d be highly unlikely to propose the same change to my mother, who is bad at technology and has been using Apple products for ages. Cost concerns are another important factor — I might be perfectly happy with my 10 year old Fiat Panda, but would I really recommend it to a person that can afford to buy a new Mercedes?
Overly simplifying human behaviour leads to bad data. No human being is as simple as having one number perfectly explaining their future behaviour.
Questions Behind the Question
The classic NPS question asks you the likelihood of recommending a product/service to your friend or colleague. The idea behind it is that only loyal and content customers would risk to provide recommendations to people close to them, thus NPS is playing a bit on the emotional side of respondents to get a truthful response out of them. In its essence the thinking is more or less legit — the higher my satisfaction is the more likely I am to pass the good stuff to my close ones.
In reality, it poses more questions than answers. First of all, there is a huge difference between B2B and B2C companies. B2B companies, in particular, would inevitably struggle with translating NPS data into reality. This stems from a simple fact that selling a product to a company is largely different from selling a product to an individual. First of all, there are much more people involved in the decision making in B2B than in B2C. People involved in making actual decisions might not even be the people experiencing the actual need for a product (think Heads of department vs normal employees). On the other side, your recommendation as a user might not have any effect on others if others don’t experience the same needs as you, don’t have the same budget, etc. In a nutshell, for a B2B recommendation to work much more conditions need to be met, which is rarely the case.
But it’s not only the type of company that is a problem. On a high level NPS raises the biggest concern in its overall usefulness. So you found out that a large chunk of your customer base is “detractors”. Too bad for you since NPS doesn’t give you any indication as to why this is the case. Some researchers try to rectify this by adding an open-ended question asking customers to specify why they gave a particular score. This is not a bad idea in itself, but as researchers, we all know that open ends rarely provide a high level of insight, let alone the fact that no one wants to spend hours every 6 months or so deciphering largely unhelpful text data when there are other much better ways to find out the same information in a more systematic way.
To NPS or not NPS, that is the question
I might be overly stern but as a researcher and not a marketing professional, I see almost zero value in NPS. At the very best, it’s another arbitrary, meaningless metric that your company can slap onto its website or distribute as part of their pitch to investors. At worst it’s misleading and leads to or overly positive or overly negative outlook on your company’s health, none of which in all likelihood will translate into real life behaviour on the part of your customers.
There are various ways in which you could meaningfully track customer experience, but it will always require much more than just one question. I’m generally not a big fan of one-size-fits-all approach, so my recommendation would be to tailor your customer experience survey to the needs of your company. Standardised CSAT surveys are a good way to start but they’d still require customisation from your side. Customer experience is a broad area and as an in-house researcher you’d know best what metrics are important for your company and customers. Don’t be misled by a fancy name and methodology. Sometimes what we need to do is go back to the basics and ask ourselves a question: what do we not know about our customer’s experience?
References and Further reading
- Devesh Gadkari — “Factors Influencing the Net Promoter Score”, 2018
- Mohamed Zak et al. — “The Fallacy of the Net Promoter Score: Customer Loyalty Predictive Model”, 2016
- Kuba Krys et al. — “Societal emotional environments and cross-cultural differences in life satisfaction: A forty-nine country study”, 2021
- Nicholas Fisher — “Good and bad market research: A Critical Review of Net Promoter Score”, 2018
- Douglas Grisaffe — “Questions about the Ultimate Question: Conceptual Considerations in Evaluating Reichheld’s Net Promoter Score”, 2022
