In your journeys across the web, you probably have encountered the now ubiquitous question “How likely are you to recommend this website to a friend?” This is the prompt behind a popular customer-loyalty metric known as net promoter score.
Definition and Calculation of NPS
The net promoter score (NPS) is a quantitative metric that measures how many more people are likely to strongly recommend your site or product compared to those likely to criticize it.
NPS is determined by asking people to provide an answer, on a scale from 0–10, to the question: “How likely are you to recommend this website/product/service to a friend or relative?” The answers are then grouped into 3 categories:
- Promoters: responses of 9 or 10, which indicate high satisfaction and strong likelihood of recommendation
- Detractors: responses of 0 to 6, which indicate dissatisfaction and likely criticism
- Passives: responses of 7 or 8, which indicate moderate satisfaction, but low likelihood of recommendation
The NPS is then calculated by subtracting the percentage of detractors from the percentage of promoters:
NPS can range from -100% (only detractors) to +100% (only promoters). A positive score indicates that the promoters outnumber detractors, while a negative score shows poor customer loyalty, with detractors outnumbering promoters.
Note: The passives are included in the total number of respondents, but do not contribute to the score otherwise. The rationale is that these users may feel that their needs are fulfilled, but will not actively promote the product or service with family or friends.
Origins of NPS
Frederick F. Reichheld, a business strategist and author of the bestseller The Loyalty Effect, first introduced the concept of NPS in Harvard Business Review in 2003. In that article, he described a 4,000-respondent survey in which he asked several questions and tracked how well the answers correlated with a number of different measurements, including repeat purchases and recommendations to friends or family. He found that the question that best predicted customer behavior was the one associated with net promoter score. NPS also strongly correlated with company growth over time.
Reichheld argued that the net promoter score is relevant because customer recommendations and word-of-mouth referrals are a direct driver of revenue growth in many businesses. A good NPS indicates a loyal customer base — arguably, a company’s greatest marketing asset.
International Differences in NPS
From a mathematical standpoint, you might expect that the average website rating be 5; so, it may seem harsh to limit promoters to scores of 9 or better and to count a 6 as a detractor, even though 6 is above the mathematical midpoint of the scale (5). However, these cutoff points are reasonable because raters generally tend to be generous and give fairly high scores. That said, a “good” net-promoter score varies depending on whom you ask — or, more accurately, where you ask.
NPS Value Varies by Country
A 2021 study by Qualtrics XM Institute, which surveyed 17,509 consumers across 18 countries, found that responses ranged widely across countries. Respondents were first asked to reflect on companies they liked, and whether they would recommend those companies to friends or relatives. The same was asked regarding companies they disliked. This method yielded two NPS scores for each country’s raters: a “high” NPS (for recommending liked companies) and a “low” NPS (for recommending disliked companies). There were significant differences in “high” ratings across countries. For example, in India and Mexico NPS scores for liked companies were 60% or greater. However, raters in Japan and South Korea gave negative (detractor) NPS (-47% and -11% respectively) even when reflecting on liked companies.
Similarly, differences between “high” and “low” NPS scores also varied greatly: the gap was largest in Brazil, and smallest in Japan. This finding suggests that in some cultures, people tend to choose more neutral ratings, and choosing extreme options may be culturally taboo.
Scoring Rationale Differs Across Countries
Another study by SurveyMonkey, which focused on whether companies “deserved” their NPS scores, also found differences in scoring rationale. Raters in the United States often cited “quality” as a key driver for their rating, while raters in the UK cited “ease of use,” and those in the Netherlands cited “innovation.” Thus, some rating differences may stem from differing service expectations within each country.
Should NPS Be Adapted?
These recent findings have spurred debate on whether NPS should be adapted to reflect the rating tendencies of each country’s respondents (e.g. lowering the thresholds for “promoters” in the Netherlands, where ratings are notoriously lower than the global average).
While the criticism of a W.E.I.R.D.-centric (Western, educated, industrialized, rich, and democratic) model may be merited, the high likelihood of variation also holds true when comparing average NPS between different industries, and even between different segments within the same industry. Changing the score calculation (for whatever reasons) risks a misrepresentation of data and may tempt teams to “bend” the metric to create more suitable outcomes. Instead, we recommend that you stick to the standard computation method for NPS, while simply acknowledging the contextual differences, as well as reporting the actual rating score (as an average and margin of error) and tracking changes over time.
Limitations of NPS as a Usability Metric
1. NPS does not capture the full picture when used in isolation.
Usability is never entirely captured by subjective scores. We’ve seen many users struggling to complete a task, yet rating a website as highly as someone who had no difficulty whatsoever. To get a complete picture of the user experience, we recommend that you also collect behavioral metrics such as task success rates and task times.
NPS, like all quantitative metrics, tells you how the experience is perceived but not why. Asking customers to report the reason for their rating might give you some insights, but self-reporting is rarely reliable and the user might not bother investing the time to explain the rating altogether.
2. NPS is relevant only with a large enough sample size.
NPS scores (like any metrics) are rarely relevant with a small sample size. Running a qualitative user test with 5 users and asking for the NPS at the end is unlikely to provide you with any valid data, yet many practitioners insist upon reporting these measures with small samples and they base design decisions on them, disregarding their lack of statistical significance.
3. By “binning” responses, NPS ignores important information.
While other satisfaction metrics also suffer from the disadvantages mentioned above, NPS has another major problem: its calculation method ignores a lot of the information provided by the ratings by grouping them into three bins (promoters, passives, and detractors) and then ignoring the passives. As a result, researchers must drastically increase their sample sizes in order to gain any statistically relevant information.
For example, if we could change a design from getting mostly scores of 2 (truly hated) to getting mostly scores of 5 (somewhat disliked), we would have made a major UX improvement, but all of those users would still be counted as detractors in NPS terms, even though they had changed from rabid detractors to modest detractors.
4. Customer loyalty is not identical to usability.
NPS is best used to assess overall customer loyalty and, indirectly, brand perception.
There are many examples of how UX design decisions impact brand perception — for example, the specific location of a logo on the web page impacts whether users remember the brand at all, or the tone of voice employed on the website changes users’ inclination to recommend the company behind the site.
That said, there are many more aspects to customer satisfaction and brand recommendations than interface design. Pricing is one obvious variable: if people feel that something is overpriced for what’s delivered, they are unlikely to recommend it, no matter how much they like an experience. Also, if users did not have a choice in the interface they are using (due, for instance, to legal compliance or mandatory use, such as with enterprise applications or operating systems), they may not feel a recommendation is relevant, even if an experience was generally positive.
So, while loyalty and usability are well correlated, they are not exactly the same.
5. Customer loyalty cannot capture satisfaction with specific parts of your designs.
Again, NPS is best used to assess overall customer loyalty to an entire company or service, or at least with an entire product.
It makes less sense to utilize NPS to assess users’ satisfaction with granular details of UI design, such as a website’s checkout process, a product page, or a specific dialog box or icon. Yet these local design elements are often what we can change in everyday design projects.
Let’s say that you improve the writing on some web pages or that you improve the usability of the icons in a mobile app. Don’t get too disappointed if the company’s overall NPS remains flat. It’s hard for a single localized design decision to move the needle much on overall customer loyalty. That’s why we need to supplement global NPS scores with lower-level measures of user satisfaction and task performance.
6. NPS is often gamed.
Many teams turn to NPS as a way to benchmark an experience and incentivize iterative design, which means this metric becomes a focal point of any design improvement. While the intention behind using NPS as a benchmark is positive — it is a good thing to improve loyalty — as Campbell’s Law states: the more important a metric is in social decision making, the more likely it is to be manipulated. In other words, it becomes more important to find ways to make the score better, as opposed to making the experience better.
For example, a call-center representative may be tempted to only mention the post-call survey after a customer appears to have a positive experience (and not in cases of negative experiences). This behavior could lead to a sampling bias where only positive ratings are collected. Or, users can be pressured, coerced, or bribed into giving a good rating (a practice known as “rating racketeering”), such as when shoppers are offered a coupon or free gift in exchange for providing a positive rating, or when an employee casually mentions that the survey helps them keep their job. All of these may artificially inflate ratings, or at the very least, will impact their validity.
Why Should You (Still) Use NPS?
Despite these limitations, NPS is still valuable as a UX metric for a number of reasons.
- It’s well-known and liked by upper management for its strong correlation to profits and because it is a quantifiable measurement of something as nebulous as customer loyalty.
- Even though it is not identical to usability, it is still closely related to the perception of user experience. In particular, scores on elaborate satisfaction questionnaires such as the System Usability Scale (SUS) are well-correlated with NPS. So, if most of your customers are reporting high loyalty to the point of putting their own reputations on the line to recommend your site, chances are that your site is also usable.
- It’s fairly easy to collect. Unlike other more complex instruments, the NPS is based on a single question, so users will be more likely to respond to that one question than to a lengthy survey.
As a result, it’s become customary to include the NPS in user interviews, surveys, or even usability-testing sessions. UX practitioners often use it as a tool to promote buy-in from their company’s senior leadership. Numbers that demonstrate increased customer loyalty (and consequently, future sales and profits) after investing in the UX process are more likely to sway skeptical managers and executives than qualitative data alone. Thus, NPS can be part of a series of metrics to measure how a redesign affected loyalty. By quantifying the site usability and loyalty before and after a redesign, companies can assess whether the redesign was worth it and whether it brought enough return on investment (ROI).
Conclusion
When used by itself, NPS, like any subjective metric, is fairly limited, variable in different geographic and industrial contexts, and far from being a good summary of the overall user experience. However, when combined with other UX metrics, NPS can help you track the overall perception of your user experience over time.
References
Frederick F. Reichheld. 2003. The One Number You Need to Grow. Harvard Business Review.
