Insight Author
Garrett Sheridan
Principal UX and Product Designer

More than a feeling: How to improve your customers’ experience by measuring happiness.

Measuring the "fuzzy" subjective aspects of a user's experience is not easy but it may be key to the success of the product. Here are some strategies for making the most of customer feedback.

November 22, 2018
Marino Insights Cover Image

When designing products we use the term “Happiness” to describe metrics related to attitudes. We aim to measure things like perceived ease of use, satisfaction, aesthetic appeal and "likelihood to recommend". These "fuzzy" subjective aspects of a user's experience may be key to the success of the product. Quantifying them can be difficult. If we want to improve we need a system of tracking these hard to measure aspects.

“I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be.”

William Thomson AKA Lord Kelvin

Net Promoter Score

A common way to measure "happiness" is by using Net Promoter Score (NPS). Many companies use this number as their key happiness metric. It comes from responses to one question.

"How likely is it that you would recommend our company/product/service to a friend or colleague?"

Researchers discovered that scores on this question correlated with actions contributing to growth. This question was a powerful behaviour predictor based on several case studies.

How is NPS calculated?

Survey respondents that give a score of 9 or 10 out of 10 are called promoters. They exhibit behaviours that are beneficial for the company. Those who respond with 0-6 out of 10 are detractors. They are less likely to exhibit value-creating behaviours. Responses of 7 or 8 are passives. They fall between promoters and detractors. Subtract the percentage of detractors from the percentage of promoters to calculate NPS.

Not a silver bullet metric

It feels like a simple method for measuring happiness but the score has its critics. Even it's creator, Fred Reichheld says he is sick of it. Another high profile detractor is experience design guru, Jared Spool. He makes some convincing arguments why NPS is harmful. Here's a summary of his key criticisms;

  • There is a "Wacky science" behind the numbers.
  • It’s not based on the current situation
  • It’s easy to game
  • Low response rates are not indicative of actual sentiment
  • People rating low when they can't think of anyone to recommend the product to.
  • “Best choice” is not the same as “delightful service.”
  • There’s no one number that represents a company’s customer experience

While useful for benchmarking, the number alone may be of little practical use. To understand our users' happiness we need to go beyond a single number and dig deeper. Spool believes the real value of NPS is in the post-rating, follow-up question. This is usually something like “What is the primary reason for your score?” This question gives us a valuable opportunity to listen to customers' concerns. With this information, we can begin to devise solutions to solve their problems. We can go beyond the number and start to understand “the why?”.

Beyond NPS

There are many ways to learn about how your product satisfies your customers. NPS verbatim responses may be a good start but what else can we do to measure happiness?

Social media sentiment and app store comments

Catalogue positive and negative posts and ratings. Score them based on sentiment. When compiled you will have a number you can track over time.

Customer Satisfaction Surveys

Use these to measure how customers feel about their experience with your company. Use the data to segment customers based on satisfaction scores. Measure these scores over time. Use them to find insights that become catalysts for improvement.

Customer Support Logs

Customer support reports are a key resource for uncovering our customers’ problems. Focusing on tackling these issues can have a two-fold effect. An increase in satisfaction with a reduction in support costs.

Customer Effort Score Survey

Measure service satisfaction from responses to this statement...

“The company made it easy for me to handle my issue”.

The customer chooses an option ranging from “Strongly Disagree” to “Strongly Agree”. You will enhance customer loyalty by reducing customer effort.

User Interviews, observation and user testing

Face to face time with customers lets us see first-hand where they are having problems. This gives us the opportunity to dig deeper with follow up, clarifying questions. There is no substitute for seeing the look of frustration on a user’s face as they try to interact with our product. Share recordings and quotes from these sessions to create a compelling case for which issues to address.

Use a framework

You need a small set of key, meaningful metrics that everyone on the team cares about. A useful tool for thinking about these metrics is Google’s HEART framework. The framework breaks down what we can measure into five categories. H (Happiness), E (Engagement), A (Adoption), R (Retention), T (Task Success). Each category is subdivided into three sections. Goals, Signals and Metrics.

The HEART framework in action

Let’s run through a quick example. First, let’s define what our happiness goal is. For this example, we will say our happiness goal is:  "User satisfaction. We want our users to be satisfied with their experience".

Next, we map our goals to appropriate signals that will help us to understand if we are meeting them. In this example, a customer satisfaction survey may be a good way to collect the data we need.

Finally, after we have chosen what our signals are we can then begin to record metrics over time. We can see how these numbers move as we tweak the product. Having these numbers available also allows us to A/B test potential solutions.

Break it down

Don't try to tackle the whole product. Focus attention and set goals on smaller parts of the experience. Target specific tasks or stages in the customer lifecycle to pinpoint issues. This makes it much easier to devise strategies to improve.

Conclusion

Despite its flaws, NPS is a common tool for measuring happiness. It will remain a key metric for many organisations. We can use NPS with data from other sources to inspire potential solutions to common pain points. Triangulation between several methods will lead to a deeper, actionable understanding.

Using a framework like the HEART framework can help you to establish the goals you want to achieve and decide which signals and metrics are appropriate to gauge progress.

Using these methods can help you build a rich picture of your customers' pains. These pains can become opportunities to innovate. It is important to treat this as an ongoing, iterative process for improvement. Customers' expectations increase over time. It is important to keep up with these expectations.


!@THEqQUICKbBROWNfFXjJMPSvVLAZYDGgkyz&[%r{\"}mosx,4>6]|?'while(putc 3_0-~$.+=9/2^5;)<18*7and:`#

More from the blog

More blog posts