Unconventional Techniques for Better Insights from Satisfaction Surveys

UX Strategy with Jared Spool, a Center Centre — UIE newsletter focused on bringing UX to a strategic level inside your organization.

The difference between subjective and objective data.

At a recent conference, I heard a speaker explain to the audience that qualitative data (data collected through methods such as usability testing) was always subjective while quantitative (data from surveys) was always objective. Unfortunately, this commonly held belief isn’t quite true.

Do you “Like” doing business with us?

In essence, the organizations sending us those feedback surveys are trying to learn if we liked interacting with their products or services. If we did, might we want to do it again? If we didn’t, how might they improve the experience?

“Like,” by itself, is not helpful.

The problem is, it isn’t helpful just knowing if customers like us or not. It doesn’t tell us what to do differently. If everyone likes us, we don’t know what they like about us. If we make changes going forward, we might wreck the thing they really like about us.

The benefit of matching up behavioral data.

What if we combine the question “whether a user likes us or not” with the data we have on how they use our product or service?” (We don’t want to ask the user if they’ve used it — that’s still subjective — instead use built-in analytics of actual usage data.) For example, an airline could match up customers’ opinions of the airline’s service with whether those customers have flown on any recent flights.

Measuring the disappointment of loss.

Recently, we’ve come across a couple of exciting enhanced approaches to subjective satisfaction data. The first one comes from investor Sean Ellis and it’s what he calls his Product-Market Fit question.

Enhancing the question by collecting more data.

Rahul Vohra, CEO of email product Superhuman, took Sean’s question and enhanced it to give his team more information to work with. His enhancement is a short, four-question survey that starts with Sean’s disappointment question:

Observation is still the best way to learn.

There are improvements we can make to subjective satisfaction scores, like shifting towards something akin to Sean and Rahul’s approaches. However, we’ll always be limited by trying to do everything through surveys.

Maker of Awesomeness at @CenterCentre/@UIE. Helping designers everywhere help their organizations deliver well-designed products and services.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store