Turn Qualitative Data Into Quantitative Data & Use the Insights in Experiment Ideation

Let’s talk about qualitative data today.

It doesn’t get enough love and quantitative always gets more. I want to give you a proper way to analyze qualitative data and then show you examples of how to use it for experiment ideation (since I’m a researcher and an experimenter).

What is the difference between quantitative and qualitative data?

We all know there are two types of data: quantitative and qualitative.

Quantitative: “relating to, measuring, or measured by the quantity of something rather than its quality” (Source)

Qualitative: “relating to, measuring, or measured by the quality of something rather than its quantity” (Source)

The TLDR in my head…
Quantitative = numbers
Qualitative = words

How do people usually analyze qualitative data?

Quantitative is typically easier for people to analyze. Processes for this are fairly consistent across teams. It’s just numbers.

Qualitative is usually the wild west as far as how people analyze it. Many people don’t know what to do with it or how to analyze it properly. They almost never know that you can turn it from qualitative to quantitative either.

You might ask, “Well why would you want to convert it? Wouldn’t you just collect quantitative data in the first place then?” Nope.

Quantitative data usually gives us the “what” of something. Qualitative usually gives us the “why” of something.

Example:

• Quantitative data point: The exit rate on the homepage is 82%.
• Qualitative data point: Users don’t understand the value proposition or benefits.

Most people’s approaches to analyzing qualitative data: (1) word clouds or (2) reading through responses and taking rough notes.

Neither of those is great. Not very methodical, right?

Word clouds are not always great (source)

Boom. Here’s where we want to turn it from qualitative to quantitative. It’s much more methodical, reliable, and repeatable.

Process to turn qualitative data into quantitative data

I’ll share with you the process that we also follow at Chirpy. All you have to do is start a spreadsheet. This is where we’ll complete a coding process.

1. Put all of your responses on the left side in one column in individual rows. (Usually, 200-300 are ideal so that you reach statistical significance.)
2. Read responses one at a time. For each one, determine one or more themes.
3. Add those themes to the top in one row in individual columns as you go.
4. For each response, add a “1” under each applicable theme. More than one theme can be counted per response.
5. After you’ve completed this process for all responses, get the sums for each theme.
6. Pull the entire list of themes and totals. Sort them from highest to lowest.

That’s it!

Quantitative Data Analysis

Pull out some of the top quotes

Optional but recommended: You can pull out some of the top quotes along the way, too. That’s always helpful.

If you want to take it a step further, do a second round of coding.

Put the first round of themes in the far left column this time and try to boil them down to no more than 3-4 themes in the top row. (Know that outside of the example above you’ll likely have several more themes to analyze, which would make a second round possible.)

What do you do with these kinds of insights?

Now that you have an amazing, prioritized list of customer insights, what do you do with them?

All kinds of things!

Examples:

• Turn them into experiments on websites, mobile apps, or products
• Use them to inform pricing changes
• Use them to inform additional research
• Use them to inform your sales workflows
• Use them to inform your customer service workflows
• Use them to inform marketing campaigns
• Use them to inform your value proposition and creative
• So on and so forth…

Let’s talk about experimentation & how to use these insights for experiment ideation

Because I’m an experimenter, I want to show you how this type of research informs test ideation. I’m constantly telling teams where there is experimentation there should always be research.

Experiments should not be willy-nilly based on “I think,” “I feel,” or “I want,” which all lend to subjectivity and your personal opinions. They should be more data-driven than that. And where does the data come from? Research.

Let’s take the number one theme from the example above: pricing is too high. This is almost always in the results. You must take it with a grain of salt to a certain extent because people always want lower prices and discounts.

However, know that aside from lowering your pricing (which you shouldn’t consider, at least in most cases), we can do other things around this.

If pricing seems too high, we need to make sure the perceived value is higher than the monetary costs. Benefits and social proof can help with that so perhaps we reinforce those in some way.

Check out GrowthMentor’s pricing page, for example.

Pricing is followed up by extremely adequate social proof with motivating copy, a carousel of mentors, and reviews. Boom.

So…

If one of your themes is that pricing is too high and your pricing page doesn’t have any social proof or reinforcement of your value, try adding some in an experiment. Test it out. See if it helps performance. In theory, you should get more signups. Down the road, you could always complete more qualitative research to see if the theme still pops up.

Here’s an eCommerce example:

Olaplex pricing example (source)

Social proof and benefits:

• Star review
• # of star reviews
• Bazaar award
• Social video
• Banner with info

Now, repeat this cycle over and over!

Data → test → data → test…

Actionable takeaways

Now that you’ve read this, I’m begging you to please go do the following to help your digital performance:

• Gather qualitative data through any methodology that allows you to do so (e.g., poll, survey, interview, user test, etc.)
• Analyze it through the coding process used here (at least one round, if not two)
• Turn your insights into test ideas
• Repeat through doing more research and tests (using the same and other research methodologies)

Important notes: Know that the more data points to back something up, the better. Pair your qualitative data with your quantitative data.

Happy researching and testing!