7 Common Mistakes in Interpreting Analytics Data: Statistical Pitfalls for Your UX Team to Avoid

All sorts of highly likely confusions, data taken out of its context, “obsessing over” numbers, approaching analytics with no clear goals in mind, metrics subjected to your own biases... We're all prone to making mistakes when analyzing data. Still, as a UX team striving to pull off an accurate picture of the user behavior, you need to take note of the most common mistakes in interpreting analytics data (UX analytics). 

… of the biggest "gaffes" in reading data.

Those responsible for all the wrong assumptions about your users that you'll end up making:

 

• that low numbers are always a bad sign
• that if results show a correlation, there is definitely a causal relationship, as well
   

... and so on.

Now, allow me to “expose” to you the 7 most common mistakes that one can make when interpreting statistics:

 

1. Visits and Views: Confusing Them and Obsessing Over Them

Using these two notions interchangeably is a pitfall that not only rookie data analysts fall into:

With different UX analytics tools using different terminology for the very same concept and (even) confusing terminology used within the same tool, no wonder that you end up taking views for visits and vice versa.

And still: make sure you fully understand the terminology, otherwise you risk to:

 

• report on the wrong data
• put together some dangerously inaccurate reports
   

This is, no wonder, one of the most common data interpretation errors.

Now, let's define views and visits and present them as two different concepts once and for all:

 

• a view (or “pageview”) refers to a view of a page on your website tracked by the analytics tracking code
• a visit (or “session”) refers to a user's whole of interactions taken on your site, within a specific time frame

 

And now, speaking of views and visits, another one of the too common mistakes in interpreting analytics data is:

Obsessing over views and visits!

As a UX designer though, you may want to leave the challenge of increasing visits and page views to the marketing people in your team to handle. And, instead, to focus your efforts on that data that 's relevant to the user experience.

 

2. Settling for a Birdseye View Instead of Digging Deeper into Data

Scratching the surface of the available data:

 

• a quick assessment of the data at hand
• rapidly going over the “headline”  figures
   

… will only tell you something about your website's current performance in terms of traffic, but won't give you any clue on how to improve UX. How to increase the conversion rate.

In other words: visits are no more than metrics signaling you how many visitors landed on your site during a given period of time, but this metrics won't reveal anything about how they actually engaged with those visited pages.

See? Analyzing data as broadly as considering sessions to be the key indicator of performance and UX is another one of those common pitfalls in interpreting statistics:

By far the best method of reading analytics data, as a UX-er, is to approach it with some well-defined goals in mind. This way, you'd focus your efforts on specific metrics, relevant for understanding user behavior, instead of getting yourself “drown” in a sea of data.

 

3. Common Mistakes in Interpreting Analytics Data: Not Looking Beyond Numbers

… and not putting them in their contexts.

For that's the proper way to interpret them. Otherwise, you're just... analyzing quantitative data stating the obvious:

The “what” and not the “why”.

This is undoubtedly one of the most common mistakes in interpreting analytics data: falling under the “spell” of numbers!

Instead, you'll need to keep in mind that:

 

• it's real users that those collected numbers represent
• once taken out of their contexts, numbers lack their true value
• they become truly valuable only when interpreted in connection with the user experience:
   

What do they tell you about the overall user experience on your website?

This is why you should always apply qualitative methods when analyzing quantitative data. User research methods that enable you to go from “what has happened” to:

“Why is it that visitors behaved that way on my website?”

 

4. Always Taking Low Numbers for a Bad Sign

Another one of those more than common mistakes in data analysis is:

Always thinking that low or a drop in numbers is a bad thing.

Context is everything here!

Just think of reading data analytics as a three-phase process:

 

1. what you want to see in those numbers
2. what the available data seems like
3. what it really means
    

Let me give you one good example:

Less time spent on a web page could be good or bad. If we're talking about your redesigned homepage, it could very well mean that users do find its new design more intuitively efficient. That they can get to the pages on your site that they're interested in far more easily.

In other words: do put those drops in numbers against their contexts before you alert everyone in your team that the site's going down the hill!

 

5. Overlooking to Segment Users

For you surely agree that every given visitor uses your website differently: 

 

• on desktop 
• on mobile
• at different times of the day 
   

And that multiple users interact differently with your site.

Need I say more?

Don't overlook these valuable considerations on your users' behavior when interpreting your quantitative data.

Before you rush to make all the wrong assumptions reading your analytics data, make sure you break those figures down into multiple relevant segments:

 

• mobile users
• desktop users
• users from different countries
• users falling into different age groups and so on
   

It's user base segmentation that turn quantitative data into... relevant data. And which, most importantly:

Provide you with priceless clues regarding the areas on your site that you should be focusing your UX efforts on.

Let's just say that your site has a conversion rate of 7%. Before you get overexcited about it, make sure you break that figure down. You might just discover that 9% comes from your desktop users and only 1% from your mobile users.

And there you have it, there's your clue! Now you know just where to focus your UX efforts.

 

6. Not Setting Clear Goals Before Approaching Your Analytics

And, as already stated, this could get you “tangled up” in a huge amount of data.

But, if you take some time to define your goals first things first, you'll know just what you'll want to achieve from your data analysis session.

And to:

 

1. direct your UX efforts towards those specific objectives
2. focus exclusively on those metrics relevant for interpreting user behavior
    

If you don't know where you're heading, how can you know just how to get there; how to improve UX on your website?

 

7. Settling for a One-Size-Fits-All Reporting Setup

Another one of those common mistakes in interpreting analytics data is sticking to a standard reporting setup.

That instead of trying to custom-tune it so that it should deliver you precisely the data you need. The one relevant for your own website.

Since each site works differently, you can't expect a one-size-fits-all approach to data analytics to perfectly suit them all, in the slightest details, now can you?

 

So, You've Analyzed Your Data: Now What?

For reading your analytics data is just the first step. Now it's time you:

 

• get some actionable takeaways from your analyzed data
• get to action
   

Are there usability tests that you need to run to figure out why the conversion rate is higher on your desktop site than on its mobile version?

Or maybe you need to implement some user research methods to identify those contexts where users visit your site from their mobile devices?

Time to put together your “data-fueled battle plan”!