Trends Reports are Overvalued

If you’re an e-commerce company, you may be glancing at weekly trend reports of key performance indicators (KPI) like the conversion rate, average order value and bounce rate to make sure the company’s site is performing as expected. Trend reports like these are great for learning top-level information and this makes trend reports a popular industry go-to for site performance analysis.

But looking at these types of reports is akin to glancing at your dashboard gauges while driving. You’re just taking a second to make sure you’re driving at the speed limit, the car’s not overheating, and you’re not running out of gas.

If you are looking for opportunities to improve user interaction and site performance, or to learn how a particular design change impacted user behavior, basic trend reports are painfully inadequate. They simply do not tell enough about the numbers in the dataset.

To really understand user behavior, e-commerce companies need to take the same data that would be used for a trend chart and present it in a different way to generate more value.

Here’s an example: Say a major bicycle retailer has a forums section that they’ve determined is critical to increase loyalty rates and generate incremental monetization revenue in a tough economy. Here’s a trend report for two of the forum’s KPIs:

This trend report gives us a general idea of how many topics in the forum are visited and how many messages are posted. It also shows us that while the metrics fluctuate from week to week, they tend to be in the same range.

This report might be what e-commerce companies are accustomed to using to measure performance, but it doesn’t tell enough to evaluate site behavior, form design recommendations or inspire major insights. One of the biggest issues of the trend report is its reliance on averages. When evaluating site behavior, there is a fundamental flaw with this approach: we know that there is no “average visitor,” but rather that visitors fall into behavioral segments.

Instead of looking at the averages for the two metrics, it’s preferred to look at the distribution of visits based on the number of topic visits and message posts in each site session, and then to identify behavioral segments. In this scenario, put the number of forum topic visits on the x-axis and number of message posts on the y-axis.

Even though we saw in the trend report that, on average, visitors to the site visit around 0.8 topics and post 0.7 messages per visit, we see from the distribution of visitors that the “average use case” (1 topic, 1 post) actually happens very infrequently.

By using this approach, we have a much more interesting report for evaluating visitor behavior. We can use the behavioral clusters to gain insights into design and strategic opportunities, such as:

• Only 30% of visits go the forums. So let’s drive more traffic to the section by promoting it on the site and start a newsletter campaign for all customers that never created a forum account.

• 40% of forum visitors aren’t posting any messages. Do we need to make the module for posting messages easier to use? Should we incentivize visitors to post (since user posts are the content that visitors return for)?

• We see that there is a segment of visitors that view 5+ topics and post multiple times in their visit (~5% of visits). Can we analyze their profiles to see who they are? Should we reward them so that they continue their awesome behavior in future visits?

There’s power and value in looking at the same numbers in different ways. So put that trend report aside—perhaps even in the recycle bin—and go explore.