What is An Absolute Change?
Absolute change in online testing and conversion rate optimization is the total difference between two numbers being measured in relation to each other. For example, if the number of leads from one month to the next changed from 40 to 50, the absolute change in leads between the two months is an increase of 10 leads.
Absolute change is most commonly used when the full quantity of change is relevant for further discussion. For example, an eCommerce merchant looking to project revenue that knows an average customer is worth $75 can easily calculate how much revenue was gained or lost based on the total number of customers that changed in a given time period.
Absolute vs. Relative Change
Unlike absolute change, relative change is expressed in terms of a percentage of change from a given number. In other words, it depends entirely on the context in which it is presented. In the above example, if the number of leads from one month to the next changed from 40 to 50, the relative change in leads between the two months is a 25% increase in leads.
Relative change is most commonly used when looking to compare time periods in terms of their change or growth. For example, If the number of customers for an ecommerce merchant grows by 10 every month for 4 months, the absolute change remains constant but the rate of growth (expressed in relative terms) actually goes down:
- 40 to 50 customers (month 1 to 2) is a 25% increase
- 50 to 60 customers (month 2 to 3) is a 20% increase
- 60 to 70 customers (month 3 to 4) is a 16.7% increase.
Common Problems With Absolute Change and Relative Change
The usefulness of absolute and relative change depends entirely on the situation. Both can present potential problems when used in the wrong context.
When showing relative change, the percentage increase can misrepresent the significance of the change. For example, a 100% increase in leads is no longer as relevant when, in a given testing period, the absolute number of leads only increased from 1 to 2. When the number on which the percentage change is based is small, relative change can look more significant than it actually is.
On the other hand, absolute change variables can look significant, even if the relative change connected to the two numbers is small. For example, an increase in 10 leads is not as significant when the business generates an average of 1,000 leads per year.
Context matters for both relative and absolute change. Showing only a percentage instead of real values, or only the absolute change number instead of the numbers on which it is based, can lead to misleading data and potentially wrong conclusions.
When to Use Absolute and Relative Change Values
Within CRO, absolute and relative change are most commonly used when comparing results of tests against each other. For example, in an A/B test, the difference between the two variables can be expressed in either the full number (i.e. increased clicks by 50) or the percentage (i.e. increased clicks by 10%).
More specifically, absolute change values are useful when working with smaller numbers. Within that context, even a single-digit increase can look massive when using percentages, like increasing leads by 300% when the absolute change was “only” 2 to 6 leads.
Generally speaking, the later in the funnel the metric to measure is, the more relevant using absolute change becomes. It’s also more helpful when looking at sales and revenue-specific data.
However, especially early funnel, large numbers benefit more from relative change. In these scenarios, understanding the percentage change is more meaningful. The same is true when comparing trend results over time, where a consistent increase in numbers might not actually be a consistent percentage growth.
What Should My Expectations Be?
When using absolute change values, expect the numbers to look about as big as the amount you’re measuring. On small numbers, absolute change values tend to look small, while on large numbers it looks large. Keep your expectations tempered based on that fact, and use relative change as a comparative value to better understand trend lines.
In online measurement and CRO, the evaluation of results matters just as much as the experimentation itself. If you need help on either end of the process, let’s talk. Contact us to learn more about our capabilities to help you optimize your conversion rates via testing and experiments.
