Verifying Retailer Uplift Claims: Check if sales definition, baseline and uncertainty measures match the method.; Confirm control group selection and consistency across products, channels and periods.; Validate SKUs, customer groups, sales basis and whether results were pre-planned.
Image: Commerce Ads

Incrementality Testing

Part of Retail media incrementality tests

Reviewing retailer uplift claims against the stated methodology

Check a retailer’s uplift headline against its outcome, comparison, sales population, uncertainty and disclosed study method.

Measurement Desk · · 3 min read

Before repeating a retailer’s uplift headline, verify that its outcome, population and comparison align with the stated study method. ‘Sales uplift’ requires a definition of sales, a baseline and a measure of uncertainty.

Retailer Uplift Claim vs. Methodology Requirements

Claim Verification Requirement
Outcome, population, and comparison must align with stated methodology
Required Elements for 'Sales Uplift'
Definition of sales, baseline, and measure of uncertainty

Rebuild the headline

Obtain the study specification, final methodology, treatment and comparison results, plus any changes made after launch. Determine the metric and unit: additional units, sales value, conversion rate and brand share address different questions. Check if the headline is an absolute difference, relative percentage or projection.

For instance, 108 units against 100 units on comparable bases gives an absolute difference of 8 units and a relative difference of 8% against 100. This is purely hypothetical. Without group definitions and uncertainty measures, it cannot determine confidence for action.

Verifying a Retailer’s Uplift Claim

  1. Obtain study specification and final methodologyInclude treatment and comparison results, plus post-launch changes
  2. Identify metric and unitUnits, sales value, conversion rate, or brand share – each answers different questions
  3. Determine claim typeAbsolute difference, relative percentage, or projection

Check the comparison

Determine when and how the control was selected: random assignment from an eligible pool, pre-campaign matching or a model. Verify that products, channels, periods and outcome measurement are consistent. Enquire how the analysis handled treatment units that did not receive ads, control exposure elsewhere, and changes in range or price.

An exposed-versus-unexposed comparison requires special care because exposure may depend on shopping behaviour. A before-and-after chart offers context but cannot alone control for promotions, stock, seasonal demand or other media. For matching or modelling, request the variables, validation and stated limits.

Key Checks for Control Group Validity

  • Control selection methodRandom assignment, pre-campaign matching, or modelling
  • Consistency across dimensionsProducts, channels, periods, and outcome measurement must be aligned
  • Handling of non-exposure and cross-channel exposureTreatment units not exposed to ads; control group exposure elsewhere
  • Analysis of range or price changesMust account for product availability and pricing shifts during campaign

Trace the sales figure

Examine the included SKUs and variants, eligible customers, sales channels, gross or net basis, returns cut-off and any extrapolation beyond observed records. A promoted-item figure and a wider same-brand figure may overlap. Do not add them without checking their definitions, and do not describe a retailer-only result as total UK brand growth.

Inspect group sizes, the uncertainty interval and its stated method, the primary outcome and material exclusions. Ask whether the headline was selected after examining several products, periods or audience cuts. Such a result may still be useful, but label it exploratory if it was not pre-planned as the primary analysis.

Critical Data Points to Trace in Sales Figures

SKUs and variants included
Ensure transparency on which products are counted
Eligible customers
Define customer pool used in analysis
Sales channels covered
Online, in-store, or both – must be specified
Gross or net basis
Returns cut-off date and method must be disclosed
Uncertainty interval
Stated method and confidence level required

Decide what wording is supportable

Use the claim as stated only when its definition, comparison, execution and uncertainty support that wording. Use a narrower label for a valid result limited to a subset, such as one product or a defined group of linked shoppers. Hold the claim when a material comparator, outcome definition or uncertainty assessment is unavailable for the intended decision.

Record the retailer’s wording alongside the wording adopted in the report, with the reason for any change. The conclusion should remain within what the disclosed method can support.

When to Use or Hold a Retailer's Uplift Claim

Use claim as stated
Only if definition, comparison, execution, and uncertainty are fully supported
Use narrower label
For results limited to one product or defined shopper group
Hold the claim
If key elements like comparator, outcome, or uncertainty are missing

More from Incrementality Testing