Choosing a control group before a campaign begins

Select the control from people or locations eligible for the proposed retail media, then decide who will be withheld before launch. People who did not see an advert may differ in buying intent, so assembling a control afterwards can misstate lift.

Define the eligible pool

Record the campaign’s retailer, products, geography, audience, placement and dates. Apply the same eligibility rules when forming treatment and control groups. If only account holders or certain locations can receive the media, the result will concern that eligible pool. State who is outside it.

Choose an assignment unit that fits the media. Shopper or household assignment may suit addressable ads if withholding and sales measurement work for both groups. Location assignment may suit media that cannot be withheld person by person.

For store media, first qualify comparable locations using pre-campaign sales, product availability and planned commercial activity; location selection needs its own assessment.

Treatment vs Control Group Considerations

Select the method before outcomes are known

Where feasible, randomly assign eligible units and retain the assignment record. Inspect pre-campaign characteristics for material imbalance. Randomisation helps reduce selection bias, but it does not prove that later delivery, withholding or measurement worked.

If random assignment is unavailable, set matching criteria before seeing campaign outcomes. Relevant characteristics may include pre-campaign sales levels and trends, product range, stock, store format, prices and promotions. For shopper comparisons, use only suitable prior behaviour available for the study. Document differences that remain; similar recent sales alone do not make groups equivalent.

Preserve the comparison

Agree the primary analysis population in advance. When treatment assignment gives shoppers an opportunity to receive ads, compare the assigned groups as planned. Do not silently replace the treatment group with only people who received an impression: exposure can depend on shopping behaviour related to purchasing.

Ask how the study will detect control exposure elsewhere, treatment delivery failures and unequal sales coverage. Record other campaigns and promotions likely to affect either group. Show exclusions and changes in group size in the final analysis.

Before booking, keep the eligible-pool definition, assignment method, intended outcome, analysis dates and expected delivery checks together. If withholding, comparable outcome measurement or the control’s construction cannot be explained, redesign the study or ask a narrower question before treating a later sales difference as uplift.

Key Metrics to Monitor in Incrementality Testing