Incrementality Testing
Part of In-store commerce media
Qualifying UK comparison stores for an in-store media test
Screen store eligibility, pre-period sales and commercial conditions before selecting UK comparison locations for an in-store media test.
The first decision in an in-store media test is which locations can be fairly compared. Build an eligible store pool before assigning screens, audio or other placements. Qualification comes before the wider test plan for treatment, outcomes and analysis.
Set eligibility rules before choosing winners
List stores where the promoted product is ranged, sales data are usable and the media placement can operate. Exclude stores with known closures, major refits, persistent stock gaps or incompatible ranges if those conditions would mislead comparison. Write the rules before anyone sees campaign-period sales. An exclusion invented after results are known can favour a desired outcome.
Record why each candidate passed or failed. A store that lacks the media hardware may be a possible comparison location, only if its customers, assortment and commercial conditions are sufficiently comparable to the treated stores. Hardware availability alone is not a matching method.
Compare the pre-period, not just the postcode
For the proposed product set, plot sales levels and trends over a relevant pre-campaign period. Note store size and format, product range, traffic where measured, and known price or promotion plans. When choosing test and control stores, relevant factors include pre-period sales, other activity, price and promotions, store size and format, and range.
Pre-assignment question / Reason to check it
- Was the product consistently available?
- A stock gap can make baseline sales look artificially low.
- Did sales move in a similar direction?
- Similar totals can hide divergent trends.
- Are format and range comparable?
- Different shopping missions can alter demand.
- Are promotions planned for only one group?
- A later sales difference may reflect the offer.
- Can media or shoppers cross groups?
- Spillover weakens the contrast.
Do not claim two stores are matched because their last-week sales are equal. Check whether a single promotion, holiday or missing-data week produced the apparent fit. Where feasible, compare several suitable stores in each group and keep the baseline plots and matching criteria.
Record the selection decision
If the eligible pool is large enough, consider random assignment within groups of similar stores. If business constraints determine which stores receive media, document them and remaining differences. Store-level holdouts and baseline matching are common approaches, but no selection method can prove a particular chosen pair is equivalent.
Save the eligible-store list, exclusions, pre-period data, matching variables and planned assignment before launch. If too few suitable locations remain, narrow the question or use another study design rather than presenting a weak comparison as a sound control. After stores are selected, the test team must still define and verify delivery, outcomes and uncertainty; qualification alone cannot establish an incremental sales effect.
