Grocery data explained: how do Kantar Worldpanel, Nielsen and IRI work?

You’ll have spotted a variance of almost £50bn – not far off the GDP of Wales – in the valuations above. But that doesn’t mean two of them are wrong. IRI, Kantar and Nielsen are trusted by the UK’s top retailers and suppliers for their analysis, after all.

So why is there such a huge difference in their valuations?

The truth is that no methodology is perfect.

IRI, Nielsen and Kantar’s parameters can vary in terms of product types and retailers covered. For example, IRI’s £99.2bn valuation for our April Focus On Own Label does not include all types of fresh produce, helping to explain the lower value here.

Buying retailers’ EPoS (electronic point of sale) data – the primary methodology of the IRI and Nielsen services above – only gets you so far. Some retailers won’t share their EPoS data at all; most restrict data deemed commercially sensitive because it pertains to goods sold exclusively or chiefly through their own outlets.

(Note: This table doesn’t display fully on mobile. Please view the desktop version of the article for the full content.)

The alternative to EPoS is using a panel of consumers who log everything they buy and then extrapolating it out to national level.

But, as we’ll see, this methodology – used by Kantar Worldpanel and Nielsen for its Homescan service – also has its limitations as well as its strengths.

 

The power of panels

There’s no disputing EPoS data. Every time a retailer sells an item, everything from price to packaging format is logged with the zap of a barcode. Every sale is recorded. That data can then be used to calculate total market values, rate of growth and loss and so on. Simple!

Not quite. Not all retailers can or will share their data. Aldi and Lidl fall into this camp. This mattered less 20 years ago, when their market share was a fraction of what it is today. But with the discounters accounting for as much as a fifth of sales in some categories now, their absence leaves a sizeable gap in any EPoS-based valuation.

The below illustrates this nicely. Left is Nielsen’s 2017 valuation of UK jam & spread sales. On the right is Kantar’s. There are a few differences to note between them – Nielsen’s numbers are a month older, for example; its numbers include sandwich spread, Kantar’s don’t – but most significant is the absence of Aldi and Lidl in Nielsen’s Scantrack read. (Its Homescan service does capture the discounters.)

Hence the £46m (10%) difference in value. The different periods and products covered explain some of this (Nielsen’s inclusion of Heinz Sandwich Spreads contributes £3.7m to its valuation) but the greatest influence is the absence of the discounters in Nielsen’s read. They sold £65.1m in spreads last year, says Kantar – 12.7% of the market.

Gaps in EPoS analysis can also arise from ‘masking’ of information pertaining to own label or exclusive products when a proportion of sales through a given retailer pass a specified threshold. For example, retailer-exclusive prosecco brands are typically masked, meaning they don’t appear in rankings of the top selling sparkling wines.

These release rules are the industry standard, so affect anyone using EPoS data. Barring industrial espionage, the only way to get a read of Aldi and Lidl’s sales or sales of goods masked by retailers is to use a consumer panel. But, of course, panel data has its limitations too.

 

The power of EPoS

We picked spreads in the above example for good reason: few ever buy a jar of jam and eat it on their way home. This seems a strange statement, but it’s worth making because Kantar Take Home Grocery (and Nielsen’s rival panel service, Homescan) only cover products brought home after purchase. The clue is in the name.

This matters more for some sectors than others. Compare Kantar’s higher read of jams & spreads in the above graph – 10% higher than Nielsen’s – with the below figures showing Nielsen and Kantar’s valuations for UK carbonated soft drinks sales, a market far more reliant on consumers buying and consuming products on the go.

This time around, Nielsen’s valuation of the market is more than £1bn or 40% higher than Kantar’s. Remember the earlier point about EPoS data: every single sale is recorded, regardless of where the product is finally consumed.

This helps explain the vast difference between Nielsen and Kantar’s valuations of the fizzy pop market above.

It’s worth noting that not all of IRI and Nielsen’s data is ‘real’.

The independent retail sector is fragmented, with 18,000 unaffiliated indies and 15,000 symbol group retailers [GRS 2017] operating across the UK. Buying EPoS data from each outlet is just not practicable, so IRI and Nielsen use representative retail panels for extrapolation.

Kantar does measure out of home consumption, through a 7,500-strong panel of individuals that logs all meals, snacks and soft drinks bought to be consumed out of home.

Kantar says these numbers cannot be integrated with the take-home panel as this measures household purchasing while out of home logs purchases by individuals.

 

Own label

Aldi and Lidl’s close guard on their data creates another problem for analysts.

Both retailers are heavily reliant on own-label products, meaning any analysis that doesn’t include the discounters is likely to underplay the significance of own label.

See the difference in IRI and Kantar’s valuations of own label’s share in five sectors below.

 

The why and the what

EPoS can be said to provide the ‘what’. As well as helping to fill in some of the gaps caused by the discounters and the masking of retailer exclusive products, panel data is often said to provide the ‘why’.

Let’s say a rival has launched a new fizzy turmeric drink. It’s niche, but it’s made a splash and a national convenience retailer has listed it. You need to track it. The only way to do that is through EPoS data because this picks up every individual sale. It’s too small to be picked up by a panel.

You buy the data and it shows blistering growth for the brand. Now you need to know who’s buying this drink, and why. That’s where the panel data comes in. You segment the data by age, socioeconomic group, motivation, past purchasing history and so on. Alarmingly, it shows your consumers are switching to this new drink. You need to act.

Without the EPoS data you wouldn’t have known about the scale of the new rival’s growth. But without the panel data, you couldn’t have known that consumers were switching from your product to this, or why. Modern tills are smart, but they’re not that smart.

That’s why smart operators use EPoS and panel data in conjunction.