Why brands are adding AI to their NPD mix

The majority of fmcg launches are flops. So concluded a 2014 Nielsen report, which analysed 61,000 newly launched SKUs across Europe over three years. Of those, three quarters failed to retain a retail listing beyond their first year. And the majority never hit 10,000 unit sales.

Those findings were backed by a 2004 study from the Product Development & Management Association, which found a 49% failure rate in grocery NPD. It’s the highest rate across all industries – a rate that has barely budged since the 1980s.

And that’s only the products that actually get launched. A Harvard Business School study estimates 95% of new product ideas that reach the development phase go no further.

The grocery sector is littered with high-profile misfires: Colgate’s frozen food range, New Coke and Crystal Pepsi have entered grocery lore. But there are thousands more failed launches from both fmcg giants and challenger brands that you’ve never heard of, and never will.

Is such high rate of failure simply a painful truth of the grocery game? Recent developments in artificial intelligence and machine learning are emboldening some in the sector to believe otherwise.

As Ben Harknett, CEO of Cambri, says: “No longer is NPD about taking a leap of faith.”

The ambition for consumer insights platform Cambri – which last month launched a suite of AI-powered tools already being used by Carlsberg, Danone, and Nestlé – is to push NPD success rates up to 95%.

Put simply, Harknett wants to “make every launch a success”, rather than a drawn-out process with little odds of paying off.

Without AI, the typical journey of product from whiteboard to supermarket shelf involves several stages, or gates. The idea stage, concept stage, product fit stage and finally the launch.

Cambri comes in right at the start, at the idea stage. It ensures brands are “setting off on the right foot” rather than trying to correct down the line, Harknett says. It stops teams relying on gut feelings like –“‘hey, like, Bob thinks this is a good idea’ or ‘I like this kind of beer’”, he says. “Sometimes those [ideas] can be great. But at scale that often falls down.”

This more analytical approach has proven useful for Carlsberg Group-owned Sinebrychoff Brewery, which has been using Cambri’s AI tool for the past month. Sinebrychoff marketing director Niklas Rinne says the company is testing “really early on in the ideation phase where we’re looking at business opportunities in specific segments and our approach, to see if it’s something we should progress with or cancel”.

There is significant value, he tells The Grocer, in knowing “it’s actually not relevant in our market. So we need to postpone this project.”

Another benefit lies in the ability to respond to consumer trends more quickly. This was highlighted by Kraft Heinz at the Generative AI Food Pioneers Summit in London last month.

“Previously, we’d look at insights from, say, Nielsen. Nielsen would come to us and say – here’s a trend we see in the marketplace. We’d sit there and respond – ‘that’s quite interesting’,” said Tom Hadwen, head of sales food service international at Kraft Heinz.

“We’d have a meeting. Then we’d probably have another meeting. And in a few months we’d say, ‘these guys at Nielsen are on to something. I like this’,” he explained. “We’d involve our R&D team and operations team – ‘we see the trend, let’s get a product that fits that trend’.

“The teams would go and beaver away in the background and, hey presto, two years later we’ve got it. And then we go to Waitrose and we put it on the shelf and we’re too late. The trend’s gone, or somebody else owns the trend. We were too late.”

That all changed when Kraft Heinz partnered with Tastewise, another insight company powered by AI. Its tool ingests online recipe searches, social media conversations, restaurant reviews and other sources to identify patterns. Better yet, its massive data crunching power means it can “accurately predict upcoming food and beverage trends”.

“Now we can see trends early,” Hadwen said. “We take out the lag. We see it in real time. The products and trends that consumers are talking about in the marketplace. The new ideas and flavours.”

The entire process of launching a product is being turned “on its head”, he summed up.

This new process has already resulted in products on shelves. Tastewise’s analysis informed Waitrose’s 26-dish Japan Menyu range, launched in September.

This was based on a rise in social media chatter and images of Japanese cuisine, plus an increase in the number of restaurants with Japanese-inspired items on menus. The AI went as far as picking out yuzu and ponzu as popular flavours.

Nestlé also used AI tools to come up with the Nescafé Dalgona coffee mixes and Nesvita plant probiotic supplements for adults in China. It uses the tech to rapidly analyse information on trends, ingredients, flavours, and health benefits from social media, online publications and other web sources.

“Our innovators cluster the captured insights, which leads us to discover new ideas or trends that can be quickly translated into compelling product innovations”

The necessity of this process is underlined by Nik Pearmine, chief strategy officer of Black Swan Data. His company uses AI to analyse vast amounts of historic data, live consumer conversations online and applies advanced algorithms to identify patterns and predict future long-term consumer trends. As Pearmine puts it: “We live in an age where it is inexcusable for an NPD team to be blindsided by a new consumer trend or competitor launch.

“Consumers are actively talking about their desires, wants, needs and frustrations in public channels online. Being aware and informed should be a pre-requisite,” he adds.

Beyond the idea

And this is just what AI can do at the idea stage. Once the idea is agreed, the work begins to develop an actual product. Then, brands will have new questions. Namely, will this concept be a success and how might it be improved to ensure it is?

Here, AI can help by interpreting “a bunch of different metrics” explains Harknett, such as surveys on ‘willingness to buy’ or analysis of uniqueness in a category and “open-ended answers”.

AI is used to make better sense of all the data – “and then we’ve looked at how those concepts have actually performed once they’ve launched and we’ve built an AI model that predicts the likelihood that a concept is going to be a success” he says.

Based on all the information at hand, Cambri’s AI generates a report on a potential product’s strength and weaknesses, its uniqueness in the market, and proposes “new and improved value propositions”.

AI can even help develop the formulation of products. Unilever used it to analyse data on the role of the microbiome – the 100 trillion microbes that live in, on and around our body – in stimulating the immune system and keeping skin healthy.

“To understand the interactions between the microbiome and external forces, we had to dive into 12 terabytes of data and extract unique insights, much of which has gone already into everyday products such as Dove, Pond’s and Vaseline to create revolutionary, patented technology,” says Alberto Prado, Unilever R&D head of digital & partnerships.

Computational models were also used to find a vegan alternative to the most commonly used red pigment in colour cosmetics – red carmine, which is derived from female beetles. It can take 1,000 of these beetles to make one single lipstick. Unilever’s resulting vegan alternative – Hourglass Red 0 – requires none.

Future developments

Unilever’s Prado believes “we have just scratched the surface of what’s possible” with AI.

The AI companies also take that tone. In the case of Tastewise, progress has come in the form of conversational chatbot TasteGPT, launched earlier this year. It says the system can answer questions like “where should I launch my new beverage product first?” and “what product ideas are the best fit for my Gen Z consumers?”.

For Cambri, work is ongoing to boost its success rates even further. The brands using its AI model are already hitting rates of 73%. With the rate of development in AI, it’s confident it can get closer to its 95% target figure.

It’s a tantalising prospect. The only possible downside is a future that relies on stats, rather than creativity. But Pearmine argues data, AI and technology are enabling “a new form of creativity” – “one that places more of an emphasis on disrupting the process a company or team uses to produce ideas, versus just focusing on the idea itself”.

“As organisations develop more scalable, data-driven and consistent ways to innovate, decisions will be driven more by logic, systems and analytics than a spark of individual brilliance,” he adds.

And let’s not forget humans are needed to bring these projects to life. Especially in areas such as taste. Despite advancing work in ‘electronic tongues’, AI cannot yet devour a development chef’s proposed product and give feedback. But it is playing an important role in assessing human taster responses.

Nestlé has trialled the use of what it calls ‘emotional AI’, where it monitors invited taste testers on camera, and uses computer vision to gauge what they’re really feeling about a product. As the company’s group head of R&D IT Carolina Pinart said of the project: “Emotions are hard to fake. Someone might write down that they love a soup, but their face says something different.”

AI is also playing a role in enabling better human taste tests. US firm Gastrograph AI has a legion of tasters, who are asked to add flavour labels under 24 attributes for each product. Crucially, they can “use whatever words they want to describe the flavour in whatever language they want” – its AI allows for the “messiness of language” in its analysis.

“Conventional testing tries to squash the tasting experience to fit into a simple system that’s easy to analyse,” the company says. “Instead, we’ve designed our system so it’s able to capture rich tasting data and understand flavour in its full complexity.”

What is possible now with AI in NPD was unimaginable a year ago. And the possibilities that will be offered a year from now are hard to get a handle on. The only hurdle is human.

“A company’s ability to adapt to change and new ways of working is the biggest barrier to the advancements of AI in a real-life application,” says Pearmine of Black Swan Data, which works with PepsiCo, General Mills and Coty.

“It’s hard to convince someone to adopt something transformative, when they feel their job depends on things remaining the same”

“The question fmcg leaders should be asking is, what happens if we don’t move at the same rate as technology?” he sums up. “Those that do will continue to disrupt and win. Those that don’t will lose relevance and get left behind.”