Introduction
The reduction in computing power cost in addition to the availability of data and big data technology are the basis for the increase demand to utilize AI (artificial intelligence) technology to solve business problems in the FMCG world.
Artificial Intelligence (AI) and Machine Learning (ML) can improve performance in many areas of the business. Typical business areas for that can include:
- Recommendations and shopping assistance
- Personalization of customers interaction
- Churn prediction
- Demand prediction
- Price and promotions optimization
- Assortment
- Evaluation of new product launch or product delisting
- Logistics and inventory management
- Others (such as HR management, business alerts, etc.)
For organizations that understand they need to make the investment in order to stay competitive there are three main paths to consider:
- Option 1: Hire a data scientist to develop AI solutions in-house. If you go on this route, you have to take into account that good data scientists are not cheap to hire and retain and that it might take several years to build the first robust in-house AI model.
- Option 2: Use a generic algorithm tool offered by a leading provider to the retail industry and try to make it fit into the FMCG industry and to your specific needs. While on paper this is a very attractive option, people familiar with the weakness and limitations of AI will tell you that this “cut and paste” approach might lead to disappointment. The assessment, testing, and on-going monitoring of the quality of an AI model is far from trivial, and the nature of AI is that small mistakes can end up leading to big embarrassment.
- Option 3: Start with a proven AI technology that is provided in a SaaS model by companies such as ciValue that focus on the FMCG domain and have a good understanding of the nature and the challenges of this specific industry. Any industry has unique points and a good understanding of the business is a must for the creation of an optimal solution.
In this three-part series, I will explain how we at ciValue have solved some of the unique challenges of FMCG retailers. To make this practical rather than theoretical, I will discuss these challenges using the example of building a recommendation for a Shopping List. A recommendation list is a common functionality in many FMCG online portals and from my assessment on how this is typically done it is clear that there is lots of room for improvement by using ML technology.
What is a Shopping List and why is this an interesting problem to solve using a machine learning?
It is not by chance that online shopping in grocery is lagging behind in comparison to many other retail categories. Many shoppers that tried purchasing online for the first time ended up having a bad shopping experience.
Indeed, it is very hard to replicate the shopping experience of a visit to a traditional supermarket. In a typical visit to the supermarket, the shopper has to select a basket of 20 to 40 items out of assortment of 10,000 to 100,000 SKUs. To support this need, the store layout has been refined over decades of experience. The store is designed to be easy to navigate, to enable shoppers to make quick decisions and to make it easy for people to find what they are looking for. The setup of a supermarket lets customers see all the main categories while walking the aisles. It also makes it easy to find alternative products if your favorite brand is out of stock or to show complementary products to encourage an unplanned purchase. It is also very rare that a customer will leave a supermarket with an empty basket, even when she can’t find everything she had in mind.
Creating a grocery shopping list or any other form of recommendation for the current visit is one of the first steps to enable a better online grocery shopping experience. In a perfect world, the shopper might want to consult with a virtual shopping assistant that will provide her the option to fill her basket with everything she needs, at an affordable price, in one push of a button.
For people outside the industry, this might not seem like a big issue. In fact, even when we talk to some retailers their first response is that all you have to do is prepare a list based on the shopper’s most common products in her last few baskets. However, when looking into the data you will quickly find that this is much harder to achieve than you initially think.
A mid-size supermarket is typically around 40,000 square feet (4,000 square meters). If there was a clear common list of 20-40 products to fulfill most of the shoppers need then the size of the supermarket could easily be reduced to 1,000 square feet (100 square meters)
To explain the challenge, we have first to understand what a typical basket looks like.
Using six months of purchase data from a mid-size grocery retailer, we analyzed the contents of a typical customer’s basket. Here is what we found:
When we looked at a customer’s total purchases over a six-month period we found the following:
So, the conclusion is clear: building a recommended shopping list that is based on the last six months of purchase history will give you a list of 310 items. Out of these 210 are irrelevant. And even with this long list, the shopper will not be able to find 7 out of the 20 items she is looking for (30%).
Taking a conservative assumption that 10% of online shoppers will abandon their basket if they cannot find what they need and that the other 90% will find and buy 6 items out of the 7 that are not in the smart shopping list, the total loss of revenue is about 15%!
If you are surprised by this data, here is another view of the same analysis, this time at the Shelf (section) and Category levels:
Past six months of analysis
Looking at this data we recognize that a good recommendation for a shopping list must take into account the following:
- Be sensitive to item cadence. E.g. Fresh, Personal Care, Appliances. To avoid a list that is too long the model should predict when is the next time the shopper might be interested to buy a product again and add it to the list at the right time.
- Be sensitive to item nature. Is it an on-going product (e.g. fresh food) or one to be used for a limited period (e.g. baby food). If someone stops buying fruit it might be a sign that she started shopping at one of your competitors. But if someone stops buying baby food it may just be that the baby has grown up.
- Analyze the source of the “first time/only once items” that represent 30% of a typical basket and 70+% of past purchases. Are these similar, alternatives, complimentary or supplementary to other high-frequency on-going items
- Define the right success criteria assuming that a simple hit map scoring is just not good enough as not all errors (false positives) are the same
- Take other elements into considerations such as seasonality, promotions, special events, returns
In the next two blogs in this series, we will dive into more detail on exactly how we can use AI and ML to deliver smarter recommendations, stay tuned.
