How to sell below cost price to make a bigger profit

Look into your shopping basket next time you shop and you may discover something. Almost half of your entire basket will most likely be items you never planned on purchasing when you came to the store. No, a store clerk didn’t secretly put them in there to increase your spend – you did it yourself.

According to recent research, depending on the age group, anywhere up to fifty percent of all purchases are based on impulse. These items were only bought because you entered the store today to buy your milk but happened to have picked up three or four other items along the way.

If you think about this for a moment, as long as a retailer can encourage you to choose their store to purchase that milk, then the cross selling can be worth a lot more to them than simply that liter of milk.

In fact, due to your other purchases, maybe the best strategy is for them to give that milk away for free?

Quantifying the power of cross selling

The team at Biarri has deep experience in retail to help make better pricing decisions such as markdowns. So it was natural that we turned our attention to this problem.

Biarri recently completed a year long project with UQ Master of Data Science students to pioneer a way, via Shapley Value Market Basket Analysis, to finally quantify the value of a product based on its power to cross sell in a basket.

Traditionally retailers have used Market Basket Analysis as one of the most common data mining approaches to analyse customer purchasing patterns in retail stores. These techniques provide association rules for the items that frequently appear together. However, they fail to recognise the significance of item quantity and price only helping retailers understand which products are purchased together and which products may lead to the purchase of other products.

How is a retailer to use such information to improve their operations?

Place items that are purchased together closer together? But why? Will this really lead to more cross selling? Who knows.

The only way this information can become valuable is if we actually quantify the cross sell value back to the item causing the cross selling. Then we can tweak the price of the original item to attract more customers and hence more cross selling!

It is only when we begin to quantify this information that we can make useful decisions to improve the operations of retailers.

The Approach – Shapley Value Market Basket Analysis

To achieve this Biarri leaned on an academic field enjoying a resurgence in recent years due to AI. This field provides a core part of the technology used to create some of the shocking deep fakes videos and pictures in the area of Generative Adversarial Networks. The key trick to creating these deep fakes is based on core concepts from this research area of (Non-cooperative) Game Theory such as the Nash Equilibrium.

Similarly, Biarri has leveraged the concepts from Cooperative Game Theory to develop a new way to quantify the true value of items in a shopper’s basket. The new approach utilises the Shapley Value (another concept playing a pivotal role in deep learning) to allocate the value to a set of items in a basket based on the value each contributed to the basket.

What this means in practice is that for all customers of a store, we examine each basket of items purchased. We label each item purchased as a different player (A, B, C, etc in the diagram below) and then look at the total value of the basket. Averaged across all baskets we can find out which items contribute to the purchase of other items and attribute that additional value to the cross selling good and understand what their true revenue contribution is – not just what their sticker price contribution is.

The Problem and Solution

Using the Shapley Value is a novel idea, however, just naively using it won’t work. The computation of it is extremely slow (combinatorially slow) so it cannot be used on anything but toy datasets. So the Master of Data Science students at UQ came up with a clever way of speeding up the calculation so that it scales linearly with the number of items and not exponentially.

This algorithmic breakthrough then made it possible to test the new method on a dataset with 500,000 transactions to come up with a unique set of items which drive additional cross selling revenue for the retailer. When combined with price elasticity calculations, these results can be used to optimally price the goods to maximise gross profits.

The results demonstrated that many items were only worth as much as their sticker price but there were a few valuable gems in the product catalogue which were driving a large chunk of the online retailer’s profits. The below diagram shows the spread of “additional” attributed value (sometimes it is negative when an item is actually worth less than what it is selling for!).

Where to from here?

Using the Shapley Value Market Basket Analysis, Biarri has finally shown how we can answer specific questions on the value of individual purchases. Biarri’s results have been accepted for publication in the peer-reviewed International Game Theory Review journal and will be published soon. We’ll update this post once they are available to ensure everyone has access to these breakthrough results.