Machine Learning and its limited application in price optimization

Let’s start by recalling what Artificial Intelligence is and its relationship with Machine Learning. According to John McCarthy, one of the founding fathers of AI, “human intelligence can be described so precisely that a machine can be built to simulate it.” The step-by-step instructions given to a robot to assemble a car on a production line are a good example of this definition.

But Artificial Intelligence is a broad discipline that covers different technologies and approaches. Machine Learning is a type of AI in which algorithms are trained using huge sets of input and output data. For example, a Machine Learning model can be shown hundreds of thousands of chest X-rays and told which ones correspond to lung cancer diagnoses. In this way, the model learns to recognize positive cases and can diagnose the disease fairly accurately when shown a new X-ray.

There are also Machine Learning models designed to classify (e.g., determining whether the patient has cancer or not) and others that calculate a numerical result (e.g., estimating temperature based on known data such as atmospheric pressure, humidity, etc.). It is precisely this latter type—called regression—that is used in analytical pricing tools to optimize prices based on known business data.

Although the idea of having a “crystal ball” that optimizes prices using historical information is very appealing, in practice there are reasons that limit its application. Here are three reasons why the use of Machine Learning models is not suitable for managing a pricing process:

Availability of input data
Training any Machine Learning model requires hundreds of thousands of input data points and their corresponding output data (results). Generally, the regression models used seek to optimize units sold, revenue, and profitability (output data) from multiple input data points such as own and competitor prices, costs, advertising and distribution levels (own and competitor), macroeconomic variables, etc.

It is highly unlikely to have this complete information for a period long enough to train Machine Learning algorithms properly. As a result, the model obtained is not robust enough, and the quality of the results leaves much to be desired.

Changes in business dynamics
But the availability of complete historical information is not the only challenge. Even if a regression model is successfully trained with complete information from, for example, the past three years, it may lose validity if structural changes occur in the market. Imagine a business whose model was built when customers bought 10% less if the company’s price was 10% above the closest competitor. It may happen that, due to the arrival of new lower-priced brands, that same price difference now causes sales to drop much more than the original 10%.

The entry of new competitors, changes in price sensitivity, and shifts in customers’ perceived value, among other factors, can cause a model to stop working correctly when projecting units sold under certain input conditions.

Price elasticity is not a number
Perhaps one of the most difficult structural variables to model is price elasticity of demand. This is because elasticity is not a number, but a function that varies according to the distance between a product’s current price and its optimal or potential price.

When a product is expensive—its price is above its potential—it will show high price elasticity, and units sold will change significantly after even a small price variation. But when the product is cheap—its price is below its potential—units sold will not change much if the price is adjusted.

In conclusion…
Does this mean Artificial Intelligence is not applicable to pricing? Of course not. What it means is that Machine Learning, as part of AI, has very limited application in price optimization.

Fortunately, there is another AI category that has proven effective in overcoming these challenges. These are Expert Systems, which convert human-created knowledge into algorithms. Under this approach, models are developed that simulate the judgment of pricing experts and consider all the variables that must be taken into account to make pricing decisions—perceived value, price sensitivity, competitor product configuration and pricing, costs, price elasticity of demand, channel margin, etc.

As always, the problem is not the tool, but the inappropriate use of it.