When talking about machine learning in a commercial context, people often think that their conversion rate will increase just because advanced calculations relating to machine learning are being made in a server room. That is not the case. In order for machine learning to improve the conversion rate, there needs to be a complete use case where a machine learning algorithm figures out something interesting and the customer is able to use this information to their benefit.
In a standard example, you would have a treatment that has an effect of, let’s say, +10% on conversion if offered to every visitor. An example of this would be offering free delivery to every customer in an online store. If you instead targeted this treatment with machine learning, that 10% would set an upper limit to how much your conversion could improve. In the best case, we would find exactly the visitors that bought because they got that treatment and not offer this costly treatment to anyone else. This would land at the same +10% improvement in conversion. In such a case we would have optimised the benefit, i.e. minimised the expense while maximising the return.
In order to reach something better, you need a slightly different setup. If you had a treatment that has a positive effect on some of the visitors and a negative effect on others, you could actually reach a better conversion rate with machine learning than you could by offering that treatment to everyone. This would be a case where you reach some conversion rate precisely because you use it together with machine learning; a case where you could not reach the same conversion rate without machine learning.- As it happens, this is precisely what we have done.