The Cost of Not A/B Testing – a Case Study

Most of the time when discussing A/B testing, regardless of context, we discuss costs such as the expense of running an experimentation program, of shipping ‘winners’ to production. Only rarely do I see references to the less obvious, but u…

Most of the time when discussing A/B testing, regardless of context, we discuss costs such as the expense of running an experimentation program, of shipping ‘winners’ to production. Only rarely do I see references to the less obvious, but usually more important costs in terms of opportunity cost (incurred during testing) and the cost of […] Read More...

The A/B Testing Guide to Surviving on a Deserted Island

The secluded and isolated deserted island setting has been used as the stage for many hypothetical explanations in economics and philosophy with the scarcity of things that can be developed as resources being a central feature. Scarcity and the need to…

The secluded and isolated deserted island setting has been used as the stage for many hypothetical explanations in economics and philosophy with the scarcity of things that can be developed as resources being a central feature. Scarcity and the need to keep risk low while aiming to improve one’s situation is what make it a […] Read More...

Inherent costs of A/B testing: limited risk results in limited gains

I’ve already done a detailed breakdown of costs & benefits in A/B testing as well as the risks and rewards and how A/B testing is essentially a risk management solution. In this short installment I’d like to focus on the trade-off betwe…

I’ve already done a detailed breakdown of costs & benefits in A/B testing as well as the risks and rewards and how A/B testing is essentially a risk management solution. In this short installment I’d like to focus on the trade-off between limiting the downside and restricting the upside which is present in all risk management […] Read More...