What is an A/B test?

An A/B test lets you investigate the popularity of one of two alternatives. For e-commerce stores it consists, for example, of preparing two versions of the store interface — with a given modification and without it — and then routing user traffic 50/50 to each version to check which is preferred and therefore better from a user-experience standpoint. For A/B test results to be valuable, you need to ensure their statistical significance — a measure of the probability that the result reflects reality and that you can draw rational conclusions from it. In other words, you need to rule out that the result is the product of chance from running the study on a sample that is too small or unrepresentative.