A/B test glossary (2021)

Experiment – alias to test. Just like a labortory experiement, it has a hypothesis (objective) to verify. Contains varients.

False positive – an incorrect winning leader results from wrong statistic calculations.

Leader –  the winning variant from statistic calculations.

Original – Existing design/workflow on current landing page

Objective – The goal, the quantitative result that need improvement. E.g. pageview, conversion rate

Probability to be Best – Statistic terminology. Higher the %, more likely this is the best variant.

Targeting – percentage of visits to join the experiment. Higher the value, quicker the experiment completes. But if variants provide negative results, more visitors are affected.

Variants – a change to the original that is possible to provide better results. 

Weight (of visitors) – percentage of visits to split to variants. Usually fair split to each variants is enough. Educated guess and adjustments to weight might speed up the experiment.

%d bloggers like this: