A/B test – is a methology to improve existing ads/landing pages/graphics that provide better results. It is a method to provide consistance improvements(so it takes time) to results, conversion rates, sales or any quantitative results.
Prerequisite: basic convesion rates, numeric focused mind, ab-test glossary
For example, in the contact us form, someone suggested that large submit button will have higher form conversation. We do not know which is true until we tested it. We can apply A/B test here to verify the suggestion above.
It is the holy grail for growth hackers, who primary focus is improve conversion.
3 steps to A/B test
1. First, you need to focus on one “Objective”
Let it be a form’s conversation rate, more pageview per session or longer session duration. Pick one focus at any time. This helps you focus, and create meaningful variants.
Variants – a change to the original that possible to provide better results.
2. Create variant(s) to an existing landing page, target some traffic to the variation
It is important to test two variants un-biased. Send traffic to one landing page, and let program to randomly select which variants to use. Test both at the same time, running in parallel.
Put on your creative hat, larger the button/add hover animation/change CTA text and create variants. More time is needed if more variant is included in the test.
3. Wait to get “enough” traffic
“Enough” means statisticly “meaningful”. We do not need to open the stat book, you can let your A/B test software to tell you. If your variants results differece is huge, a short period will be enough. But if variants difference is small, more time is needed.
DO NOT manually compare the figures. You can easily fall into “false positive”. You need high school math & stat forumla to calculate a “Probability to be Best” and select the winner.
4. Apply the winner variant to become the original
And move to the next experiment. Happy growth hacking!
Document On Ready 