AB testing is a [[Data Engineering]] strategy when you compare two versions of a page with each other. Having a third version or more in the same test is called Variant or *Variance* testing. Each test has a *weight* which determines how often it's shown to the user. there's also a name for test, and a target element. Generally, the whole page isn't AB tested, but only a specific element. Like any testing, it needs to be approached scientifically. *Dependent Variables* need to be controlled for, and only one *independent variable* should be modified at a time. When doing variance testing, the independent variable should be the same between the campaigns. A/B testing results are useless unless a statistically significant sample size is obtained. ## Methods ## Wordpress One of the simplest ways to do an A/B test is to have two pages, and simply swap them out for each-other, remembering the start and stop times. It's best to run for the same period of time (a week each?). You can also do weighted redirects. ## Learned Lessons Do not run AB tests at all the same time, you get muddy data. --- # References