SlideShare a Scribd company logo

What is A/B split testing

•
•
Emailer Go
Emailer Go

A/B split testing, more commonly known as Hypothesis Testing, is a term used to describe a controlled experiment with two varying components. In the case of an email campaign, you could for example, sequentially alternate between 2 different Subjects. The objective is to identify which of the two subjects achieves better results by comparing one subject against the other.

1 of 2
Download to read offline
www.EmailerGo.com What is A/B split testing? A/B split testing, more commonly known as Hypothesis Testing, is a term used to describe a controlled experiment with two varying components. In the case of an email campaign, you could for example, sequentially alternate between 2 different Subjects. The objective is to identify which of the two subjects achieves better results by comparing one subject against the other. Immediate results within a single campaign At Emailer Go we take the concept one step further. Rather than applying the test to an entire campaign and then analysing results for use in your next campaign, for larger Mailing Lists, you could just as accurately apply it to the first 20% of your contacts in a Mailing List, gauge results, and then send the better performing subject to the remaining 80% of your contacts. How does A/B split testing work? A/B Split testing works by gathering statistics in real time, such open rates or click rates achieved after a percentage of your contacts have been sent your latest campaign. After a specified amount of time the results are tallied and whichever of the components performed better is sent to the entire remaining percentage of your contacts. Consider the following: You create an email campaign where you will be offering a $100 discount on all trampolines. In your A/B split testing campaign, you enter the following two subjects which will be alternately sent to your contacts. Subject A: Sale now On Subject B: $100 off all trampolines Let assume you have a Mailing List with 10,000 contacts. The first 2,000 messages dispatched to your Mailing List would see 1,000 contacts receive Subject A and 1,000 contacts receive Subject B. Our email software could then pause the campaign for X hours while waiting for the contacts to check their email inboxes. If after the lapsed time, Subject B had an open rate of 25%, while Subject A had a meagre 5% open rate, Subject B would be dispatched to the remaining 8,000 contacts on your Mailing List ensuring a much more successful campaign. But its not just limited to Subject and Open rates. We’ve also built in provision for testing From Names and Message bodies, as well gauging results for click through rates. How do I use A/B split testing? Log into your account and create a message as you normally would: 1. Campaigns 2. New Campaign 3. Enter campaign details, click next 4. Click the Yes option for Use A/B Split Test to reveal the options available.
www.EmailerGo.com 1. Select which field you would like to apply the test to. Remember you will need to enter two different Subjects or two different From Names or two different Messages. Some contacts may immediately delete your message if the Subject is not appealing; others may have filters set up reject messages with certain words in the body of the message. 2. Select the deciding factor for selecting the better performing component. This would depend on your objective for the campaign. If you had no hyperlinks in your campaign then you would select Most opens rather than Most clicks. 3. Click and drag the first slider to select the number of hours EmailerGo’s dispatcher should wait for statistics to be gathered. This would depend on how many contacts were in your Mailing List as well as how fast you were sending messages to them. You can select the speed at which messages are dispatched, but for optimum results, we recommend around 50 messages per minute. 4. Finally, select the percentage of your contacts the experiment should be applied to. Again this depends on how many contacts are in your Mailing List. There would be little point in testing 5% of a Mailing list with only 200 contacts, as results would vary wildly. Once set, enter the remaining campaign details and click next to set your campaign in motion.

More Related Content

What is A/B split testing

  • 1. www.EmailerGo.com What is A/B split testing? A/B split testing, more commonly known as Hypothesis Testing, is a term used to describe a controlled experiment with two varying components. In the case of an email campaign, you could for example, sequentially alternate between 2 different Subjects. The objective is to identify which of the two subjects achieves better results by comparing one subject against the other. Immediate results within a single campaign At Emailer Go we take the concept one step further. Rather than applying the test to an entire campaign and then analysing results for use in your next campaign, for larger Mailing Lists, you could just as accurately apply it to the first 20% of your contacts in a Mailing List, gauge results, and then send the better performing subject to the remaining 80% of your contacts. How does A/B split testing work? A/B Split testing works by gathering statistics in real time, such open rates or click rates achieved after a percentage of your contacts have been sent your latest campaign. After a specified amount of time the results are tallied and whichever of the components performed better is sent to the entire remaining percentage of your contacts. Consider the following: You create an email campaign where you will be offering a $100 discount on all trampolines. In your A/B split testing campaign, you enter the following two subjects which will be alternately sent to your contacts. Subject A: Sale now On Subject B: $100 off all trampolines Let assume you have a Mailing List with 10,000 contacts. The first 2,000 messages dispatched to your Mailing List would see 1,000 contacts receive Subject A and 1,000 contacts receive Subject B. Our email software could then pause the campaign for X hours while waiting for the contacts to check their email inboxes. If after the lapsed time, Subject B had an open rate of 25%, while Subject A had a meagre 5% open rate, Subject B would be dispatched to the remaining 8,000 contacts on your Mailing List ensuring a much more successful campaign. But its not just limited to Subject and Open rates. We’ve also built in provision for testing From Names and Message bodies, as well gauging results for click through rates. How do I use A/B split testing? Log into your account and create a message as you normally would: 1. Campaigns 2. New Campaign 3. Enter campaign details, click next 4. Click the Yes option for Use A/B Split Test to reveal the options available.
  • 2. www.EmailerGo.com 1. Select which field you would like to apply the test to. Remember you will need to enter two different Subjects or two different From Names or two different Messages. Some contacts may immediately delete your message if the Subject is not appealing; others may have filters set up reject messages with certain words in the body of the message. 2. Select the deciding factor for selecting the better performing component. This would depend on your objective for the campaign. If you had no hyperlinks in your campaign then you would select Most opens rather than Most clicks. 3. Click and drag the first slider to select the number of hours EmailerGo’s dispatcher should wait for statistics to be gathered. This would depend on how many contacts were in your Mailing List as well as how fast you were sending messages to them. You can select the speed at which messages are dispatched, but for optimum results, we recommend around 50 messages per minute. 4. Finally, select the percentage of your contacts the experiment should be applied to. Again this depends on how many contacts are in your Mailing List. There would be little point in testing 5% of a Mailing list with only 200 contacts, as results would vary wildly. Once set, enter the remaining campaign details and click next to set your campaign in motion.