Skip to content
Marketing

How to set up A/B testing for your Email Campaigns

Email marketers! Are you looking for a way to test and improve the impact of your emails? With A/B Test Campaigns, you can now test two different versions of an email campaign with your recipient list. By testing multiple variations of your email camp

4 min read890 words6 explained images
View as markdownOpen in ClaudeOpen in ChatGPT
Each image has a one line explanation. Switch to full detail for the step it belongs to, the fields and buttons involved, examples and the whole procedure.
On this page
  1. How A/B Test Email Campaigns work
  2. Step 1: Go to Marketing > Emails > Select "Campaign" from the dropdown. After creating a new Email Campaign, there is an option to enable "A/B Testing."
  3. Step 2: Choose which you would like to split test: Email subject line or Email content.
  4. Step 3: Choosing the A/B Test Duration time frame
  5. Step 4: Choosing the number of split test variations
  6. Step 5: Choose the Winning Criteria
  7. Step 6: Testing your A/B Split test

Email marketers! Are you looking for a way to test and improve the impact of your emails?

With A/B Test Campaigns, you can now test up to 6 different versions of an email campaign with your recipient list.

By testing multiple variations of your email campaigns, you can get valuable insight to help create content that resonates with your audience and drives them to action.

Try out A/B Test Campaigns today and see why they’re the best way to measure the success of your emails. Follow the guide below to get started.

YouTube video player Open in a new tab

Video 2 Open in a new tab


Please Note:
- A minimum 10 contacts are needed for each variation. At least 40 total contacts are needed to conduct any split test.
- A/B test campaigns can be only sent via using send now or scheduling the campaign. It will not work for batch schedule or RSS schedule.
- If no results are revealed by the end of the stipulated timeframe, the initial variation will be declared as the winning variation by default.

Covered in this article:

How A/B Test Email Campaigns work 

How to Set up an A/B Test Email Campaign


How A/B Test Email Campaigns work 

  • Choose one element to test, such as your email subject lineor email content.
  • Set the number of variations to be tested - A maximum of 6 variations can be tested. 
  • Set the duration for when the test should run - Test duration can be set from 30mins to 24 hours. 
  • Select the sample size of recipients for the test (each variation will be sent to exactly half of the sample)
  • Choose the method for determining the winner: open rate or click rate.


How to Set up an A/B Test Email Campaign

  1. How to Set up an A/B Test Email Campaign (video 3) Open in a new tab

  1. Step 1: Go to Marketing > Emails > Select "Campaign" from the dropdown. After creating a new Email Campaign, there is an option to enable "A/B Testing."


Please Note:

A/B test campaigns can be only sent via using send now or scheduling the campaign. It will not work for batch schedule or RSS schedule.
  1. Step 1: Go to Marketing > Emails > Select "Campaign" from the dropdown. After creating a new Email Campaign, there is an option to enable "A/B Testing." (image 1 of 6) What this shows Step 1: Go to Marketing > Emails > Select " Campaign " from the dropdown. What this shows Shows what to choose for step 1 of 1 in the "Step 1: Go to Marketing > Emails > Select "Campaign" from the dropdown. After creating a new Email Campaign, there is an option to enable "A/B Testing."" section of "How to set up A/B testing for your Email Campaigns". This screenshot accompanies step 1 of 1 in the "Step 1: Go to Marketing > Emails > Select "Campaign" from the dropdown. After creating a new Email Campaign, there is an option to enable "A/B Testing."" section of "How to set up A/B testing for your Email Campaigns". For context, this section explains: Step 1: Go to Marketing > Emails > Select " Campaign " from the dropdown. After creating a new Email Campaign, there is an option to enable " A/B Testing. ". Step 1 of 1Image 1 of 6
    Buttons and menus referenced MarketingEmailsCampaignA/B Testing.send nowscheduling the campaignnot

Step 2: Choose which you would like to split test: Email subject line or Email content.

Step 2: Choose which you would like to split test: Email subject line or Email content. (image 2 of 6) What this shows A/B test campaigns can be only sent via using send now or scheduling the campaign. What this shows Illustrates the "Step 2: Choose which you would like to split test: Email subject line or Email content." section of "How to set up A/B testing for your Email Campaigns". This screenshot appears in the "Step 2: Choose which you would like to split test: Email subject line or Email content." section of "How to set up A/B testing for your Email Campaigns". The text alongside this image reads: A/B test campaigns can be only sent via using send now or scheduling the campaign. It will not work for batch schedule or RSS schedule. Immediately after, the guide continues: Email Subject Line (Optimize Open Rates) - It is necessary to play around with the length of the subject line or add personalization to get better open rates. Email subject lines are the first thing the recipients will see when receiving…. Image 2 of 6 What this coversA/B test campaigns can be only sent via using send now or scheduling the campaign. It will not work for batch schedule or RSS schedule.
Buttons and menus referenced Email Content (Optimize Click Rates)-
Next stepEmail Subject Line (Optimize Open Rates) - It is necessary to play around with the length of the subject line or add personalization to get better open rates. Email subject lines are the first thing the recipients will see when receiving an email campaign since subject lines are visible to them before they even open your campaign. Email Content (Optimize Click Rates)- You can test different elements in the campaign content that could influence or not the conversion or action from the recipients, like different section titles, article length, call-to-action buttons, images, videos, and more.
All 2 steps in this procedure
  1. Email Subject Line (Optimize Open Rates) - It is necessary to play around with the length of the subject line or add personalization to get better open rates. Email subject lines are the first thing the recipients will see when receiving an email campaign since subject lines are visible to them before they even open your campaign.
  2. Email Content (Optimize Click Rates)- You can test different elements in the campaign content that could influence or not the conversion or action from the recipients, like different section titles, article length, call-to-action buttons, images, videos, and more.

  • Email Subject Line (Optimize Open Rates) - It is necessary to play around with the length of the subject line or add personalization to get better open rates. Email subject lines are the first thing the recipients will see when receiving an email campaign since subject lines are visible to them before they even open your campaign.
  • Email Content (Optimize Click Rates)- You can test different elements in the campaign content that could influence or not the conversion or action from the recipients, like different section titles, article length, call-to-action buttons, images, videos, and more. 
  • Step 2: Choose which you would like to split test: Email subject line or Email content. (image 3 of 6) What this shows Email Subject Line (Optimize Open Rates) - It is necessary to play around with the length of the subject line or add personalization to… What this shows Shows what to choose for step 1 of 1 in the "Step 2: Choose which you would like to split test: Email subject line or Email content." section of "How to set up A/B testing for your Email Campaigns". This screenshot accompanies step 1 of 1 in the "Step 2: Choose which you would like to split test: Email subject line or Email content." section of "How to set up A/B testing for your Email Campaigns". For context, this section explains: Email Subject Line (Optimize Open Rates) - It is necessary to play around with the length of the subject line or add personalization to get better open rates. Email subject lines are the first thing the recipients will see when receiving an email campaign since subject lines are visible to them…. Step 1 of 1Image 3 of 6
    Buttons and menus referenced Email Content (Optimize Click Rates)-

Step 3: Choosing the A/B Test Duration time frame

A/B testing is a key part of any email marketing strategy, and properly selecting the duration time frame is essential to success. It is important to consider how long it will take for customers to recognize, read, and act on emails when determining the duration time frame, as this can impact your results. 

In this step, we select the time period after the combinations are sent out, compared to the results for open or click rates. 

Data collected during the test phase can be used to determine the email campaign's winning combination. Test duration is between 30mins to 24 hours.

Step 3: Choosing the A/B Test Duration time frame (image 4 of 6) What this shows Data collected during the test phase can be used to determine the email campaign's winning combination. What this shows Illustrates the "Step 3: Choosing the A/B Test Duration time frame" section of "How to set up A/B testing for your Email Campaigns". This screenshot appears in the "Step 3: Choosing the A/B Test Duration time frame" section of "How to set up A/B testing for your Email Campaigns". The text alongside this image reads: Data collected during the test phase can be used to determine the email campaign's winning combination. Test duration is between 30mins to 24 hours. Immediately after, the guide continues: During the duration that is set, if there were no results for click or open rate as chosen, by default the first variation will be sent out as winning variation. Image 4 of 6 What this coversData collected during the test phase can be used to determine the email campaign's winning combination. Test duration is between 30mins to 24 hours.
Buttons and menus referenced 30mins to 24 hours.no
Next stepDuring the duration that is set, if there were no results for click or open rate as chosen, by default the first variation will be sent out as winning variation.

Please Note:

During the duration that is set, if there were no results for click or open rate as chosen, by default the first variation will be sent out as winning variation.

Step 4: Choosing the number of split test variations 

The number of variations in the email campaign will be based on the strategy that you are following. To set the Test Size, use the slider to decide what percentage of contacts will receive different variations. You can also set the slider to variations to 50%.

Please Note:

A minimum 10 contacts are needed for each variation. At least 40 total contacts are needed to conduct any split test. 
Step 4: Choosing the number of split test variations (image 5 of 6) What this shows A minimum 10 contacts are needed for each variation. What this shows Illustrates the "Step 4: Choosing the number of split test variations" section of "How to set up A/B testing for your Email Campaigns". This screenshot appears in the "Step 4: Choosing the number of split test variations" section of "How to set up A/B testing for your Email Campaigns". The text alongside this image reads: A minimum 10 contacts are needed for each variation. At least 40 total contacts are needed to conduct any split test. Immediately after, the guide continues: Select which winning metric you want to measure: Unique Open Rate or Unique Click Rate. Image 5 of 6 What this coversA minimum 10 contacts are needed for each variation. At least 40 total contacts are needed to conduct any split test.
Buttons and menus referenced Test Size,At least 40 totalany split test
Next stepSelect which winning metric you want to measure: Unique Open Rate or Unique Click Rate.

Step 5: Choose the Winning Criteria

Select which winning metric you want to measure: Unique Open Rate or Unique Click Rate. 

Step 5: Choose the Winning Criteria (image 6 of 6) What this shows Select which winning metric you want to measure: Unique Open Rate or Unique Click Rate. What this shows Illustrates the "Step 5: Choose the Winning Criteria" section of "How to set up A/B testing for your Email Campaigns". This screenshot appears in the "Step 5: Choose the Winning Criteria" section of "How to set up A/B testing for your Email Campaigns". The text alongside this image reads: Select which winning metric you want to measure: Unique Open Rate or Unique Click Rate. Immediately after, the guide continues: Once the A/B Test emails are prepared, and you are happy, you may test the email campaign or preview the campaign by clicking on the 3 dots. Image 6 of 6 What to chooseSelect which winning metric you want to measure: Unique Open Rate or Unique Click Rate.
Buttons and menus referenced Unique Open RateUniqueClick Rate
Next stepOnce the A/B Test emails are prepared, and you are happy, you may test the email campaign or preview the campaign by clicking on the 3 dots.

Step 6: Testing your A/B Split test

Once the A/B Test emails are prepared, and you are happy, you may test the email campaign or preview the campaign by clicking on the 3 dots.

To publish/launch your campaign to the selected receipts, please set the delivery method to either 'Send Now' or 'Schedule For Specific Date/time'.

  1. Please Note:
A/B test campaigns can be only sent via using send now or scheduling the campaign. It will not work for batch schedule or RSS schedule.

Once the email campaigns are sent out, you may check the enhanced statistics and reports of variations and overall email campaigns (Mentioned at 5:19secs)

Was this guide helpful?

Related guides