Solved

Set up A/B test campaigns

  • 29 August 2023
  • 3 replies
  • 138 views

Badge

I am trying to set up a series of campaigns where I’ll be running a split a/b test. For example Day 1 campaign on October 2 with audiences A and B, Day 2 on October 4 with audiences A and B, Day 3 on October 6 with audience A and B and Day 4 on October 8 with audience A and B.

When I am setting up in campaign, I can only see an option to send a split test that is random, even on days 2-4. I want to set it up so that the first send is 50/50 random, but then for each additional day send, whoever got version A will also get version A for subsequent sends, and whoever got version B will get version B over subsequent sends.

Then if there’s new people added to the segment, they would be split 50/50 over each subsequent version. 

How do I go about doing this?

icon

Best answer by Akers Digital 1 September 2023, 05:03

View original

3 replies

Userlevel 7
Badge +60

Hey @PJ_FGWC 

Welcome to the community and congrats on your first post, it’s a great question.

In my opinion it would actually be best to set up a flow for your use case. With campaigns, its a one time send and will not address any new customers who are added to your segment. My recommendation would be to set up to A/B Test a Flow Branch.

First set up segment triggered flow > Conditional Split (Random Sample 50%) > Email #1 > Time Delay. It will look something like this: 

This will allow you to easily make sure that 50% of your customers receive the Yes path (Group A) and the other 50% to receive the NO path (Group B). The article I linked above will go into more details on A/B testing the branch. 

Hope this helps!

Badge

Hi Stephen, thank you so much for the reply! I had actually thought of this and started building this out...however, it doesn’t seem there’s a way to trigger to a specific segment at a specific date and time...how would you go about doing this?

Userlevel 6
Badge +21

I would do this via segments.

  1. Day 1 Send, do a split test and give each version a different name.
  2. After the send, create two segments using message variation filter, where you enter each message variation name into the respective segments.
  3. Day 2 send, send separate campaigns to the above two segments you made
  4. Day 3 send, send separate campaigns to the segments you made in step 2.

Here is a screenshot of what the definitions would look like for the segments. You would just need to change the message variation name to match the names you picked. 

 

 

Hopefully that helps!

Reply