Solved

How do I change weights on an *existing* a/b test email flow

  • 24 September 2022
  • 1 reply
  • 43 views

Badge +2

I’ve got an A/B test running on a message in an email flow.

I’ve got multiple variations set to send with certain weighted percentages.

How do I change these percentage weights without ending the test??

I feel like I used to be able to do this at-will… maybe the new A/B test flow prohibits it?

Or is there still a way? Super frustrating.

icon

Best answer by Omar 24 September 2022, 12:02

View original

1 reply

Userlevel 7
Badge +43

@Wear Skinnys changing anything in an experiment normally invalidates the experiment. Coming from the conversion optimization field we've ran hundreds of a/b tests and are happy that Klaviyo has disabled making changes on running experiments. I do advice you to dive into a/b testing and statistics behind it. Most of the times I've seen people run experiments and make decisions on email tests that could have been noise and where it either was not significant or volumes and changes very small. 

If you want to make sure you're gaining most from a running experiment choose the distribute automatically Klaviyo will use a “Multi-Armed Bandit” method which basically distributes traffic to the variant that's performing the best. In most cases I think that's a very good compromise to capitalize on early signs of better performance. 

 

From Klaviyo documentation found on this page (I recommend reading it)

Luckily, in Klaviyo, you cannot make changes to a live A/B test for campaigns and forms, so this best practice applies to flows specifically. If you find yourself wanting to make a change to a template, end your A/B test first or, in the case of a campaign, cancel the email send. Then, restart the test to ensure you’re getting the most accurate data.

 

There's also a good article written by Klaviyo about this here: https://klaviyo.tech/ensuring-efficiency-and-accuracy-in-signup-form-a-b-testing-62bd3f2f2f3f

 

Omar Lovert // Polaris Growth // Klaviyo Master Platinum Partner

Reply