Skip to main content
Contributor I
September 24, 2021
Solved

When to call A/B test on a flow statistically significant?

  • September 24, 2021
  • 1 reply
  • 134 views

Hi there,

 

I would like to know if anyone has any insights when to call an A/B test on a flow statistically signifant.

 

Klaviyo determines it for you on a campaign, but I was wondering if anyone had some guidelines for flow. 

How many ppl would have had to receive the email lets say within 30 days to call to make a decision?

 

Thank you,

 

PJ

This topic has been closed for replies.
Best answer by stephen.trumble

Hey @plejuliana 

Thank you for reaching out to the Klaviyo Community about determining statistical significance of your A/B test.

This is a great question! You can run an A/B test on Flow Emails and on Flow Branches. The link above covers the different categories of statistical significance and how they are determined. Flow A/B tests are calculated the same as campaigns.

An A/B test result is deemed statistically significant when:

  • 50 people have received each variation
  • The win probability is at least 90%

Also check out these other community post about A/B testing that include addition resources to help you measure your results:

Thank you so much for being part of the Klaviyo Community!

1 reply

stephen.trumble
Klaviyo Alum
September 24, 2021

Hey @plejuliana 

Thank you for reaching out to the Klaviyo Community about determining statistical significance of your A/B test.

This is a great question! You can run an A/B test on Flow Emails and on Flow Branches. The link above covers the different categories of statistical significance and how they are determined. Flow A/B tests are calculated the same as campaigns.

An A/B test result is deemed statistically significant when:

  • 50 people have received each variation
  • The win probability is at least 90%

Also check out these other community post about A/B testing that include addition resources to help you measure your results:

Thank you so much for being part of the Klaviyo Community!

-Stephen