Hey @kimmia,

Great question!

Most of those questions are actually answered in our How Statistical Significance in Klaviyo Is Determined Help Center article. For example, the statuses you’ll see for your test results are listed under the Categories of Statistical Significance subsection which are: 

• Statistically significant
• Promising
• Not statistically significant
• Inconclusive

Overall, how long or how many test you run to get your results will vary from business to business, brand to brand and will be impacted by a number of variables such as your brand/business, your sample size, your sample behaviors, your marketing strategy/test strategy, etc. There’s just no clear answer to this without running through your very own tests. So, like I always like to say, test, test, and test some more!

Although we can’t share the model and algorithm we use to help determine win probability of your tests, one of our Data Scientists, Michael Lawson actually has a great Medium blog article they shared to help users further understand our data philosophy: Ensuring efficiency and accuracy in signup form A/B testing. If you’re interested, Michael also runs our Klaviyo Data Science Podcast which dives into a number of different topics, including A/B testing. 

I hope this helps!

David