Solved

About A/B test

  • 16 August 2022
  • 4 replies
  • 28 views

Userlevel 1
Badge +2

Hello , i  have the following doubts about AB test. I hope to get answers

1.when test size is 100% flow ,there is no test duration,I want to know how long it will take to reach an experimental conclusion

 

2、i did a test, with duration  lasting 1 hour,but  there is no experimental conclusion,Is there a requirement for the minimum number of samples in the experimental group

3.I found that the data in the following figure were not significant at 95% confidence level by chi square test, but it tells me there is 97.7% win confidence variation B is winner, I want to know what our A/B test rules are

looking  forward to your reply ,  thanks ! 

icon

Best answer by alex.hong 19 August 2022, 23:29

View original

4 replies

Userlevel 6
Badge +54

Hello @kimmia,

Thanks for sharing to the Community.

For your first question, I believe this would depend on how long it would take for a user to complete your flow correct?  It would also depend on the # of contacts you are sending the flow variations too. In the same vein, while still testing one variable at a time, the more variations you add, the bigger the audience you’ll need to make sure you get reliable results. To get your results in a timely manner so you can perform more tests, use no more than four variations of one variable per test.

 

Imagine you have a great idea for a form, you A/B test it, and, on the first day, the first 10 people who see the form click through and make a purchase. Over the course of a week, the next 2000 people exit out of the form immediately. If you had ended the test after the first day, you would think that your form was great, even though a larger audience showed otherwise. By waiting until your results are statistically significant or until you have a good sample size of viewers, you’ll ensure that you know which form is truly better for your brand. Although we do not have a minimum # of people for a test, the more involved the better your data will be (that is true for anything).

 

For further information regarding your A/B test, I would recommend reaching out to Support to see if they can provide more detailed info into your account that we cannot see from screenshots. 

 

Thanks,
Alex

Userlevel 1
Badge +2

@alex.hong 

1.How many status  does the A/B Test has , whichs are they

2.What is the logic of the test duration

 

Looking  forward to your reply ,  thanks ! 

Userlevel 1
Badge +2

@alex.hong 

How is Win probability calculated , Chi-Squared Test or something else?

Userlevel 6
Badge +54

Hello @kimmia,

I see my colleague has responded to another one of your posts so I will redirect you to this post:

Thanks,

Alex

Reply