Hi Community, has anyone tried to conduct a hold out test on their Abandoned Cart email to measure effectiveness?
We are looking to utilize the “random sample” conditional split in the flow to conduct this but are uncertain for the best strategy on how to see the impact on those who don’t receive the email.
Thanks in advance,
Best answer by david.to
Love the idea of wanting to compare how sending an abandoned cart email will impact you customer’s conversion rates!
I would be highly interested to hear what other strategies other Community members have come up with as well. For my recommendation, I would agree with using a random sample conditional split at the beginning of your flow to separate out traffic of those users who have triggered the flow. I would then suggest creating two segments; one to capture those customers that went through the abandoned cart flow experience, and another capturing those customers who have triggered the abandoned cart flow but was actually removed from the random sample.
You can create the first segment by using the rules of “What someone has Received email at least once in the last X-timeframe where flow equals X (Name of your abandoned cart flow)”.
For your second segment, you can use the rules “What someone has done, Checkout Started at least once in the last X-timeframe AND What someone has done, Placed Order zero times in the last X-timeframe where Attributed Flow equals X (Name of your abandoned cart flow) AND What someone has Received email at least once in the last X-timeframe where flow equals X (Name of your abandoned cart flow)”
When creating both of these segments, you’ll want to ensure the timeframe selected is the same for both segment and for all the rules.
With both of these segments created, you can run an engagement report which can offer you the average performance of both segments. Because each of these segments are capturing customers who have received the flow and a second group of users who abandoned their cart but did not receive the flow, we can broadly compare the performance of each segment’s engagement report to extrapolate how effective your abandoned cart flow is. You can learn more about running a list or segment’s engagement report from through the Understanding Engagement Reports for Lists and Segments Help Center Article.
Thanks for being a Klaviyo Community member!