Skip to main content
Question

A/B Testing - Placed Order Rate Deficiencies

  • February 27, 2025
  • 1 reply
  • 13 views

Forum|alt.badge.img+1

I have been A/B testing specific sections of email campaigns to see which email version drives more revenue. I feel like there’s a definite deficiency in the reporting of “placed order” when testing campaigns. You can see which version drove the most revenue, but there’s no where in the results where it shows you which purchases were attributed to which email version. Due to this, it’s not very informative as far as a conversion rate.

 

For example, I send a weekly inventory email for my company. I wanted to test if different looks/feels/formats of the “On Sale” section of this inventory email drove customers to purchase more sale items. However, figuring this out has been a bit of a nightmare because I have to manually try to figure out which Shopify orders were attributed to Version A of the email, and which were attributed to Version B. I can see how much total revenue each version is attributed to, but not a list of which sales. Thus, I’m not able to figure out which one drove more “Sale Collection” purchases.

 

Am I missing some functionality and just not fully informed how to get this info, or is this actually not possible? It seems like something that could be easily added to the testing results, since the revenue is already reported, so the information has clearly already been gathered.

 

In the meantime, does anyone else have a work around that could help me more quickly figure out which sales came from which email testing versions? Thank you in advance!

Did this topic or the replies in the thread help you find an answer to your question?

1 reply

Byrne C
Community Manager
Forum|alt.badge.img+14
  • Community Manager
  • 98 replies
  • February 27, 2025

Hi ​@Dekl1544,

Thanks for the question - I agree that it would be really useful to have a list of which purchases were attributed to each version, rather than just the numerical amount. Happy to reach out to our product team and recommend we implement this in the future.

In the meantime, I’d recommend segmenting these customers to determine which purchases from certain categories were attributed to each version.

You can do this by creating a segment of people who opened/clicked a message variation of your campaign AND placed an order for a certain category within the attribution dates. This would give you the number of people who purchased specific items after clicking a variation of your campaign.

Let me know if this makes sense. I get that it’s an imperfect workaround, but it should get you the numbers you’re looking for!

-Byrne