Solved

Deleting a Branch in a Flow A/B Test

  • 24 February 2022
  • 4 replies
  • 226 views

Badge +3

I’ve been testing two branches in my onboarding flow: one short & sweet with 3 separate touch points and one that’s 8 emails long.  The longer series is clearly shown it superiority in conversions and sales (and no spike in unsubscribes), so I want to move everyone over to the longer flow.

Since I set up the flow for a 50/50 conditional split, I’ve now moved it to a 0/100 split, favoring the long one.  Now that I’ve waited a few days for the people in the short flow to have completed it, I’d like to remove the branch. 

My question: If I delete this branch, will it delete the data associated with it? 

icon

Best answer by alex.hong 25 February 2022, 00:35

View original

4 replies

Userlevel 1
Badge +3

@mostlyalex This is a great question, and I suspect that the reason you haven’t gotten more responses on this thread is because most Klaviyo users aren’t aware of this flaw in the system, and the enormity of the implications.

The only way to preserve all historical data for a flow (as far as I know) is to never delete any emails from it that were sent at any point in the past. In other words, if you use a conditional split to test something like time delays, number of emails in the flow, different offers, etc, you must keep all of the emails in the losing branch in draft mode within that flow for all of eternity. If you delete them, you lose the data, and your MoM, YoY reporting on that flow will forever be incomplete.

The only way that I know of to keep your flow from becoming this spaghetti tangle of ghostly draft branches is to create one Random Sample Conditional Split somewhere in your flow, set it to 100%, and then move all drafts into the No path. The more testing you do, the longer and longer this branch will become.

The bigger problem with this solution, however, is that there’s no way to label that split as anything more descriptive than “Is part of 100% random sample.” You can’t, for example, label it as "IMPORTANT: Keep these drafts for historical reporting purposes." Therefore, everyone within your company needs to know, and remember forever, not to delete that branch, otherwise you’ll totally distort all of your historical analytics for that flow. And this knowledge will need to be handed down to future generations of people managing flows for your company.

I’d love to hear from Klaviyo how we’re meant to deal with this issue, especially considering how commonly reps recommend setting up a conditional split to test something. So far all I’ve heard from reps is that my understanding is correct, and that they’ll “pass along this feedback to our product team.”

I’d also love to hear from other brands what system they’ve devised to make sure their historical data is preserved for all of their flows. 

Userlevel 7
Badge +58

Hi there @mostlyalex,

The conversion period begins when an email is received. The conversion data of your 30 day flow test should be visible on your account I believe.

When a flow message is deleted so are the analytics that were tied to that message. We suggest putting the flow into 'draft' mode. Then when you go to view your Flows, filter the status by 'live' so that you only see your live flows but keep the historical data of 'draft' flows.

If you are deleting part of a flow (ie. conditional split or a particular message) and wanting to keep the analytics - you could always clone the flow keeping the old flow the way it is with the analytics (just be sure that the old flow is in draft mode)and then delete the conditional split /message in the cloned flow. This article will give detailed instructions on how to clone the flow: How to Clone a Flow

Another option that you could keep the analytics of your old flow before you delete anything could be to run a flow performance report on that particular flow and group by those specific email messages, and then export the data. This way you will have the data of the messages and can edit the flow as needed. To accomplish this please follow the detailed instructions within this article: How to Build a Flows Performance Report.

Thanks,

Alex

Badge +3

So I’ll lose the analytics - but will I lose any of the conversion data?  My worry is that I delete the branch and then the entire flow results will be missing the sales that came from the branch (its about 30 days of an AB split, so the $$ is significant).  I dont want to look at incomplete data.

If i delete the branch will the sales data be gone? 

 

Userlevel 7
Badge +58

Hi there @mostlyalex,

Thanks for posting to the Community!

If you click the trash icon a pop up modal will appear asking you to confirm which branch you want to delete: 

?name=inline1794642452.png

You will lose the analytics for the deleted emails. If you need to save the content for future use, you can save each email to your template library.

 

Hope this helped,

Alex

Reply