Skip to main content

Hello Everyone,

I often create campaigns by cloning a previous one and then modifying the content for the new campaign. That is what I did today, and then when I got to the screen to do the Subject Line, etc,  I decided to choose A/B Testing based on Personalization in the Subject Line.

I then edited the content of the campaign and sent it. But it turns out that not only was A/B testing done on personalization in the subject line, but half of the subscribers (the ones who’s subject line were not personalized, received the content of the previous campaign while the ones whos email was personalized in the Subject line, received the new campaign contents.

How did this occur when I only wanted A/B testing on personalization in the Subject Line?

And how best to deal with this as half the subscribers received an email with old content and incorrect special offers that no longer exist?

EDIT/ADDITION: I now see what happened. But to be honest, I can also see how why I made the mistake. When I clicked through to edit the content, there are two options: “Test Content” or “Test Send Time.”

I assumed that “Test Content,” because I had already selected Personalization in the previous screen, needed to be checked and then I did not realize there were also two Content emails (if I am explaining this properly) that were A and B. I did not notice for whatever reason, or understand that in addition to the Subject Line, I now also had an option to create complete different content for the new campaign.

In reality, all I wanted to do was send out the same campaign to all, but with the Subject Line being the only thing involved in A/B testing.

But what is the best way now, to resend (and apologize) to those who received the wrong campaign content?

Hi @Ian, welcome to the community.

As you now know, when you start an A/B test, Klaviyo duplicates the existing version into two complete separate instances, including what I like to call the “envelope” (From/Subject/Preview) as well as the Content body itself. 

When you clone a Campaign that has an A/B test, it also clones all the instances (From/Subject/Preview/Body) exactly as they are.  That way, you have a choice of deciding what you want to test - Subject, Preview, Content, and even the From/Reply-To.  Since Klaviyo doesn’t know what you want to test, it’s going to make the whole thing available for you to modify.

Once you get use to this concept, I always double check in the last “Review Step” all the versions of the A/B Test by not just looking at all the information on the left, but paging through the variants in the little preview section like this:

Click the Arrows back/forth to see all versions.

As for how to rectify, you can create a Segment of people who Received Email with the specific Message Variatione from the Campaign like this:

Then, you can send a mea culpa campaign to those affected. 

Don’t worry too much about it, we all make mistakes. Ironically, sometimes the “We goofed apology” emails actually do better in performance and it shows that there’s a person behind the brand.

As a side note, as you get more familiar with Klaviyo, you’ll adopt a more streamline workflow for checking everything.  I’ve found that creating yourself a pre-send checklist of all the things to verify, will help reduce errors in the future.  There was a Harvard study that said even top-knotch surgeons and pilots use checklists - surely we can do the same!


Joseph, thank you so very much for your detailed and helpful reply. Much appreciated!

Ian


Reply