Skip to main content

Hi Klaviyo Community,

First-time poster here, hoping to get some expert eyes on a confusing A/B test result we're seeing.

We're currently running a campaign for a new product launch called "Fruit Boom". We decided to A/B test our standard, simple email template (Version A) against a new, more visually-driven template we've developed internally, codenamed the "ModHello" template (Version 😎.

The results are baffling us:

  • Version A (Old, Simple Template): Is performing as expected, with a Click-Through Rate (CTR) of around 4.2%.
  • Version B (New "ModHello" Template): Is performing terribly, with a CTR of only 0.7%.

We were convinced Version B would be the winner as it's more colorful, modern, and has some nice interactive-looking elements. We've already checked the usual suspects:

  • All links in both emails point to the correct URL and are working.
  • Both versions look great on mobile and desktop (tested on multiple devices).
  • The subject line is identical for both versions in the test.

To give you the full context of the user journey, this is the landing page that the email directs to. We felt the email design in Version B was a better match for the page's vibrant feel, but the numbers are telling a different story.

Landing Page: 

https://modhello.com/fruit-boom/

Our team is stumped. Are we missing something obvious here? Maybe the "ModHello" design is too distracting and pulls attention away from the Call-To-Action? Has anyone experienced such a dramatic drop in performance when trying to "upgrade" their email design?

Any advice or theory on what to investigate next would be a massive help.

Thanks in advance!

Hey ​@modhellooo 

Welcome to the community!

This is a super interesting test, and thanks for sharing such a clear breakdown. I’ve definitely seen this kind of performance gap before, especially when moving from a simple, text-forward template to (i am assuming) a image-heavy one.

A few thoughts and questions that might help you troubleshoot:

  1. Image Weight & Email Size: Do you know the total size of each version of the email (in MB or KB)? If Version B is significantly heavier due to high-res images, it might be triggering clipping in Gmail or longer load times, which can reduce clicks, especially on mobile.

  2. Image Blocking: Some email clients (especially Outlook) will block images by default. If your CTAs or key messaging are embedded in images rather than text, that could explain the dip. Are the CTAs in Version B still visible with images turned off?

  3. Deliverability: Heavier, image-dominant emails can sometimes result in lower deliverability or land in Promotions/Spam folders. Did you notice any changes in open rates or inbox placement between A and B, and was the spam rate higher?

  4. Visual Hierarchy & Distraction: While Version B might look great, is it possible the visual design is pulling attention away from the CTA? Sometimes cleaner layouts make the path to click clearer.

Let us know what you find. Always interesting to see how design vs. simplicity work for different businesses.
Also feel free to share screenshots of the two designs for us to have a better look

Thanks

Tim