What is A/B testing — and how to run your first experiment with tracking links

What is A/B testing?

A/B testing — also called split testing — is the practice of comparing two versions of something to determine which one performs better. You show version A to one group and version B to another, then measure the outcome that matters to you: clicks, sign-ups, purchases, replies.

The idea is simple: instead of guessing which email subject line will get more opens, or which landing page headline will convert better, you run both and let real user behaviour give you the answer. A/B testing replaces opinion with evidence.

A/B testing is used across every marketing channel. It started in direct mail (send two versions of a letter, see which gets more responses) and has since become a core practice in email marketing, digital advertising, landing pages, social media, and product design.

How it works

The mechanics of an A/B test are straightforward:

• Define what you're testing: Change one variable at a time. A subject line, a headline, a call-to-action, an image, a price display. If you change multiple things at once, you won't know which change caused the result.
• Split your audience: Randomly divide your audience into two equally sized groups. Group A sees version A; Group B sees version B. The random split is important — it ensures the only difference between the groups is what you're testing.
• Run both versions simultaneously: Don't run version A on Monday and version B on Tuesday. External factors (day of week, news events, time of month) affect results. Simultaneous exposure is what makes the comparison valid.
• Measure the metric that matters: Decide before you start what success looks like. Open rate, click-through rate, conversion rate, revenue. Choose one primary metric per test.
• Wait for enough data: A test with 50 visitors is not conclusive. You need enough data for the result to be statistically reliable — typically hundreds to thousands of impressions per variant, depending on your expected effect size.
• Pick the winner and apply the learning: The winning variant becomes your new baseline. And the insight — what worked and why — informs your next test.

Why A/B testing matters

Marketing involves hundreds of micro-decisions. Subject lines, button text, hero images, ad copy, pricing display, landing page layout. Each of these decisions affects your results — and most businesses make them based on preference, convention, or gut feeling.

The problem with intuition is that it's consistently wrong in predictable ways. People are poor at predicting what will resonate with an audience that isn't them. Designers prefer their own aesthetic. Founders overestimate interest in their product's features. Sales teams overweight the arguments that worked on a handful of early customers.

A/B testing corrects for this bias systematically. The data doesn't care what anyone on your team thinks — it reports what your actual audience did.

• A subject line A/B test might reveal that curiosity-driven headlines outperform feature-led ones by 40% — changing how you write every email going forward
• A landing page headline test might show that addressing the customer's fear converts better than describing your product's benefits
• A CTA button colour test might be inconclusive — which is also useful information, telling you the colour wasn't the variable that mattered
• A flyer design test (via QR code tracking) might reveal which neighbourhood or which event format drives more actual visits

Small improvements compound over time. A 10% improvement in email open rate, combined with a 10% improvement in click-through rate, and a 10% improvement in landing page conversion rate, compounds to a 33% improvement in end-to-end campaign performance — from the same budget.

What you can test

A/B testing applies to almost any marketing touchpoint. Common test subjects:

• Email subject lines: The single highest-impact lever in email marketing. A subject line determines whether the email gets opened at all.
  ANew summer collection available now

  BYou asked for this — here it is
• Landing page headlines: The first thing a visitor reads. Test feature-led vs benefit-led vs problem-led framing.
  AOur project management software

  BStop losing track of what needs to happen next
• Call-to-action copy: Button text that describes the action converts differently than text that describes the benefit.
  ASubmit

  BStart my free trial
• Ad creatives: Image vs video, product-focused vs lifestyle, direct offer vs educational content.
• Send time: Tuesday morning vs Thursday afternoon. Different audiences have different peak engagement windows.
• Printed materials: Two versions of a flyer with different QR codes let you compare which design, message, or distribution location drives more traffic.

Common mistakes

• Stopping too early: A test that shows 60/40 after 100 visitors may show 51/49 after 5,000. Small samples produce noisy results. Stopping as soon as one variant looks better is one of the most common errors in A/B testing — it produces false positives at a high rate.
• Testing too many variables at once: If you change the headline, the image, the button colour, and the body copy simultaneously, you can't attribute the result to any single change. Change one thing per test.
• Optimising for the wrong metric: A subject line that generates maximum opens might do so by being misleading — leading to low click rates and unsubscribes. Always tie your test metric to business outcomes, not vanity metrics.
• Not running tests long enough: Week-over-week behaviour varies. A test run only on Mondays may not represent your typical audience. Run tests across at least one full week cycle.
• Treating inconclusive results as failures: A test where neither variant wins definitively is still valuable — it tells you that variable doesn't significantly affect your metric. That's information.

Running A/B tests with tracking links in Zapia

Most A/B testing tools require dedicated software — email platforms with built-in split testing, expensive experimentation platforms, or technical implementations on your website. For many marketing scenarios, tracking links give you the same answer at zero additional cost.

The approach uses the utm_content parameter — designed exactly for this purpose — to distinguish between two variants of the same campaign. Here's how to set it up with Zapia:

• Create variant A: In Zapia, create a tracking link for your destination URL. Set your utm_source, utm_medium, and utm_campaignas usual. Set utm_content to something that identifies this variant, like headline-a or flyer-blue.
• Create variant B: Duplicate the link and change only the utm_content value to identify the second variant — headline-b or flyer-red. Keep every other parameter identical.
• Distribute each variant to half your audience: Send email A to half your list and email B to the other half. Print flyer A and flyer B in equal quantities. Post ad creative A to one half of your ad audience and creative B to the other.
• Let it run: Give the campaign enough time to accumulate meaningful data — at least a few days, ideally a full week.
• Check your analytics: In your analytics dashboard, filter by campaign and compare the two utm_content values. You'll see exactly how many clicks, sessions, and conversions each variant drove.

This works particularly well for comparing printed materials. Two flyer designs, each with its own QR code pointing to a uniquely tagged tracking link — you'll know exactly how many people scanned each one, and what they did afterward.

To learn more about how tracking links work and what each UTM parameter does, read our guide on tracking links and why every campaign needs them.

Reading and acting on results

When a test concludes, you have three possible outcomes:

• Variant B wins clearly: Roll it out as your new baseline. Document what changed and why you think it worked — this insight is as valuable as the result itself.
• Variant A wins clearly: Your original was better. This is still a win — you've confirmed something that works and avoided a change that would have hurt performance. Document it.
• No clear winner: The variable you tested didn't significantly affect the metric. Move on and test something else. The learning is that this variable isn't a lever worth optimising.

The most important habit after each test: write down what you learned. "Subject lines with a number outperformed those without by 22% — hypothesis: specificity reduces uncertainty." That note becomes part of a growing knowledge base about your audience that compounds in value over time.

A/B testing is most powerful when it becomes a systematic practice rather than an occasional experiment. Even one test per month, consistently executed and documented, will substantially improve your marketing performance over a year.