Monday to Wednesday, 10am - 4pm, ET
1.855.281.5499 (toll free)

How (and Why) to A/B Test Your Landing Pages: Part 1

SEOWebsitesMultimedia & Design

By: Jasmin Bollman,

Picture it: you are about to launch a new fundraising campaign that you have spent countless hours perfecting. You’ve got the beautiful landing page complete with copy that you are certain will compel people to donate, sign-up, or share. The imagery? Eye-catching. The call-to-action? Motivating. It’s time to press publish on your landing page.

But then nothing happens.

Why aren’t people connecting with your campaign the way you were certain they would? Enter A/B testing to give you those all-important answers.

Back-to-basics: What is A/B testing?

A/B testing is the art (and science) of creating multiple versions of a landing page or email that differ from each other slightly. In marketing-speak, it’s a method of gathering information to optimize interactions and conversions.

For an email, this could be a different subject line or a different “From” address. For a landing page, you can test any number of important items, from the main imagery to the call to action (more on the type of things you can test later).

Why A/B test?

Let’s face it – we all have inherent biases, whether we are cognizant of them or not. A/B testing allows you to understand what compels and motivates the greatest number of people to take an action, helping you to confirm (or disprove) what you think they’ll like.

We all like being right, of course, and often these tests will help to prove your initial thinking. What makes A/B testing so vital, though, is that it will often surprise you and provide you with insights to make the decisions that will propel your campaign forward.

What to test

There are any number of things you can test on your landing pages, including:

  • Imagery
  • Lead headline
  • Call-to-action
  • Page length
  • Page layout
  • Videos

Don’t go overboard and test too many variables, as it will lead to confusing results. That is to say, each variation should only have one item being tested (after all, if you test multiple things on one page, how will you know which one motivated someone to act?). 

In the following example, you can see that the only difference between Version A (the control) and Version B (the variation) is the location of the signup form on the landing page:

Testing sign up form location

Figure 1 source: Instapage

How long should my test run?

A good A/B test, say the experts, should last at least a few weeks in order to gather the most insights. If your test only runs for a few days (or worse yet, a few hours), your results will be incomplete. I wish I could tell you that I have never done this, but in my email marketing adventures I have made the error of running an A/B test that was only 24 hours. The result? The “winning” variation that was selected ended up not being the real winner in the end. Looking back at statistics a few weeks after the test concluded proved that the “losing” variation was the one that ended having the higher open rate.  The lesson: take your time. You want your results to be statistically significant (that also means you drive enough traffic to the pages you’re A/B testing). 

The industry-standard level of statistical significance is 95%. In the example from Moz below, you can see how long each variable they wanted to test took to reach that magical percentage:

Multi-variable testing

Figure 2 source: Moz

Can I A/B/C test?

If you want to test more than one variation, go ahead and publish 2 different versions of your control. However, you’ll only want to test one variable across all of the landers – don’t test the CTA on one, the imagery on another, and the location of a sign-up form on the third. Stick to one variable across all three. Otherwise, how will you know what really moved people to act?

But what about….?

When it comes to A/B testing landing pages, there are lots of things to think about. As you read this, you probably had many questions pop into your head, such as, “How will this affect my SEO?” and “What can I do to prevent Google from indexing my test page?”

All great questions, and ones that will be covered in part 2 of this blog. But for now, your homework is to determine what you want to A/B test and define your hypothesis. Do you think more of your site visitors will connect with a video explaining your fundraising campaign or will they want to see images only? Then, start to design what that test will look like using your website building software (many of which have built in a/b testing features already). 

Next, we’ll tackle how to interpret your test results and how to handle that pesky SEO problem.

About the author: 

Jasmin Bollman is the Marketing Manager of Social Media and Content for With a background in journalism, she spent several years working behind-the-scenes in public relations before making the leap to full-fledged brand builder. Her love/hate relationship with Twitter has given her a unique way of looking at marketing and how brands can stand out online. She also moonlights as a freelance writer and is an aspiring introverted inspirational speaker.