As we’ve learned, an A/B test is where we change one element of a webpage, for example, in order to see if the change has an effect on conversions or another success metric. To take this one step further, these two variations need to be tested by two different groups of buyers, one variation on one group and one variation on another.

This is where the concept of segmentation comes into play. Segmentation brings a level of focus to your test that you could not obtain without it.

Segmentation is the grouping of prospective buyers based on their wants, needs, and attributes. The thought behind this is that those with similar wants, needs, and attributes will also have similar buying behavior and will respond similarly to a change during an A/B test.

If you don’t segment it’s as though you are treating your entire audience as one person. Doing so could negatively affect your A/B test results, since with A/B testing, you need to be as specific and as in control of the variables as possible.

Your buyers vary greatly, and so in order to draw solid conclusions from your A/B test, you must first segment them. With that being said, here are four common segmentation approaches to consider, according to Conversion XL:

• Segment by source: Separate people by which source led them to your website or other channel, e.g. did they land on your website by clicking a paid ad on a related site, or did they reach your site by clicking on a link that popped up in their Facebook newsfeed?
• Segment by behavior: Separate people by how they behave when using a certain channel—which actions do they typically take, and which do they typically avoid? For example, they may often be compelled to click on a CTA that offers a product discount, but they may seldom click a CTA that simply encourages them to “learn more” about the product.
• Segment by outcome: Separate people by the products/services they’re interested in or regularly purchase or by the type of event they typically register for. For example, they may attend every webinar your company holds, which suggests that they’re very interested in your product/service, but they may stay away from the networking parties.
• Segment by demographic: Separate people by their age, gender, location, or other defining qualities.

Let’s take a look at an example. In segmenting by demographic, you could perform an A/B test on two groups that both consist of people who are 18-25 years old. The objective is to make sure that the groups mirror one another as a way to maintain control over the conclusions drawn from the test.

You wouldn’t want to perform the same test on two different demographics because then you wouldn’t know whether the outcome of your test was a result of the variation in element or the variation in demographics. Therefore, setting up the test without segmentation will likely lead to skewed results. This works against your attempt to be in control of your A/B test at all times.

A/B Testing Best Practices

Just like with any marketing strategy, there are a set of best practices that marketers should adhere to in order for A/B testing to do its job and do it well. Here are some best practices for you to consider:

• Consult your co-workers: Ask your co-workers for their input when creating tests. It’s a good idea to gain insight into what needs to be tested from those on the front lines. Consult members of different teams to cover perspectives from across the board.
• Test the entire customer journey: It’s easy to get caught up in testing elements on webpages and other channels that only pertain to the early stages of the customer journey. Why? Because the most energy from your marketing team as a whole tends to focus on the development of these initial attention-grabbers. But as an A/B tester, it’s important to test elements on webpages and other channels that pertain to all stages of the customer journey, so that the entire journey is optimized.
• Test one-by-one: This point may be repetitive, but it’s an important one. Only test one element at a time so that you’ll be sure as to which variation is responsible for the change in conversions.
• Test incrementally: We’ve heard that slow and steady wins the race, and with A/B testing, this couldn’t be truer. Before getting started, you need to map out a strategic plan of attack. It would help to draw a tree showing what exactly you will test and based on the results of those tests, what you will test next, and so on and so on. The point here is to test several elements in a particular order, all leading up to the end where you can draw a firm conclusion and validate (or invalidate) your hypothesis.
• Be realistic: Not every test will produce slam-dunk results every time. And honestly, with good A/B testing, this is the way it should be. What’s important is that there is subtle positive change with each test. Therefore, with a combination of many tests, a much more telling result will present itself. This is the real goal of A/B testing—drawing conclusions about your customers based on the bigger picture instead of just based on one isolated test. So, know that good things will come as long as you remain patient.
• Test in full: On the topic of patience, here’s another best practice for you. Even if your test is yielding good results right off the bat, you should always see it through to completion. This means to test for the duration you had originally planned on or until you reach the number of visitors you had originally decided upon. Why? Not only will you be able to see how your users are interacting with your webpage or other channel, but you’ll also be able to obtain stronger data, which can be used to better back up your recommendations to company stakeholders.
• Go with your instinct: If you’re not convinced by the results of a test (i.e. it greatly differs from your hypothesis and/ or doesn’t make sense), then don’t be afraid to run the test again. Odds are, your instinct is right. When re-testing, assess how you set up the original test and correct any technical mistakes. Remember, just a millimeter of difference in set-up can dramatically affect the outcome of an A/B test.