Pushnami uses cookies to enhance your experience. We use these for advertising and analytics purposes. By continuing to use our site, you agree to our use of cookies.

Should You A/B Test With Segments?

Should You A/B Test With Segments?

There are two things anyone working in push is told to do: A/B testing on their customers, where you change one aspect of your copy and test it on a randomly selected group. Segmenting their customers, or dividing them into groups based on various characteristics. The advice is right; segmenting helps you better understand your customers, while A/B testing keeps copy fresh and can point you towards more effective ideas. But don’t they conflict? In truth, segmenting and A/B testing are layers, one resting on top of the other, and may lead you to some fascinating insights, if you do it just right.

Should You A/B Test With Segments?

Posted on Friday, May 18, 2018

A/B testing is a great thing, but does it pair well with segment marketing?

There are two things anyone working in push is told to do:

  1. A/B testing on their customers, where you change one aspect of your copy and test it on a randomly selected group.
  2. Segmenting their customers, or dividing them into groups based on various characteristics.

The advice is right; segmenting helps you better understand your customers, while A/B testing keeps copy fresh and can point you towards more effective ideas. But don’t they conflict? In truth, segmenting and A/B testing are layers, one resting on top of the other, and may lead you to some fascinating insights, if you do it just right.

Segment By Segment

The top “layer” of any marketing approach needs to be segmentation. Your customers are not a monolithic block of people who engage with you for the same reasons and act the same way every time they visit your website. They are a diverse group of people, from across the globe, using different technologies to engage with you, and who do so for wildly different motivations. Walk into your local coffee shop, and you’ll see people using it as a place to work on their own, as a place to meet a date, as a place to conduct informal professional meetings and, oh yeah, a place where you can buy a coffee. And how you use that coffee shop might change from day to day, depending on anything from your schedule to your health.

That makes segmentation important because it helps you better understand your audience. As we get more personal and develop a better understanding of our customers, they, in turn, want us to view them as more than an invoice. Customers want to feel understood and heard, and A/B testing is part of that.


Segments build the targets you A/B test on.

Always Be Testing

The joke about A/B testing is that it stands for “always be testing,” and that’s true, but it’s also important to test with a specific goal in mind. You can fiddle with your copy in a dozen different ways, but A/B testing needs focus, and that’s where segments come in.

Sticking with the coffee shop example, let’s say you’re trying to sell more carafes of coffee, instead of individual cups. You’d approach each segment differently, but within that segment, you can engage in A/B testing. For example, for half of your customers holding meetings, you might push out a preorder link with the copy; for the other half, push a coupon for a free carafe. Then keep an eye on preorders, and how often the coupon is used, and that’ll offer you some useful data on that particular segment.

You can go much more granular than that, of course, but the goal here is to test within the segment, to better understand what appeals to them and why. A/B testing is a useful way to gather small little data points on each segment, as you learn what they engage with and don’t, and it ultimately makes a big difference. Think of segmentation as what you learn about a person before you arrive at a meeting and A/B testing as the small talk that fills in the rest. If you take opportunities to ask questions, and to listen to the answers, your customers will have plenty to tell you. To see the power of segmentation and A/B testing in action, sign up for a live demo of Pushnami!