Why and How to A/B Test Your Push Notifications
Constant testing means better results.
You hear constantly about A/B testing in push notifications, but rarely do people stop and explain what it is. What is A/B testing? How do you do it? Why should you do it? Here are some quick tips on how A/B testing works.
What Is A/B Testing?
At its simplest, A/B testing is changing one part of your notification, trying the change out on a similar group, and comparing it to previous results. What you change could be anything, such as the product you link to, the copy, the punctuation, or whatever you are thinking of changing. You try your control, that is, what you had before, and your new notification and see if one works better than the other.
If this sounds a bit like trial and error, it is! However, it is always worth doing. Even simple changes can have a surprising effect. For example, consider “20% off all products.” versus “20% off our very best!” One might get a stronger reaction than the other, but you are not going to know which one performs best until you try.
Why A/B Test?
A/B testing lets you change with your users. Your userbase is not a giant monolith but a constantly shifting mass of individuals, who each react differently to every element of your campaigns. Users will drift away, replaced by entirely new users, and sometimes an enormous audience shows up thanks to a sudden breakthrough or other surprises. It is worth exploring who reacts to what and try to puzzle out why from everything you know.
How well do you know him?
Users also shift due to outside factors. To draw from another part of web marketing, remember when the internet was infested with headlines like “You Won’t Believe What Happened Next!”? These “clickbait” headlines worked at first, but eventually, people decided that they did believe what was going to happen next, and stopped clicking. The same is true of any other form of copy. There are some ways to cater to users that work. There are others that do not. Complicating the issue, those goalposts move constantly as your audience changes.
How To A/B Test
Fortunately, running A/B tests is a simple process. First, choose the metric you want to measure. It is best to pick one you are interested in and focus on that, although broader metric tracking might make sense in some cases. Then, pick a marketing segment and divide it into two randomly. Next, look at the notification you are about to send out, and consider that segment. Think of a tweak you would like to make according to that segment. This can be as simple as changing words and punctuation.
Then, push out the current notification to one group, and your tweaked one to the other, and watch what happens. It is that simple!
The joke about A/B testing is that it really stands for “always be testing,” and the truth is that this is not far off the mark. A/B testing can yield dramatic results, so design and run those A/B tests. You might be surprised by what you find. Ready for better browser notifications? Sign up for Pushnami today!
