After a great run, the Winning the Internet blog has been retired. However, you can still keep in touch with New Media Mentors here.
If you were given the choice of two presents of the same size – one wrapped in sparkling silver paper with a fancy bow, and the other covered in Sunday comics and topped with a bow made from duct tape – which would you choose?
Personally, I’d gravitate toward comics and duct tape, but I don’t assume that everyone shares my preference. Similarly, at PAN, we don’t assume we know which email subject line will most appeal to our online supporters.
This is the point of A/B testing nonprofit email blasts. Subject line testing not only helps you learn more about your list, it can make a big difference in how many viewers actually take action.
Back in December of 2010, Pesticide Action Network (PAN) sent out a blast asking our supporters to sign a petition urging EPA to suspend a class of pesticides (neonicotinoids) known to undermine honey bee immunity.
Honey bees are in decline, and colony collapse disorder had been a hot topic in the press at the time. So this issue resonated with our community – but to ensure we got the most bang out of our blast, we set up an A /B subject line test. It’s one of the simplest tests an organization can perform and the results not only helped increase eyeballs on our individual action, but gave PAN insights to how our list reacts to particular messages.
At the time our list size was around 36,600. So we sent the exact same email blast, but with two different subject lines, to 10 percent of our list. The results were enlightening.
Test A’s subject line: Take a stand with beekeepers scored a 19.8% open rate and 11.4% click-through rate.
Test B’s subject line: It shouldn’t take a ‘Wiki-Bee’ leak to protect pollinators came in at a 17.9% open rate and 9.2% click-through rate.
Take a stand with beekeepers won by a knockout. And for our test audience, that higher open rate resulted in 67 more viewers actually opening the email and 57 more clicking through to the action. While that might not sound like much, the difference when applied to our full list is critical.
At 19.8 %, the open rate for our full list comes in at 7,246. If we had sent it with the weaker subject line we would have only garnered 6,551 viewers, a difference of 695 sets of eyeballs.
Not bad. But it gets better. Using our sexy 11.4% click-through rate, we achieved 4,172 supporters going on to the action. Our clunkier subject line would have only pulled in 3,381 viewers, a difference of 791 activists making it to the petition and acting on behalf of our friends, the honey bees.
Why did Test A, Take a stand with beekeepers, perform so much better? We think it had to do with telling our audience exactly who they stood with, beekeepers, versus the more insider and technical Test B: It shouldn’t take a ‘Wiki-Bee’ leak to protect pollinators.
But to be sure PAN truly gleaned as much as possible from A/B testing, we kept going. We’ve performed 25 subject-line tests so far and collected the results in a simple spreadsheet. By scanning these test results in a single document, the takeaways become apparent. In PAN’s case, naming the target (whether EPA or companies like Monsanto) pays off. Keeping them punchy and intriguing, without being too poetic, also resonates with our audience.
And we will keep going. Our list has grown significantly since we started A/B testing, and as it continues to grow its preferences may change. We continue subject-line testing to monitor the temperature of our list and ensure that our emails stand out in people’s flooded inboxes.
Photo by Nicolas Raymond.
Comments are closed.