When it comes to e-mail testing, you can’t overplan or overanalyze, according to a recently released white paper from Experian Marketing Services. Released this week, “A Guide to Effective Email Testing” confirms what many of us already know: Companies aren’t testing as much as they should be and, when they do, they often have it all wrong.
“Practitioners do not pay much attention to testing,” said Tamara Gruzbarg, digital analytics manager on the Decision Sciences Team at Experian Marketing Services. “It’s critical to understand that comprehensive planning is the key to success.”
Gruzbarg highlights these four tips that marketers can use to boost their testing power.
1) Match the test to your objective. Testing subject lines won’t affect opt-out rates as much as open rates. Frequency testing, however, absolutely can, Gruzbarg said. This is why it’s important to figure out which metric you’re trying to improve before selecting a testing strategy, she said.
2) You may need more than one control cell. Recently, an Experian client ran a test with a control group and a subject group. The control group got a message with a single image; tyhe subject group got a message with three images. The subject group outperformed the control group, and the client thought it had a winning design. It was wrong, Gruzbarg said. “It could have been the first image that did well; it could have been the third image,” she said. “There was no way to figure out which new element was actually the reason behind the lift.” The lesson? Test new elements one at a time, and if you do want to test everything at once, consider multivariate testing—simultaneous testing of multiple elements and variables—which provides much better feedback than A/B split testing in cases like this.
3) Frequency testing can be very helpful if it’s done right. Before you start blasting out different numbers of e-mails to different customers, take note: Unless you segment your list by how engaged recipients are, this type of testing won’t result in any meaningful data, Gruzbarg said. “Frequency testing should be stratified based on the engagement levels of the list. Certain segments that are highly engaged might need much higher frequency levels,” she said. In addition, you might decide to suppress or delete parts of your list that aren’t responding to frequent e-mails, but unless you analyze your customer data, you may be deleting revenue. “For b-to-b especially it’s important to pay attention to the seasonality effect. Does your client have a business that only requires purchases at certain times of the year? If so, those are the times they may need more frequent e-mails,” Gruzbarg said.
4) Timing tests are important. Over the years the “perfect” day and time of an e-mail send has been debated and discussed, but marketers shouldn’t just go with what industry pundits say about the subject, Gruzbarg said. Again, do a little analysis before you do any testing. “Are your customers strictly 9-to-5 businesses? Are they sales agents who have an unrestricted workday? You need to think about what kind of schedule they are on, and what time zones they are in before you put them into control groups.”
“A Guide to Effective Email Testing” is posted at http://www.cheetahmail.com/corp/resource/wp/effective_email_testing_wp.html.
|
