testing

q. what variables should be considered when testing?

a. the things to test are those that usually have the greatest impact on response rate: offer, price, premium, creative (including strategy, format and copy), seasonality and list selection. remember, of course, to test one element at a time, so you'll be able to clearly link cause and effect. first, test more substantial differences in the approach you can fine-tune your program later on by testing finer points. (mail is a responsive medium, and you'll often achieve profitable results quickly; your next level of testing then becomes a matter of seeing how you can make a "good thing better.") by testing with small but statistically significant mailings, you can keep costs to a minimum. modestly priced testing can also encourage you to explore innovative approaches that might turn out to be breakthrough winners. you'll find testing to be a fascinating, instructive and highly profitable exercise, full of marketing and advertising lessons you would not have gained otherwise.

q. when should you test?

a. there are as many answers to this question as there are companies with mailing programs. but here are some general guidelines:

  • when you want to fine-tune a successful mailing to improve results even further.
  • when your cost-per-order or cost-per-inquiry isn't what you had hoped.
  • when you're presented with new creative that you feel might be a significant performer but (wisely) requires more than gut instinct to justify a major rollout.
  • when you seek to expand your market via wider-ranging list selection.
  • when something in your marketing mix changes a different price, a new offer, a promising premium.
  • When there's a new-product introduction.

q. how often should testing be done?

a. you want to have an ongoing testing program. As you know, the marketplace is constantly changing and testing is a powerful tool for keeping up with changes. Also, testing supplies knowledge, and no marketer can have too much of that. Hunches, intuition and artistry will always play an important role in our discipline but so must the realities of response rates and numerical analysis.

q. what kind of response is necessary for a test to be reliable and projectable?

a. here again, there's no answer that applies to every company and every program, but there is a good rule of thumb. In order for a test to be projectable, you should receive a minimum of 1% in the mailing in question. For example, on a 5,000-piece mailing, this would represent 50 responses.