How to Analyze & Test Direct Mail Campaigns
There’s an age-old theory in marketing that a prospect needs to see or hear your message multiple times before they will act. Just like any other marketing channel, this concept holds true for direct mail.
Companies that consistently stay in front of their audience tend to get more business. However, reaching out to prospective customers over and over again with the same ineffective message can end up turning them away and costing you money.
But how do you discover which components of your direct mail piece are working and which aren’t? The answer to that question isn’t mind reading – it’s testing.
Follow the advice and suggestions below to learn how to analyze and test your direct mail pieces to create the most compelling advertisements for your audience.
Figure Out What to Test
If you’re not happy with the results of your direct mail, don’t be discouraged. It’s not hard to improve your campaigns, but first you have to figure out what’s going wrong. To diagnose the problems with your direct mail, try out some of the suggestions below.
Consult Past Data
Without data, it’s impossible to determine which tests you should perform. Before proceeding, it’s essential that you’ve collected data from previous campaigns, or at least know how to. For more on that topic, check out our guide on how to track and measure direct mail campaigns.
Once you’ve gathered data from your past campaigns, start asking yourself some questions. While they’ll vary based on your business and marketing goals, here are a few basic things to consider:
Did you lose money on your last campaign? If your cost per acquisition was higher than your revenue per order, then you lost money. You might consider mailing fewer pieces to a more targeted list to lower your CPA and improve response rates.
Was your response rate low? If so, are you trying to sell a complicated or expensive product? Would the mailing have performed better if you’d targeted a particular demographic or neighborhood? Were the coupons or offers you included enticing enough?
Was your response rate high but conversion rate low? If so, your mail piece might have done a good job generating interest, but ultimately the offer or the product you were selling wasn’t appealing enough for recipients to take action.
Think About Segmenting Your Audience
Sometimes, the biggest issue with your campaign is who you’re mailing to. Try audience segmentation to target neighborhoods or specific demographics that are a better fit for your products or services.
If you mailed your piece to every address within a five-mile radius but saw a poor response, you could try removing certain address types, like apartments, P.O. Boxes, and businesses. Alternatively, you could limit your list to households that fall within a desired demographic, like average income, property size, age, car type, etc. You’ll pay more for a targeted mailing than you would for saturation, but depending on your product or service, this can be more economical than mailing to every address in an area.
Understanding your ideal customers’ demographic and behavioral information will help ensure you’re getting your mail to the right people.
Take a Hard Look at Your Direct Mail Piece
If you’re overwhelmed by the data and don’t know where to start, try taking a closer look at your direct mail piece. It can be helpful to ask friends, family members, current customers, or subject experts to review your mail, since you might not have the objectivity necessary to find errors. Ask yourself and others the following questions:
Was the offer good enough? Maybe your design, messaging, and CTA were perfect, but you only offered $1 off a $20 product. Customers will only act if they view the offer as too good to pass up. Make it so valuable that your prospects can’t say no, and present it so they understand what you’re giving away and how they’ll benefit.
Was there a clear call to action? Maybe your piece did an excellent job of explaining your services but didn’t prompt the reader to do anything. Make sure there’s a clear CTA, and make it stand out from the rest of your mailer with bright colors or high contrast. It should be easy to find at a glance, and even easier to understand.
Were there any stumbling points in the design or messaging? If the design isn’t clean and logically formatted, readers may be confused or ignore the piece altogether. And if the copy on your piece lacks clarity and focus, prospects might not understand your marketing message.
Did it stand out? Your audience sees a ton of advertisements throughout the day, which means your piece must grab their attention. Color can have a significant subconscious effect on readers, so make sure you do some research on the psychology of color to create a design that’s visually striking and appropriate for your product or service.
Test Your Revised Direct Mail Pieces
Now that you know what you’d like to change, you’re ready to begin testing, right? Not quite. Before you can send out your next campaign, you’ll have to lay the groundwork for a successful test. This helps ensure your results are valid, which enables you to draw more accurate conclusions.
Establish a Control Piece
Before you run your first test, you have to establish a control piece and baseline metrics to judge the results of your future campaigns against.
If you’ve run direct mail campaigns in the past, you can simply choose a piece with a proven record of success. However, if you’re just starting out, you’ll want to put together a mail piece with a compelling offer, clean design, and a clear call to action. Send it out and gather your baseline metrics before you begin testing.
As you test and refine your direct mail piece over time, you’ll want to adopt your new best performing piece, or “winner”, as your new control piece so you can continually improve. Just make sure you test your “winner” in multiple scenarios before you make the switch.
Choose Which Type of Test to Run
With a solid control piece to measure against, you must now decide which type of test to run. In most cases, you’ll be choosing between two types of tests: A/B (or split) tests and multivariate tests.
Regardless of which type you choose, you’ll need to send out your control piece alongside your test pieces each time you experiment. Otherwise, your results will have no context and you may draw inaccurate conclusions.
A/B Tests
Conduct an A/B test (also known as a split or bucket test) by sending out two direct mail pieces: one control and one variation. In other words, both direct mail pieces will be identical except for one difference, such as a different offer, call to action, mailing list, or format.
You’ll be able to determine if your change made a difference based on the response you get to each piece.
A/B tests are most appropriate if you find yourself saying, “I want to change the offer to make it more enticing.” Or, “I want to see what response we get by mailing half our pieces to this list and half to another.”
A/B/C tests are also an option and would involve testing a third variation of whatever component you want to change.
Multivariate Tests
If your time between campaigns is limited and you have many ideas on what could improve your mailer, a multivariate test may be better.
Multivariate tests involve testing multiple components of your direct mail piece at the same time and comparing them to your control piece. These kinds of tests are better for direct mail programs in their infancy stage when you may have no idea which components are working. If you find yourself saying, “I want to change the CTA and the offer. And should we update the design?”, multivariate testing may be your best bet.
The more components you test, the more test pieces you’ll need. For example, say you want to experiment with a new call to action and a new hero image. You would have four variations to test, and each of those variations would need unique tracking methods so you could accurately measure their performance.
To keep track of all the variations, you can use something simple, like this example testing matrix:
Send Out the Campaign and Measure the Results
Once you’ve set up your test and printed all the variations, you’re ready to mail out. As the results come in, compare your new set of data with your previous campaign’s performance. Did that new CTA improve your conversion rate? Did the new hero image resonate more with your audience, resulting in an improved response rate?
Depending on your results, you may need to go back and test the same components again, or you may be ready to move on to a different element of your mail piece. There’s no such thing as perfection, so you should always be able to identify areas where you can improve.
Conclusion
Testing is an ongoing activity, not a one-time event. When you’re evaluating your direct mail campaigns, you should approach the process objectively. Let the numbers inform your decisions, even if your test results clash with your intuition.
Over time, you’ll discover what influences your customers to act and which design elements make your piece stand out from the competition. Just be patient and continue refining each time you send out a mailing. And remember to always measure and compare your results to track your improvements.