Split Testing Resources

Category: Free Placement, Paid Placement, General Marketing Knowledge

Summary & Overview

A/B testing, split testing, or bucket testing is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response rates. A classic direct mail tactic, this method has been recently adopted within the interactive space to test tactics such as banner ads, emails and landing pages.

Significant improvements can be seen through testing elements like copy text, layouts, images and colors. However, not all elements produce the same improvements, and by looking at the results from different tests, it is possible to identify those elements that consistently tend to produce the greatest improvements.

Employers of this A/B testing method will distribute multiple samples of a test, including the control, to see which single variable is most effective in increasing a response rate or other desired outcome. The test, in order to be effective, must reach an audience of a sufficient size that there is a reasonable chance of detecting a meaningful difference between the control and other tactics: see Statistical power.

This method is different than multivariate testing which applies statistical modeling which allows a tester to try multiple variables within the samples distributed.

Setup Time
Less than 1 Hour.

Ongoing Maintenance
Split testing your marketing is a critical campaign management strategy that will assist you in increasing your profit margins. Once you have grasped the basic concept of split testing you should consistently evaluate, split test and improve your campaigns and websites on an ongoing basis.

Cost
Varies depending on the campaign or website you are split testing.

Speed of Results
Varies depending on the amount of traffic directed to the campaign or website.

Basic Use
As a marketer you may get lucky and hit on the “goldmine” ad your first try, but usually writing effective ad copy that generates the results you desire is a process of gradually improving an ad that works well into an ad that works great. Split testing is a simple, straightforward methodology for improving your ad copy systematically in order to improve your results.

Basic Action Steps
For this demonstration, we’re going to examine a standard 3 line ad with:
• Headline
• Body Copy
• Domain/URL

Assume you’ve got an ad that is pulling well and you are now ready improve that ad to increase your results. You are going to write two more ads and throw them in to the same rotation with your original ad and then measure which ads pulls better. The key to split testing is creating an environment where you can identify the single element in your ad copy that is influencing different results. If you’re in to statistics you would call this a “control”.

Step One: Start with your original ad copy. This is your “control”. Nothing changes.

Step Two: Next, write 2 new headlines and insert them above your original body copy and URL. These are your “tests”. For example:

Control Ad
This is my original headline.
This is my original body copy.
This is my original URL.

Test Ad 1:
This is my new headline #1.
This is my original body copy.
This is my original URL.

Test Ad :
This is my new headline #2.
This is my original body copy.
This is my original URL.

Step Three: Give your campaign time to experience significant traffic and then simply evaluate your click-through-rate (or other measurable action) for each ad. Once you have identified the best performing headline, begin split testing your body copy.

Control Ad:
This is my best performing headline.
This is my original body copy.
This is my original URL.

Test Ad 1:
This is my best performing headline.
This is my new body copy #1.
This is my original URL.

Test Ad 2:
This is my best performing headline.
This is my new body copy #2.
This is my original URL.

Step Four: Give your campaign time to experience significant traffic and then simply evaluate your click-through-rate (or other measurable action) for each ad. Once you have identified the best performing body copy, begin split testing your domain/URL.

Control Ad:
This is my best performing headline.
This is my best performing body copy.
This is my original URL.

Test Ad 1:
This is my best performing headline.
This is my best performing body copy.
This is my new URL #1.

Test Ad 2:
This is my best performing headline.
This is my best performing body copy.
This is my new URL #2.

Step Five: Give your campaign time to experience significant traffic and then simply evaluate your click-through-rate (or other measurable action) for each ad. By split testing the headline, body copy and URL separately you should now have a great performing ad.

Basic Tips & Tricks
• IMPORTANT: Ensure your three ads are rotating equally as they must get the same exposure in order for you to evaluate their results. This may work differently depending on the distribution network you are using (i.e. Google, Yahoo, etc.) but you do not want the network to optimize delivery of your ads – that is show the better performing ad more often. If your ads are optimized it will skew your results!
• Don’t overlook simple changes such as capitalization and punctuation … they can make a surprisingly big difference in response rates.
• Remember, the purpose of ad copy is not to sell or explain, the purpose is to compel your reader to take the next step (i.e. click on the ad, call the telephone number, fill out the form, etc.). Good ad copy generates curiosity!
• Always remember your domain name/URL is a critical piece of your ad copy. Invest in multiple domain names (URLs). They’re inexpensive and registrars such as Godaddy.com routinely have sales and send out discount coupons. Never cut corners on the purchase of a new domain by adding a subdomain name (i.e. www.mywebsitename.com/google). The more closely your URL aligns with your ad copy and keywords/search terms the better the results. Domain names should be professional, hint at what’s coming next and, if possible, plant a seed for buying.
• Consider what type of person your ad is attracting. Do you want 18 to 24 year olds or baby boomers? Which demographic converts better? If you don’t know your demographics you can always go to www.Alexa.com and click on “Audience”. To write effective ad copy you must learn to speak your demographic’s language and connect with their pain.
• One word about “Free Offers”: if you include the word “FREE” in your ad you might notice a marked increase in clicks on your ad (thereby costing you money) without a corresponding increase in sales. Use the word “FREE” carefully and be sure to check your conversions in order to determine if the increased traffic is profitable.
• Split testing is a game of “beating your best effort”. Once you have identified the ad with the best pulling headline, body copy and URL begin the process over again, starting with your headline.

• Keep track of each ad variation and its statistics in a spreadsheet. Tweaking your ad copy and split testing your ads is an ongoing process and you will want to review past variations to ensure you’re not reinventing the wheel as you continue to “beat your best effort”!
Intermediate Strategies
Once you have traffic going to your website, split testing may become your most important weapon. It is actually a marketing strategy unto itself because it can dramatically increase conversions and sales overnight. Split testing your website works just like split testing your ad copy: you simply change one element on your site and then alternate traffic between your old page (the control) and your new page (the test). The page with the highest conversions is the one you keep (your new control page) and you will select another element to split test on your new test page.
Sales funnels are traditionally structured with a few common elements:
• Opt-in Page
• Sales Page
• Order Page
• Folllow Up (Autoresponders, etc.)

Inside each of these areas are infinite split tests to be run. Split testing will turn a good marketer into a great marketer as this strategy gives the marketer the power to take an idea, test it against the control page, analyze the results and pinpoint small, but impactful, improvements that ultimately will increase sales conversions and profit margin. Split testing puts you, the marketer, in total control of your campaigns and their subsequent effectiveness.

Intermediate Action Steps
You may be asking yourself “where do I start?” This is not uncommon and the best answer is, you can start anywhere. Here are some ideas for elements on your page to test first for the quickest effect on results:

Guarantees:
Offer the longest guarantee you possibly can. A long guarantee will increase your conversions and decrease your refund requests. You want to give your customers time to realize the value in your product. If they believe they only have a short time to determine if your product will work for them chances are they won’t give it the benefit of the doubt if their refund window is closing in on them. Split test the following guarantees against each other:
• 30 Days
• 90 Days
• 6 Months
• 1 Year

Certification Seals:
Statistics show that conversions are increased dramatically with a “seal of approval” on the site. A certification, or seal, instills trust in your prospective customers that the site is safe and that you are trustworthy. This is especially important when you aren’t dealing with your customers in a brick and mortar environment. One good certification resource is the Online Business Bureau, at http://www.onlinebusinessbureau.com.

Headlines:
A headline is generally regarded as the single most important factor for increasing website conversions. Statistically the best performing headline is in the Tahoma font and sized at 20 points in Red ( HEX={cc,00,00}).
Experiment with your headline copy, starting with a variation of these options:
• In the next three minutes I will show you the dirty little secret to …
• Shocking discovery …
• Who else wants to learn …
• My neighbors wouldn’t leave me alone when …
Readers have a short attention span when it comes to deciding whether to stay on your site or click away. You must capture their attention within the first TWO SECONDS of their landing on your website or they will leave. If you address their pain and offer a quick solution in your headline, chances are they’ll stick around and see what you have to say.
“Quotation marks” around headlines is an effective strategy. It tells your prospective customer that you’re talking to them.

Audio and Video:
Adding audio and/or video to your website is an excellent method to increase your conversions. While there are very effective internet marketers who have made a fortune using shock value (i.e. being rude, offensive, etc.) they are few and far between. If you have the personality to pull this off, by all means try it. We generally recommend, though, that your goal with audio and video is to (a) be likable to your audience, and (b) instill trust in your audience. Audio and video can be tricky, which is why it’s so important to split test your pages with, and without, each element. If you find that your video tests do not convert well, simply go with an audio that auto-starts when someone hits your page. Be relaxed, authentic, and loud. This is your chance to connect with your prospect.

Colors & Fonts:
Make sure your color and font selections are not overbearing or distracting. Stick with 3-4 colors on your page and don’t use more than 3 different fonts on your page. As discussed above, headlines generally convert well in large red text. Your body copy should always be black in an extremely legible font and size. Body copy should spaced out (3-5 lines at a time) so that it’s easy to read even on the smallest monitor. Background colors should be simple - off white, robin’s egg blue, simple black, etc. Sites that are too busy or confusing will cause your reader to click away before they ever read a word on your page. Statistics show the confused mind says “No!”.

Photographs:
Split test your site’s photographs. Pictures of groups generally make people feel like they’re part of a team or a movement. They feel included. Test pictures of smiling people against more serious photographs and monitor the results.

Autoresponders:
You can also test your autoresponders, which you should be using to build trust with your customers or your prospective customers. Staying in close contact with your new customers (especially those who are still waiting to receive their product) will greatly reduce your refund requests. Consider teaching your customers or prospective customers something or showing them what they should be doing next, step-by-step. If they are a brand new lead, give them suggestions for books and resources they will find helpful. If they’ve just purchased a product from you, tell them what they should do next, where they should start, what they should expect to learn or accomplish and when they can expect to hear from you for ongoing support. Testimonials are great in autoresponders. If possible, try to closely align your testimonial choices with your mailing list's audience. You can actually customize your mailing list by demographic and tailor your testimonials accordingly.

Related Marketing Strategies
• Banners
• Directories
• Free Classifieds
• Google

For questions about Split Testing, please send an email to resources@michaelwyrosdick.com with “Split Testing” in your subject line.