Mailchimp Tactics: How-to Guide to A/B Test your Farm’s Email Campaigns
With the Barn2Door and Mailchimp integration, Farmers can easily manage contacts, add products to newsletters and automate, test and track the success of email campaigns. Whether you are a beginner or are a ‘well seasoned’ marketer, the best practices below are useful for any Farmer seeking better insight to the effectiveness of their campaigns.
In this blog we will cover the basics of A/B testing email campaigns, including: (1) sample size, (2) testing variables and (3) best practices. We also provide a step-by-step guide on how to run tests in Mailchimp.
A/B testing is when you send multiple versions (usually 2) of an email to a small segment of your contact list, then measure which email performs better. The test winner is then sent out to the remainder of your email list. This process is made easy by the tools and functionalities built into Mailchimp.
1. Sample Size
To get accurate results from a test, you will need a reasonable ‘test size’. The larger the test group, the more accurate the results. Ideally your test size is at least 100+ emails. Even better test results occur if you send 1000+ emails. Whatever number you start with, use the same number for your A Group and your B Group! Once you learn ‘what works best’, you can incorporate that ‘winning email’ into your next test or email campaign.
Try to run an A/B test with a newsletter as well as a promotional email to get comfortable with the process! Only test one component at a time so you are able to get a better idea on the impact of each variable. If you are sending to only a subset of your contacts, send the test then wait until you have collected enough data to send the ‘best of the 2 emails’ to the rest of your email list. In all cases, be sure to plan with the delivery timeline in mind for consistency.
2. What Variables to Test
Focus on just one variable per test. Below we’ll run through some common areas to test in your email campaigns and newsletters.
Test this Variable: Your Subject Line
The subject line is one of the most important aspects of your campaign to test, given the downstream impacts on click-through and reply-rates. Subject lines that are stale, impersonal or too lengthy have the lowest performance. Whereas subjects lines that are specific, personal, inquisitive, seasonal or fun can produce exceptional results. Here are some different subject line styles and the intent behind the phrasing.
Time sensitive: We’re in your area today
Exclusive: Special Offer To Subscribers
Personalized: Thanks for stopping by, Liz!
Seasonal: Did you survive Winter Storm Harper?
Puns: Steak your claim, beef this good is rare
Using emojis: Plucky Cluckers Delivered Daily 🐔🐔🐔
The ‘Sending Time’ Variable
The goal of testing ’when to send your email’ is to gauge when customers are most active --- most likely to open your email and make a purchase. Consider testing different days of the week as well as different times of the day. One way to help narrow it down is to see which days and times the most orders are being placed in your Barn2Door store on your Dashboard.
Identify a few day and time options that you can start A/B tests, then determine the best combination for your Farm and your Buyers. Remember too, ideal send times might vary by customer type or groups. CSAs or a Mom’s Group in a ‘neighborhood’ might expect to shop on Sunday nights; whereas your Chefs are always thinking about supply on a Monday or Thursday morning.
Test ‘Body’ Variables: Content, Images, Product Links
The Body of your email includes everything from links to shop buttons, to content to formatting and imagery. There are many ‘variables’ and opportunities to test in the Body of an email. Give yourself permission to learn what works, over time. Remember it is important to test one variable at a time.
Keep your entire newsletter 100% identical except for a call-to-action (CTA), image or video. For instance, if your CTA is a hyperlink to shop (in group A) and the CTA is a red Shop! button (in group B), then you can determine which CTA drives more clicks and sales.
3. A/B Testing Best Practices
Ultimately the goal is for recipients to (i) open the email, (ii) click on a CTA (link or button), to (iii) make a purchase from your Farm store. You will want an enticing subject line, and images and buttons that compel a customer to Shop from your Farm. You will want to know what time of day to send, to increase opens and orders.
A few suggestions to get you started:
Messaging. Alter the types of message you’re sending - keep the shop beef button and everything the same but in one talk about the cow’s life and in another about the quality of the beef.
Add-Ons. Include a recipe - say for Apple Pie - with a button to buy apples. Alter only the image: one is a bushel of apples and the other is the ‘apple pie’.
Images v. Video: Switch out the header images on your newsletter --- try a short video or gif versus a still image. We see good sales conversion when Farmers include videos of their Farm or products.
Personalization. Consider making the email appear personal, and include the recipient's name and location by using merge tags.
Step-by-step to Create an A/B Test
Before you begin write out the goals of your A/B test and your hypothesis. Then, follow these steps to configure an A/B test with Mailchimp:
Login to Mailchimp
Go to Create Campaign, and then select Create an Email
Choose A/B Test at the top of the pop-up and name your campaign. Be sure to use a consistent naming format so it is easy for you to keep track of all your campaigns.
Select the list, group, or segment to receive your emails.
Decide on the variable you want to test and the number of combinations. Stick to 2 combinations to start with, especially if you have a smaller contact list.
Determine what percentage of recipients will receive your A/B test combos:
To start, try sending the split test options to 50% of your list:
25% version A
25% version B
50% to the winner of the test
Note: If you are testing sending times, use your whole list:
50% at time A, then 50% at time B
Choose how Mailchimp will determine the winner:
Set the metric that matched the single variable you are testing.
When testing a Subject Line, the relevant metric will be the open rate
When testing a variable from the Body of your email - images, buttons, fonts - , you determine the winner by Click Rates.
Conclusion
Mailchimp and Barn2Door integrated together is a game-changer for Farmers. Leveraging your Mailchimp account for email marketing, newsletter creation and A/B testing, combined with your Barn2Door contacts, customer groups, product inventory, and promos will help you refine and target your marketing activities. Ultimately, you’ll save time and increase sales for your Farm, while building customer loyalty and ensuring consistent sales.
Watch this 5:00 video to learn how Barn2Door can help your Farm build a thriving business.