Testing With a Goal in Mind

Tuesday, August 18, 2015
Digital Marketing
Testing With a Goal in Mind

Getting the most out of your digital marketing campaigns — that’s the goal of every online marketer. Of course, the best way to do that is to put together a plan to test and refine your digital advertising. But deciding what and how to test can seem like a big task. Following are some helpful guidelines.

Before you test, first identify the desired goal or end result of both your digital marketing and testing strategies. Identifying the goals of the digital marketing strategy will help you know which engagement tactics to use and then better outline the testing strategies and tactics that will help you reach your goals.

Below are six simple steps to define which testing strategies and tactics to use in optimizing your digital marketing campaign or program:

Step #1: Define goal(s) of digital marketing strategyStep #2: Define digital tactics/channels to help reach goal(s)Step #3: Define engagement approachStep #4: Build testing strategy to coincide with engagement approach — with goal(s) in mindStep #5: Set benchmark and goal KPIs to track ongoing resultsStep #6: Repeat


Step #1: Define goal(s) of digital marketing strategy. First, you need to define the goals (desired results) of your digital marketing strategy. Such goals typically intersect with the goals of the company/division, and may include:

·        Increasing the number of website visitors a certain percentage YOY·        Increasing online sales/revenue a certain percentage YOY·        Increasing the total number of subscribers a certain percentage YOY·        Driving a percentage increase in the number of subscribers to free/trial subscriptions to paid subscriptions, etc.

Step #2: Define digital tactics/channels to help reach goal(s). This step may seem like a no-brainer, but coming up with a perfect mix of tactics and digital channels (paid search, email, content marketing, etc.) that will best help you achieve your digital marketing goals and stay within budget is harder than it looks. Marketers sometimes think they can take advantage of all digital channels at their disposal, but you need to approach this step objectively. Creating detailed financial forecasts for each channel based on past or estimated performance and then sharing those with key internal stakeholders will help you make an objective decision on which channels will help you stay focused and achieve your goals.

Step #3: Define engagement approach. Engaging with customers and prospects through email, social, content and other digital means is done for a variety of reasons. These reasons include informational, promotional, continued engagement, lead generation, direct sales, relationship-based, etc. You may use several engagement approaches depending on your audience, the product you are trying to sell, the channel and the end goal. If the goal is to increase the amount of sales from an e-commerce site, a direct sales approach coupled with informational and other promotional tactics makes sense.

Step #4: Build testing strategy to coincide with engagement approach — with goal(s) in mind. Once you’ve defined your strategy, channels and engagement approach, you’re ready to begin developing a testing strategy. Testing strategies are going to be tied to individual digital channels. When developing testing strategies for each channel, you first need to define all the testing categories. If you were developing a testing strategy for email, your testing categories could include: subject line, send classification, design/format, content, cadence, etc.

Once you’ve defined your testing categories for each digital channel, you will then define the variables in each category you would like to test. Keep in mind that the test variables need to relate to your engagement approach as well as help you achieve your digital marketing goals.

Once the variables are defined, you can start developing a testing calendar specific to each individual marketing channel. For example, if you were developing a testing schedule for email, you may want to begin testing at the top of the email and work your way down to the bottom before you start testing outside variables like audience, landing pages and cadence. If you were taking this approach, you could do subject-line testing for two months, then test the email send classification for two months, and then test the design/format of your emails for two months, and so on.

You can start simple by testing one variable at a time in an A/B testing format, or you can test multiple variables at once. An A/B test for email subject lines could simply include testing a subject line with personalization (using the recipient’s first name in the subject line) and testing the same subject line without personalization.

You can also test multiple variables at once (multivariate testing). The goal of multivariate testing is to determine which combination of all possible variations performs the best. Multivariate testing can be helpful with landing pages when multiple elements on the same page can be modified concurrently to improve a single conversion goal: clicks, form submissions, shares, sign-ups, etc. If done properly, multivariate testing can reduce the need to run several A/B tests sequentially, thus reducing the amount of time it takes to select a winning combination.

The total number of variations in multivariate testing can be easily calculated with this formula:

[# of variations on component A] X [# of variations on component B] = [total # of variations]

For example, if you’d like to test two different images and two different headlines on a landing page (keeping everything else on the pages the same), you would test four different landing pages concurrently:

1.      Image #1 + Headline #12.      Image #2 + Headline #13.      Image #1 + Headline #24.      Image #2 + Headline #2

There are upsides and downsides for using both A/B and multivariate testing. Choosing between the two or a combination of these tactics depends on the digital channels you’re testing and the timeline in which you need to test.

Here’s a brief summary of Step #4, building a testing strategy:

1.      Develop testing strategy for each digital channel2.      Define testing categories for each channel3.      Define testing variables for each category (relate to engagement approach and goals)4.      Create testing calendar based on test categories and variables5.      Define appropriate test type for each channel (A/B, multivariate or a combination of the two)

Step #5: Set benchmark and goal KPIs to track ongoing results. Next you’ll want to set benchmark and goal KPIs to track the ongoing results of your tests, programs and campaigns. Benchmark KPIs are typically defined by your own programs over a length of time. Goal KPIs are set by the marketer based on benchmark KPIs and/or what they would like to accomplish over a set period of time. For example, an e-newsletter may have the following average KPIs from the prior year:

·        18.60% open rate·        4.22% CTR·        22.71% CTOR·        0.51% opt-out rate

These averages could be the benchmark KPIs for the e-newsletter program moving into the next year. For the e-newsletter goal KPIs, you’ll not only want to take a look at the previous time period’s averages, but a number of data points (i.e., KPI range) to help determine what the KPI goals should be for the upcoming time period. For instance, if you were to examine the trends, ranges, averages and so on of KPIs from the previous time period, and say that the KPIs were improving by roughly 18% per time period, you might want to set a goal for each KPI to improve by 20% in the next time period.

Step #6: Repeat. The final step in a testing strategy is to never be complacent with how your digital channels are performing. If you find something that works, keep testing it to make it better. Consumer preferences, technology, algorithms and user experiences are constantly changing. So if you test for only a small amount of time and stick with the winner indefinitely, that winning choice will eventually start experiencing diminishing results. Once you’ve completed your testing timeline and/or reached your program goals, create a new testing timeline and set new program goals.

Remember to keep testing to find new, better and creative ways to engage your audience, improve performance, reach your goals and to continue building strong customer relationships.

Scott Hauck

Sr. Director, Marketing Operations


January 2016 – Present (7 months)Greater Salt Lake City Area

ThomasARTS is a full service marketing agency with offices in Utah (HQ), California, Texas, Minnesota, Washington D.C. and New York. ThomasARTS has formed many trusted partnerships with Fortune 500 players in the healthcare, insurance and financial services industries.As the Sr. Director of Marketing Operations, I am responsible for the following:- Marketing Automation- Business Intelligence- Email Marketing & Strategy- Analytics & Reporting- Data Management, Mining & ETL Development- Lead Generation- Customer Contact Strategy- Forecasting, Predictive & Statistical Modeling- eCommerce/Web Optimization Testing & Personalization- User Experience Analysis- Print Production Management

Related Posts