Overview

Testing a Campaign is a critical part of a successful marketing strategy. This is especially important with Campaigns and Templates enhanced with dynamic content. It is a best practice to send a successful test before each Campaign Launch.

Methods of Testing

While testing Templates can be done within the Template editor, the same tools and more are accessible through the Campaign Overview screen. When viewing a Campaign, testing tools are found in the sidebar of the screen beneath the Scheduling options.

 

Manual Testing

There are four types of testing within a Campaign.

Personalization

The first type of testing is the same testing tool found for Templates. Testing Personalization combines Audience Data and the Campaign Template to ensure that all personalized data is rendered accurately.

 

 

Use this option to ensure any FreeMarker is rendered correctly. There is one major difference between testing Personalization within the Campaign screen and testing a Template within the Template Overview screen. When testing Personalization in the Template Editor, the User interface allows different recipients to be selected. When testing within the Campaign, recipients cannot be switched.

 

Spam Filter

The Spam Filter option within Testing uses the SpamAssassin service to determine the likelihood that a Template is detected as spam. SpamAssassin uses a complex set of rules that analyzes message content that are typically found in spam messages. The more of these elements found in an email contact, the higher the likelihood that the email is detected as spam. A higher SpamAssassin score, the higher the likelihood of being detected as spam.

 

 

Any reasons that an email may be detected as spam is displayed within the Spam Filter screen.

SpamAssassin is a service provided by Apache. Read the Official Documentation here.

Recipient Data

The Recipient Data testing is a method of testing the Campaign Audience. Use this tool to confirm that the Audience is returning recipient information correctly.

 

 

 

Additionally, use this screen to send test contacts. Using the Send Test button, a copy of the email that would be sent to a specific recipient may be sent to another email address for testing purposes.

 

Add more emails if necessary.

Multivariate (A/B) Testing

Multivariate Testing is a method of comparing different subjects or Templates, evaluating the results, and then Launching a Campaign using the winning item. While Multivariate Testing is found within the Testing section of the Campaign Overview screen, it is different from other testing options. Multivariate Testing functions in place of a Campaign Launch. When using a Multivariate Testing session, live contacts are sent to recipients from the Audience.

Select the Multivariate (A/B) option from the Campaign Overview screen. This opens the Multivariate testing screen. If any testing has been used on this Campaign before, the testing configuration is saved and can be reused.

 

 

Click on the Saved Test Id to open an existing test. To create a new test, click Start New Test in the top right of the screen.

 

 

A Multivariate test can operate with either Subject Lines or entire Templates. Use the radio buttons to select which type of test to use.

Multiple items can be created for Multivariate testing. After creating and saving the different items, clicking Save & Continue navigates the User to the next screen.

 

 

This screen provides options for testing the multiple subject lines or Templates, depending on the previous screen.

The first two radio buttons allow customization for dividing records for the multivariate testing. When contacting records and recording results, the contacts can be split evenly across variants or, when using the By Variant option, different values can be selected for each variant.

When using multivariate testing, the entire Audience does not need to be contacted. Instead, use the Distribute slider to select a percentage of the Audience to contact.

The count of records displayed on this screen is based on the last count of records returned by the Audience. If the true number of records in the Audience has changed since the last count, that is not reflected in this screen. Click the refresh button to update the Audience count.

Click Save & Continue to move on to the next screen.

 

 

 

The next screen prompts the User to trigger the multivariate test immediately or to schedule the test to run. Launching the test will send contacts to the percentage of Audience records chosen from the previous screen.

The Auto Launch Winner checkbox monitors the results of the multivariate test. The winner is the subject line or Template that results in more opens at the completion of the test. If the Auto Launch Winner checkbox is enabled, the Campaign is automatically launched following the test and uses the winning subject line or Template.

If the Auto Launch Winner checkbox is enabled, a new option displays.

 

 

Conditions must be set to tell the Multivariate test how to determine which item wins the test. The User chooses between evaluating the Click Rate or the Open Rate to compare items. The User must also include the number of hours to monitor results before determining the winner.

Click Save & Continue to continue to the final screen to confirm the configuration of the Multivariate test.

 

 

Confirm the configuration of the Multivariate test and click Launch.

This launches the Multivariate test and sends contacts to a percentage of the audience.

After the completion of a Multivariate test, the winning item is not automatically configured in the Template for future use. The winning item is only used for the following Multivariate Auto Launch. If the winning subject line or Template should be a permanent part of future Campaign contacts, the values must be manually updated by the User.