Campaign Testing

Overview

Testing a Campaign is a critical part of a successful marketing strategy. This is especially important with Campaigns and Templates enhanced with dynamic content. It is a best practice to send a successful test before each Campaign Launch.

Table of Contents

Methods of Testing

Manual Testing

Personalization

Spam Filter

Recipient Data

Multivariate (A/B) Testing

Things to Remember when Campaign Testing

Methods of Testing

While testing Templates can be done within the Template editor, the same tools and more are accessible through the Campaign Overview screen. When viewing a Campaign, testing tools are found in the sidebar of the screen beneath the Scheduling options.

Manual Testing

There are four types of testing within a Campaign, Personalization, Spam Filter, Recipient Data, and Multivariant Testing. The following sections should provide you with some of the details and benefits of running these tests.

Personalization

The first type of testing is the same testing tool found for Templates. Testing Personalization combines Audience Data and the Campaign Template to ensure that all personalized data is rendered accurately.

Use this option to ensure any FreeMarker is rendered correctly. There is one major difference between testing Personalization within the Campaign screen and testing a Template within the Template Overview screen. When testing Personalization in the Template Editor, the User interface allows different recipients to be selected. When testing within the Campaign, recipients cannot be switched.

Spam Filter

The Spam Filter option within Testing uses the SpamAssassin service to determine the likelihood that a Template is detected as spam. SpamAssassin uses a complex set of rules that analyzes message content that are typically found in spam messages. The more of these elements found in an email contact, the higher the likelihood that the email is detected as spam. A higher SpamAssassin score, the higher the likelihood of being detected as spam.

Any reasons that an email may be detected as spam will be displayed within the Spam Filter screen.

Recipient Data

The Recipient Data testing is a method of testing the Campaign Audience. Use this tool to confirm that the Audience is returning recipient information correctly.

Additionally, use this screen to send test contacts. Using the Send Test button, a copy of the email that would be sent to a specific recipient may be sent to another email address for testing purposes. Additional emails can be added to this Test Message if necessary.

Multivariate (A/B) Testing

Multivariate Testing is a method of comparing different subjects or Templates, evaluating the results, and then Launching a Campaign using the winning item. While Multivariate Testing is found within the Testing section of the Campaign Overview screen, it is different from other testing options. Multivariate Testing functions in place of a Campaign Launch. When using a Multivariate Testing session, live contacts are sent to recipients from the Audience.

Select the Multivariate (A/B) option from the Campaign Overview screen. This opens the Multivariate testing screen. If any testing has been used on this Campaign before, the testing configuration is saved and can be reused.

Click on the Saved Test Id to open an existing test. To create a new test, click Start New Test in the top right of the screen.

A Multivariate test can operate with either Subject Lines or entire Templates. Use the radio buttons under the test box to select which type of test to use.

Multiple items can be created for Multivariate testing. After creating and saving the different items, clicking Save & Continue navigates the User to the next screen.

This screen provides options for testing the multiple subject lines or Templates, depending on the previous screen.

The first two radio buttons allow customization for dividing records for the multivariate testing. When contacting records and recording results, the contacts can be split evenly across variants or, when using the By Variant option, different values can be selected for each variant.

When using multivariate testing, the entire Audience does not need to be contacted. Instead, use the Distribute slider to select a percentage of the Audience to contact.

The count of records displayed on this screen is based on the last count of records returned by the Audience. If the true number of records in the Audience has changed since the last count, that is not reflected in this screen. Click the refresh button to update the Audience count.

Click Save & Continue to move on to the next screen.

The next screen prompts the User to trigger the multivariate test immediately or to schedule the test to run. Launching the test will send contacts to the percentage of Audience records chosen from the previous screen.

The Auto Launch Winner checkbox monitors the results of the multivariate test. The winner is the subject line or Template that results in more opens at the completion of the test. If the Auto Launch Winner checkbox is enabled, the Campaign is automatically launched following the test and uses the winning subject line or Template.

If the Auto Launch Winner checkbox is enabled, a new option displays.

Conditions must be set to tell the Multivariate test how to determine which item wins the test. The User chooses between evaluating the Click Rate or the Open Rate to compare items. The User must also include the number of hours to monitor results before determining the winner.

Click Save & Continue to continue to the final screen to confirm the configuration of the Multivariate test.

Confirm the configuration of the Multivariate test and click Launch.

This launches the Multivariate test and sends contacts to a percentage of the audience.

After the completion of a Multivariate test, the winning item is not automatically configured in the Template for future use. The winning item is only used for the following Multivariate Auto Launch. If the winning subject line or Template should be a permanent part of future Campaign contacts, the values must be manually updated by the User.

Things to Remember when Campaign Testing

It's important to thoroughly test a campaign before scheduling or launching, to be sure it meets your standards. A few elements to keep in mind while testing would be:

• Content accuracy - Does this convey the information I want it do?
• Correct URLs in links - Do my links all work and lead where I want them to?
• Personalization - Will this email be relevant and personalized for my recipient?
• Spam Filtering - Does my content contain any red flags to avoid or address?

Accelerator provides built-in options for easily sending test messages to desired recipients, as well as features to provide insight into the end result.

Testing tools available for Marketing Campaigns:

• Personalization Testing - preview rendering of personalization in the template
• Spam Filter - see anticipated spam score via SpamAssassin
• Recipient Data - preview message for specific recipient data
  • From the Recipient Data page it’s also possible to send a test email emulating the live message which will be received by a specific recipient. This allows you to see an email in your inbox which has the personalization of a specific recipient you indicate. The Send Test button appears to the left of each Recipient on this page.

Testing tools available for Transactional Campaigns:

• Personalization Testing - preview rendering of personalization in the template
• Spam Filter - see anticipated spam score via SpamAssassin