An A/B testing campaign is a Quanda email campaign type that sends two versions of one email to two small groups of recipients, evaluates which version works better and sends the winning version to everyone else on its own. You create it in the top Emails menu with Create a new Campaign and the A / B Testing card. It helps you fine-tune subject lines and content: instead of guessing, you find out what really works for your recipients.

How an A/B test works

  1. You prepare version A and version B that differ in one thing, for example the subject line.
  2. On the Sending Settings page you use a slider to set how many recipients get version A and version B and how many wait for the winner.
  3. You choose how the winner is picked (opens or clicks) and the evaluation time from 1 hour to 7 days.
  4. When the time is up, Quanda automatically sends the winning version to the remaining recipients.

Example: out of 1,000 recipients, 100 get version A, 100 get version B and the remaining 800 get the winning version.

How to create an A/B testing campaign

  1. In the top Emails menu, click Create a new Campaign.
  2. Type a name you will recognize in the campaign overview into the name field.
  3. On the A / B Testing card, click Create.
  4. On the Settings page, check the fields described below.
  5. Click Save → Recipients and continue by choosing the recipients.
The new campaign window in Quanda with six campaign types: Simple, A / B Testing, Email Series, Birthday Wish, Anniversary Wish and Event-triggered, each with a Create button.
Choosing the campaign type, A / B Testing is in the top middle.

What to set on the Settings page

  • Name: the name of the campaign, recipients do not see it.
  • Status: shows which phase the campaign is in. Before sending it has to be Ready to use.
  • Language: the language of the supporting texts of the email, that is the unsubscribe text, the unsubscribe page and the link for viewing the email in a browser. If you send an email to German speaking recipients, choose German and all these texts will be in German.
  • Content category: if you have categories set up in the account settings, recipients can choose on the unsubscribe page which content they want to receive from you.
  • Project Manager: the user who creates and manages the campaign.
  • Co-workers: users who have permission to work on the campaign and see it in their overview.
The Settings page of an A/B testing campaign in Quanda with the fields Name, Status, Language, Content category, Project Manager and Co-workers and the buttons Save → Recipients, Save and Cancel.
The Settings page of an A/B testing campaign.

Who sees the campaign depends on the user role. Admin and Manager see all campaigns. Project manager and Seller only see campaigns they created or in which they are among the co-workers. A Seller therefore has to add themselves to the Co-workers field. Otherwise they lose access to the campaign after saving the Settings page and cannot continue preparing it.

The data on the Settings page can no longer be changed once the A/B testing campaign has been sent.

How to finish the campaign

  1. Recipients: you choose at least 11 contacts.
  2. Email Editor: you create version A and then version B with the Save and create version B button.
  3. Sending Settings: you set the size of the test groups, the winner criterion and the evaluation time and send the campaign.
  4. Statistics: you compare the results of both versions.

A/B testing is included in the Professional and Enterprise plans. Ideas for subject lines worth testing are in the article How to write an email subject line.

Do you have questions? We will be happy to answer them. Just contact our customer support at +420 605 163 892, e-mail: support@onquanda.com.

Frequently asked questions

It is a Quanda campaign type in which you send two versions of one email, A and B, to two small groups of recipients. After the time you set, Quanda evaluates which version performed better and automatically sends the winning version to all remaining recipients.

The subject line. It decides whether people open the email and the test is the quickest to prepare, because version B is a copy of version A in which you only change the subject and possibly the preheader. If you want more clicks, keep the subject and change the buttons and links. Change only one thing per test.

The technical minimum is 11 recipients. With a few dozen contacts, though, the difference between the versions tells you little, because one extra open moves the percentages by whole points. Test groups of hundreds of recipients give a more reliable result.

You set the evaluation time from 1 hour to 7 days. In our experience, 1 day gives the most reliable results. A shorter time only shows the fastest readers, which tends to be misleading.

A/B testing is included in the Professional and Enterprise plans. It is not part of the Standard plan. The pricing page calculates the exact price for your number of contacts.

No. The data on the Settings page can no longer be changed once the A/B testing campaign has been sent. Check the language, content category and co-workers in advance.