Audience A/B testing from a journey

Audience A/B Testing allows you to split an audience into two or more variations and create connections for each variation independently, to help you to compare the performance of different targeting approaches. For example, if you have an audience of low engagement users that you want to reengage with your app, you might devise a test like this:

• Send 40% of the audience to Messaging Platform A
• Send 40% of the audience to Messaging Platform B
• Keep a control group of 20% who are not targeted with any messaging

You can then compare the engagement outcomes for each group and apply the most successful strategy to the entire audience.

Create a journey

1. Following the instructions in Create a journey, begin creating a journey.
2. Once you’ve added a milestone with criteria, click the plus sign beneath the milestone’s audience and select A/B Test (Random Split).

3. Click + Add A/B Test Variation to add a new variation to your A/B test. Enter the percentage of users you want each variation to contain. If the sum of your variations’ percentages is less than 100, the remainder is automatically assigned to the Control group. For example, if you create Variation A and Variation B with 40% in each, you will have a Control group with 20%. You can also provide a custom name for each variation.
4. Create as many variations as you need for your test, up to a maximum of 5. The total of all your variations must always add up to 100%. If you try to assign a percentage to a variation that would cause the total to exceed 100, you will see an error message.

5. When you are satisfied with your variations, click Save.

After saving your variations, you will see each variation in a new branch within the Journey Builder.

Create a connection

After defining your variations, you can connect each one, including the control group, to any output by clicking + Connect Output. There is also an option to connect the full audience to an output by clicking + Connect Output on the original audience card prior to the A/B split.

In the Audiences summary screen, audiences with an active A/B test will be marked with a % symbol.

Note that whenever the Audience Name is used in forwarding the audience to downstream partners, variant audiences will be named using the format [Audience External Name] - [Variant Name].

Change an audience definition

You can edit an audience definition without affecting the audience split, even after connecting to an output. When the audience is updated, the variants will still be balanced as defined when you created the test.

You cannot modify the percentages of an A/B test after creating the test.