A/B Testing in Mobile

Brief Overview

WalkMe Mobile AB testing allows you to manage any new campaigns or changes in a way that will ensure your users are getting the best experience. Having the ability to experiment with multiple versions of a campaign simultaneously provides more value to UI/UX designers and marketing teams, while also maintaining a seamless experience for your end-users. 

How It Works

In the Mobile Console, users manage AB Tests by creating groups, adding campaigns, and assigning percentages.

In App Settings, you will see a new tab called A/B Testing. Here you can view a list of all the A/B Test groups created, including:

• Indicators for the number of campaigns in use
• The last modified date; and
• Whether the A/B Test is active or inactive

Limitations: Supported from SDK 2.14.0 for both iOS and Android.

Use Cases

Common use cases include:

• Multiple versions of the same campaign (different images, text, etc)
• Single campaign to % of user base
• Control group (sees no campaign)

Creating an A/B Test

From the App Settings (Settings>App Settings):

1. 1. Go to A/B Tests tab.
   2. Click New A/B Test to display the test builder. 
      • An entry for the control group is predefined with 0%. 
   3. Type the Group Name.
   4. Click Add Campaign and select the desired campaign from the dropdown.
      • Repeat this step for up to 9 entries.
   5. Assign percentages as desired.
      • Sum of percentages must equal 100.
   6. Mark the test as Active.
   7. Click Save.
   8. Publish the campaigns.

Reviewing Results

Once A/B Tests have run their course, you can review the campaign engagement and compare the impressions/clicks between the campaigns in each test.

Download the “A/B Testing Campaign Engagement” report.