Showing results for 
Search instead for 
Do you mean 

Configuring mPulse to measure A/B (Bucket) Tests

by Community Manager on ‎08-03-2015 01:45 PM - edited on ‎07-25-2016 11:40 AM by tpoggi

A/B testing is a great way to compare two (or more) versions of your site to see if one is more popular with users than another. Perhaps you'd like to test out new CDN providers, a different page layout, or new performance tricks. mPulse's A/B test integration allows you to compare your tests for performance. We allow you to tag the mPulse code with the name of the currently running test so we can report on each test separately.

There are two steps to using A/B Tests:

    1. Define a Test Name in your page.
    2. Define a JavaScript variable on your page that contains the name of a page group. This variable name may be namespaced.

For example, use SOASTA.ab_test or ab_test.

SOASTA = {}; // don't use var to ensure it's global
SOASTA.ab_test = "test name"

  1. Add the test variable name to the mPulse Domain by entering it in the A/B (bucket) tests variable name field, which is located on the General tab in the Configuration screen of your app.
  2. Click OK after making any change.



Test Name Restrictions

There are two restrictions on the page group name:

  • Test names may only contain alphanumeric characters, dashes, underscores and spaces.
  • Test names must be at most 25 characters long.

mPulse uses the regular expression below to validate an A/B Test Name, and you may use the same to verify your code:

/^[a-zA-Z0-9_ -]{1,25}$/

Note: Measurements that fall into a test bucket will not affect your main site measurement. This allows you to use your main site (ie, with no ab_test variable defined) as the control, or baseline measurement, and define separate tests under the main site to test out new features. This also means that the overall beacon count on your main site will be lower than first expected. There is only 1 A/B variable available per App, but you can assign it many values; however, we recommend keeping the different values assigned as small a number as possible for easier analysis.