Branching: A/B Split Testing in Surveys

Percent Branch in SurveyGizmo is a powerful tool that allows you to perform A/B Split Testing. Using the Percent Branch action, you can assign a percentage of respondents to see a specific set of questions that differ from another branch's set of questions.

Let's discuss a simple example with 2 branches. The survey contains 2 different image elements with your company's 2 logo finalists. The branching is set up to give each branch, and thus each logo, 50%: Group A would see logo 1 while Group B would only see logo 2; a coin flip chance.

Setup

Percent Branch must be on a prior page to the questions, or, in this case images, that will be branched. If you wish to hide questions on page 3, the Percent Branch action should be added to page 1 or 2.

1. On your survey's Build tab, click Add Action on the page you wish to add the Percent Branch.

2. Select Percent Branch from the list.

3. Your Percent Branch action will have a default Branch A (100%). You can set up up to 10 branches. At the bottom of the list of branches is the total percentage allocated across all the branches. Click Add to create a new branch.

4. Click on the branch name to edit that branch:

(a) Give your branch a name.

(b) Assign a percentage for that branch.

(c) Set up the Hide/Show and Require settings of the questions in the survey for this branch. Using branching you can assign the visibility of any element within a survey that follows the Percent Branch action in the survey:

Hide: Checking this box will make the element hidden from that group.

Show: Checking this box will display hidden by default elements to that group.

Require: Checking this box will set the question to required for this particular branch. If you are using require settings in branches be sure to leave questions unrequired in the question editor. The require option is available for questions only.

Hide and Show are not compatible, and if both are checked, Hide will take precedent. Typically, when setting up a Percent Branch, since all questions added to the survey are visible by default, you'll want to check the Hide column for any questions that should not be seen by that Branch. For instance, if there were four branches, then you would want check Hide for the three survey elements that would not be seen in that branch.

Remember, survey elements means more than simply questions in the survey, this also includes actions.

Reporting on Your Branches

Within each individual response the branch that was assigned will display.

Results can also be filtered or segmented on the branch in reports. 

To filter by branch in Standard Reports click Add Filter > Create Advanced Filter and select your branch in the first dropdown menu. In the second menu, enter the exact text for your branch. This will filter your report to only show responses from the specified branch.

If you prefer to segment your report for all your branches click Add Segment > Create Advanced Segment. Give your segment a name, select a color and click Edit Logic. Select your branch in the first dropdown menu. In the second menu, enter the exact text for your branch. Repeat these steps to create a segment per branch. This will allow you to show all responses segmented by branch in your report.

Exporting Your Branch Data

For each response in CSV/Exports the Branch that was assigned will export by default.

Compatibility

• The Percent Branch action is not compatible with Piping.
• Please use extra care when using the percent branch action with logic in your surveys. Logic rules will override the percent branch show/hide settings. If you choose to use the percent branch in combination with logic we recommend testing thoroughly to ensure the survey is behaving as you intended.
• If you are using the Percent Branch Action in conjunction with Page Randomization, note that randomization is determined as soon as the response session is started. As such the page order will be determined prior to the Percent Branch Action being initiated. You should always place the Percent Branch Action on a page prior to any randomized pages.
• If one is utilizing Hidden value actions in the Percent Branch Action projects, the Percent Branch Action does not hide or show Hidden Value Actions that are submitted automatically via auto submitting pages.

FAQ

What if the total of all branches does not equal 100%?

If you were to setup a 3 branch split of 33% per split, that means that 1% of the time they would not be matched to a particular split. In this case, they are assigned to the A branch. Thus, if you setup 3 branches each with a 10% chance (30% total), that would mean that branch B and C would have a 10% chance, but branch A would have an 80% chance (10% + 70% unassigned).

How does the system assign respondents to a particular branch?

The percent branch is using the php rand function. It randomly chooses a number between 1 and 100 and then assigns that number to your various branches based on the percentage you've set up. For example, this means that if the random number that comes up is 27, then if you had a 3 branch split with (Branch 1: 33%, Branch 2: 33% and Branch 3: 34%) it would assign that respondent to Branch 1. Same thing if the number came out as 89, it would show up in the third branch.

What if the branch hides all elements present on a particular page?

That page will be automatically skipped. 

I want my Percent Branch Action on a page by itself but I don't want my respondents to see or have to submit this page, what do I do?

You can put a Percent Branch action on a page by itself and set it to hide. This will auto-submit the page while still assigning the respondent to one of the branches.

Does Percent Branch force respondents into each branch to ensure that over 100 responses, a 50%/50% branch split would have exactly 50 responses in each branch?

No, because SurveyGizmo offers the ability to collect multiple responses at anytime, this could lead to skewing of the results. It is random each time, just like a coin flip: you may have 56 heads and 44 tails over 100 flips. The percentages would increase in accuracy over a larger sample of responses (1000 responses would be a closer distribution than 10 responses).

How do you use the Show attribute?

The survey element must be set as Hide this question by default on the question's Logic tab to be affected by the Show attribute in Percent Branch.

Can I use the Show attribute to show a question that is affected by Hide all subsequent questions on this page or has its own Logic rule built?

No, the Show attribute is only effective on questions set as Hide this question by default.

What does the merge code display for a Percent Branch action?

When using a merge code to display which branch is assigned to a respondent, the merge code [question("option value"), id="X"] where X is the id of the Percent Branch action, will display the title of the branch (#2 in the labeled image at the top of the tutorial).