Max Diff Question Type

• Check it out in an example survey!
• Add a survey with this question to your account!

Using the Max Diff question type (aka maximum difference scaling) respondents are shown a set of the possible attributes and are asked to indicate the best and worst attributes (or most and least important, most and least appealing, etc.).

When to use it

Max Diff is an alternative to standard rating scale results that might lead you to believe everything is important. By forcing respondents to make choices between options, Max Diff delivers results that show the relative importance of the items being rated.

Consider the above example in which a respondent evaluates four fruits: apples, oranges, bananas, and pears.

If the respondent says that apples are best and pears are worst, these two responses inform us on five of six possible paired comparisons:

• apples vs. oranges,
• apples vs. bananas,
• apples vs. pears,
• oranges vs. pears,
• bananas vs. pears

The only paired comparison that cannot be inferred is oranges vs. bananas. As you can see, this produces better data than a standard ranking question. And people are much better at judging items at extremes than in discriminating the importance of items in the middle.

The Max Diff question type has the added benefit of showing random sets of attributes so respondents are not evaluating all attributes at the same time, which is really hard for people to do. And, by displaying the same attributes several times, you gather even more robust data.

What is an attribute? - An attribute is the property of the object, product, brand, service or advertisement that you are comparing.

What is a set? - A set is a randomly selected group of attributes.

Setup

1. Click the Question link on the page where you would like to add your Max Diff question.
        Max Diff questions must be on a page by themselves on the Build tab as the respondent will need to use the next button to submit each set.
2. Select Max Diff from the Question Type dropdown and enter the question you wish to ask.
3. Under the MaxDiff Format section, select the Orientation you desire.
   

        Best Practice Tip: Orientation

        We recommend the Worst | Item | Best layout option. We have found that this best conveys the task at hand. Survey respondents can easily comprehend that you would like for them to compare the attributes and select one for the each column. In addition, worst/least to best/most is a standard layout that most respondents will expect (this is for left-to-right languages).
   

4. Next, you can also customize the Best and Worst Labels, as well as the Label for the Items respondents are evaluating.
   
5. Next, scroll down and add the attributes you would like your respondents to evaluate.
6. Finally, above your attributes, customize the Number of attributes per set and the Number of sets per respondent. By default, your question will be setup to show 4 attributes and 6 sets. You will see the below warning as a result:
   
   Warning! With this combination of attributes per set and sets per respondent, not every attribute will be shown to each survey taker.
   
   We recommend setting up your question to at least show each attribute 3 to 5 times for each respondent. To do so, use one of the below equations; plug in the number of attributes (K) and number of attributes per set (k). 

• 3 times per respondent: 3K/k = number of set to show
• 4 times per respondent: 4K/k = number of set to show
• 5 times per respondent: 5K/k = number of set to show

Survey Taking

On a desktop and most laptops, the Max Diff question type looks like the below. Remember, your Max Diff question should be the only question on the page!

When optimized for mobile devices the Max Diff question type will convert your question to a mobile friendly format as pictured below.

By default, survey questions show one at a time on mobile devices to prevent the need for scrolling on smaller screens. You can turn off this one-at-a-time interaction if you wish.

Reporting

Standard Report

We have improved the reporting of the Max Diff question in the Standard Report! It's much easier to understand and allows for more visibility into how items were ranked. For each attribute you will see the percentage of times it was ranked as most appealing, least appealing, or not chosen.

The attributes will be ranked based on the score which is computed using the below formula:

# times attribute was selected as best - # times attribute was selected as worst
____________________________________________________________________________
                                        # times the item appeared

From the score we can determine a couple of things:

• The higher the score, the more the feature is appealing to respondents.
• A positive score means that that attribute was selected as MOST appealing more often than least appealing.
• A negative score means that that attribute was chosen as LEAST appealing more often than most appealing.
• A score of zero means that that attribute was chosen as MOST and LEAST appealing an equal number of times OR it has never been chosen as most and least appealing.
• If a score of an item is two times bigger than another item, it can be interpreted that it is twice as appealing.

 See additional compatible chart types

Within the Standard Report there are various chart types available for visualizing your data. The below grid shows which of the chart types Max Diff questions are compatible with. 

Chart Type (Aggregate)		Compatible
Pie Chart
Arc Chart
Donut Chart
Vertical Bar Chart
Stacked Vertical Bar Chart
Horizontal Bar Chart
Stacked Horizontal Bar Chart
Spiderweb Chart
Histogram
Line Chart
Spline Chart
Area Chart
Area Spline Chart
Scatter Plot
Word Cloud
Collapsed Response List
Grid
Diverging Stacked Bar Chart
NPS Chart
Contact Cards
DIY Grid/Cards
Table

Chart Type (Longitudinal)		Compatible
Longitudinal Line Chart
Longitudinal Area Chart
Longitudinal Spline Chart
Longitudinal Area Spline Chart
Longitudinal Vertical Bar Chart
Longitudinal Stacked Vertical Bar Chart
Longitudinal Stacked Area Chart
Longitudinal Stacked Spline Area Chart

 See what other report types are compatible

Exporting

The Max Diff question has its own export which will be available under Results > Exports. Once there, select the Max Diff Raw Data option.

The Max Diff question does not export via the SPSS Export.

In the export, there will be a row per combination displayed to the respondent. There will be a column per attribute displayed in the combination and a Best and Worst column. The reporting values of your Max Diff question will populate the rows. 

Looking below at Response ID 1, Combination 1 you can see that the following attributes were displayed to the respondent as their first combination: bananas, oranges, apples, kiwi. Apples were selected as best and oranges were selected as worst. 

FAQ

 Will each respondent see each attribute?

 How is the experiment set up?

 Will sets be shown equally?

 How many responses do I need? How many sets should I show? How many attributes per set should I show?

 Can I hide the x per y sets that displays below my Max Diff question?

.sg-maxdiff-progress {
  display:none;
}