Decipher MaxDiff Survey Programming Guide

MaxDiff (best-worst scaling) is one of the more technically demanding question types in Decipher survey programming. The platform has no native Decipher MaxDiff widget — programmers have to build the behavior from scratch using XML structure, Python <exec> blocks, and some careful attribute choices. This post walks through a complete Decipher MaxDiff implementation using the Indices Method — a pattern that produces balanced item assignment, per-set row filtering, and clean data output ready for hierarchical Bayes (HB) analysis.

By the end you will have working XML for an 8-item, 6-set MaxDiff with balanced item rotation, per-set row filtering, and correct variable naming for downstream analysis.

The Indices Method

Because Decipher has no native MaxDiff question type, programmers have to assemble the behavior from primitives: a loop, a radio question, and Python exec blocks. The Indices Method is the standard pattern for doing this — the name refers to how item selections are recorded by their index in the full item list rather than their position within a given set.

In the Indices approach, each respondent is assigned a set of item groups before the survey begins. As they work through the MaxDiff tasks, the survey presents one group at a time, and their best and worst selections are recorded against the item's global index. This makes the data immediately usable for hierarchical Bayes modeling without any post-processing to resolve positional ambiguity.

Decipher MaxDiff implementation overview

The core problem is this: a MaxDiff survey needs to show each respondent a different subset of items on each task, in a pattern that is balanced across the full pool. Decipher has no native MaxDiff widget, so the balanced subset assignment, the per-task filtering, and the response recording all have to be built from scratch.

The approach here solves it in three phases.

Generating the assignment at survey init. When the respondent first enters the survey, a Python <exec> block runs a balanced sampling algorithm over the full item pool. It produces an array of sets — one per task — where each set is a list of item indices from the pool. For example, with 8 items and 6 tasks showing 4 items each, the output might be [[1,4,6,8],[2,3,5,7],...]. Item identity is tracked by index throughout, which is what gives the Indices Method its name and what makes the data clean for HB analysis.

Storing the assignment in a hidden variable. The array is serialized and written into a hidden Decipher variable before any question is shown. This serves two purposes: it persists the assignment across the entire respondent session (Decipher re-evaluates page context on each navigation), and it appears in the data export alongside the responses, so analysts can reconstruct exactly which items each respondent saw in each task without a separate lookup file.

Looping over tasks and filtering rows on each iteration. A <loop> iterates once per task. At the start of each iteration, an <exec> block reads the current task's index list out of the hidden variable and disables any item row whose index is not in that list. All item rows are always declared in the XML — the exec is what makes only pickN of them visible per iteration. The radio question inside the loop records the best and worst selections against the item's global index, not its position in the set.

Design parameters determine how the algorithm behaves. Before writing any code, fix these four values:

A balanced design distributes item appearances as evenly as possible. With 8 items, 4 per task, and 6 tasks, each item appears in exactly 3 tasks — the minimum exposure typically needed for stable individual-level HB estimates. Keep pickN at or below roughly 40% of your pool; higher than that and respondents see too much overlap between tasks, which weakens the discrimination the model depends on.

Part 1: Helper Functions in <exec when="init">

The <exec when="init"> block runs once when the survey initializes. This is where you define the Python functions that handle assignment and row filtering. Paste it near the top of your survey, before any questions.

def q_json(value):
    return repr(value)
 
def q_parse_json(raw):
    if raw is None or raw == "":
        return []
    try:
        return eval(raw, {"__builtins__": None}, {})
    except Exception:
        return []
 
def q_assignment_slot(raw, slot):
    assigned = q_parse_json(raw)
    if slot < 0 or slot >= len(assigned):
        return []
    return assigned[slot]
 
def q_shuffle(items):
    out = list(items)
    shuffle(out)
    return out
 
def q_assign_balanced(pool, pick_n, rounds):
    values = [item.get("value", item) for item in pool]
    deck = []
    assigned = []
    for round_index in range(rounds):
        if len(deck) < pick_n:
            remaining = list(deck)
            refill = list(values)
            shuffle(refill)
            safe = [x for x in refill if x not in remaining]
            deck.extend(safe + [x for x in refill if x in remaining])
        assigned.append(deck[:pick_n])
        deck = deck[pick_n:]
    return assigned
 
def q_sample_assign(pool, pick_n, rounds, strategy):
    if strategy in ("balanced", "sparse", "rotational"):
        return q_assign_balanced(pool, pick_n, rounds)
    return [q_shuffle(pool)[:pick_n] for round_index in range(rounds)]
 
def q_setup_rows_by_labels(question, labels):
    wanted = [str(label) for label in labels]
    row_index = dict((str(r.o.label), r.index) for r in question.rows)
    for r in question.rows:
        r.disabled = str(r.o.label) not in wanted
    question.rows.order = [row_index[label] for label in wanted if label in row_index]
 
def q_setup_rows_by_value_map(question, values, value_to_row):
    labels = []
    for value in values:
        key = str(value)
        if key in value_to_row:
            labels.append(value_to_row[key])
    q_setup_rows_by_labels(question, labels)

How the balanced assignment algorithm works

q_assign_balanced works like a card deck. It shuffles all item values and deals pick_n cards per round. When the deck runs low, it refills from a fresh shuffle — but it avoids repeating items still in the remaining deck from the previous pass (the safe/remaining split). The result is that across 6 sets, each item appears roughly the same number of times, and no two items always co-occur. That even co-occurrence is essential for MaxDiff analysis.

q_setup_rows_by_value_map handles the per-set filtering. It maps the current set's assigned item values to row labels, then calls q_setup_rows_by_labels, which sets r.disabled = True on every out-of-set row and reorders question.rows.order to match the assigned sequence.

Part 2: Hidden Variable and Set Generation

Right after your helper functions, declare the hidden variable and the exec that populates it.

<!-- Hidden variable stores the per-respondent item assignment -->
<text label="claimSets" optional="1" where="execute">
  <title>HIDDEN: claimSets</title>
</text>
 
<!-- Generate balanced sets once when the respondent enters the survey -->
<exec>
pool = [
  {"value": 1, "weight": 1},  # fewer_meetings
  {"value": 2, "weight": 1},  # integrations
  {"value": 3, "weight": 1},  # easy_setup
  {"value": 4, "weight": 1},  # realtime_sync
  {"value": 5, "weight": 1},  # social_proof
  {"value": 6, "weight": 1},  # flexible
  {"value": 7, "weight": 1},  # security
  {"value": 8, "weight": 1},  # time_savings
]
claimSets.val = q_json(q_sample_assign(pool, 4, 6, "balanced"))
</exec>
 
<suspend/>

where="execute" keeps the variable out of the rendered survey but saves it to the data file. Do not skip this variable. Without claimSets in your data, you will not know which items were shown in which set for any given respondent, making post-fieldwork MaxDiff analysis impossible.

The <suspend/> after the exec commits the assignment before the respondent advances into the loop.

Part 3: The MaxDiff Loop

The loop is where the survey executes the assignment computed in Part 2. It iterates once per task, presents the radio question with only the assigned items visible, and records best and worst selections.

Loop structure and iteration

The outer <loop> element declares a single loop variable — task — that steps through 1 to rounds. Setting randomizeChildren="0" is required; Decipher's randomize option would shuffle set order, which produces uneven item exposure when combined with the deck-based balanced assignment.

Inside the loop, the <exec> calls q_assignment_slot with the serialized claimSets value and the current task index (converted from 1-based to 0-based with - 1) to retrieve the item list for this task. It passes that to q_setup_rows_by_value_map, which disables every row not in the set and reorders the remaining rows to match the assigned sequence.

The <looprow> elements at the bottom define the iteration — one per task, each setting [loopvar: task] to the task number. That value is referenced in the radio label and in the exec to look up the correct assignment slot.

<loop label="Q1_md_loop" randomizeChildren="0" vars="task">
  <title>Q1 - MaxDiff Loop</title>
 
  <block label="Q1_md_block" randomize="1">
    <radio label="Q1_[loopvar: task]"
           adim="cols"
           grouping="cols"
           shuffle="rows"
           ss:questionClassNames="Q1_maxdiff"
           unique="1">
 
      <title>Which statement would most or least convince you to try a new project management tool?</title>
 
      <exec>
Q1_values = q_assignment_slot(claimSets.val, int([loopvar: task]) - 1)
q_setup_rows_by_value_map(
    Q1_[loopvar: task],
    Q1_values,
    {"1": "fewer_meetings", "2": "integrations",  "3": "easy_setup",  "4": "realtime_sync",
     "5": "social_proof",  "6": "flexible",        "7": "security",    "8": "time_savings"}
)
      </exec>
 
      <col label="best">Most Convincing</col>
      <col label="worst">Least Convincing</col>
 
      <!-- All 8 items declared; the exec above disables non-assigned ones -->
      <row label="fewer_meetings">Cuts time spent in status meetings by automating progress updates</row>
      <row label="integrations">Works seamlessly with the tools your team already uses</row>
      <row label="easy_setup">Easy to set up — most teams are fully onboarded in under a day</row>
      <row label="realtime_sync">Updates sync instantly across devices and time zones</row>
      <row label="social_proof">Trusted by more than 10,000 teams across 80 countries</row>
      <row label="flexible">Flexible enough to match how your team actually works</row>
      <row label="security">Enterprise-grade security with SOC 2 Type II certification</row>
      <row label="time_savings">Teams report saving an average of 5 hours per week on coordination</row>
 
    </radio>
  </block>
 
  <looprow label="1"><loopvar name="task">1</loopvar></looprow>
  <looprow label="2"><loopvar name="task">2</loopvar></looprow>
  <looprow label="3"><loopvar name="task">3</loopvar></looprow>
  <looprow label="4"><loopvar name="task">4</loopvar></looprow>
  <looprow label="5"><loopvar name="task">5</loopvar></looprow>
  <looprow label="6"><loopvar name="task">6</loopvar></looprow>
</loop>

Styling the MaxDiff grid

Decipher's default radio grid renders as a flat table — all radio inputs in a single row per item. Three <style> elements inside the <radio> override this to produce the two-column best / worst layout.

question.header injects scoped CSS that strips the default table borders from the header legend and item rows, replacing them with a single bottom divider.

question.top-legend and question.row are Decipher template overrides that restructure the rendered HTML. The default templates output all radio inputs together; these split on </th> and </td> boundaries to insert a $(left) spacer column in the middle, producing the three-column layout — best radio, item label, worst radio — that a MaxDiff task requires.

<style mode="before" name="question.header"><![CDATA[
<style type="text/css">
.Q1_maxdiff tr.maxdiff-header-legend {
    background-color: transparent;
    border-bottom: 2px solid #d9d9d9;
}
.Q1_maxdiff tr.maxdiff-header-legend th.legend {
    background-color: transparent;
    border: none;
}
.Q1_maxdiff tr.maxdiff-row td.element {
    border-left: none;
    border-right: none;
    border-top: none;
    border-bottom: 1px solid #d9d9d9;
    text-align: center;
}
.Q1_maxdiff tr.maxdiff-row th.row-legend {
    background-color: transparent;
    border-left: none;
    border-right: none;
    border-top: none;
    border-bottom: 1px solid #d9d9d9;
    text-align: center;
}
</style>
]]></style>
 
<style name="question.top-legend"><![CDATA[
\@if ec.simpleList
    $(legends)
\@else
    <$(tag) class="maxdiff-header-legend row row-col-legends row-col-legends-top ${"mobile-top-row-legend " if mobileOnly else ""}colCount-$(colCount)">
        ${"%s%s" % (legends.split("</th>")[0],"</th>")}
        $(left)
        ${"%s%s" % (legends.split("</th>")[1],"</th>")}
    </$(tag)>
    \@if not simple
  </tbody>
  <tbody>
    \@endif
\@endif
]]></style>
 
<style name="question.row"><![CDATA[
\@if ec.simpleList
    $(elements)
\@else
    <$(tag) class="maxdiff-row row row-elements $(style) colCount-$(colCount)">
        ${"%s%s" % (elements.split("</td>")[0],"</td>")}
        $(left)
        ${"%s%s" % (elements.split("</td>")[1],"</td>")}
    </$(tag)>
\@endif
]]></style>

Data Output Structure

After fieldwork, your data file contains:

• claimSets — the JSON assignment string for each respondent, e.g., [[3, 7, 1, 5], [2, 8, 4, 6], ...]. Each number maps to an item: 3 is easy_setup, 7 is security, and so on.
• Q1_1_best, Q1_1_worst — the item value selected as most/least convincing in task 1. For example, a value of 3 means easy_setup was chosen.
• Q1_2_best, Q1_2_worst — selections for task 2
• ... through Q1_6_best, Q1_6_worst

To run a MaxDiff hierarchical Bayes model, you need to reconstruct the choice task for each respondent: which items were shown (from claimSets), which was selected as best, and which as worst. This is why claimSets is not optional — it is the mapping between your response data and your experimental design.

Design Recommendations

Items per set. 4–6 is the standard range. Below 4, the best-worst tradeoff loses discriminative power. Above 6, cognitive load climbs and satisficing increases.

Number of sets. 6–8 sets gives stable individual-level estimates for HB modeling. Fewer sets are acceptable for aggregate-level analysis only.

Item appearance frequency. With 8 items, 4 per set, and 6 sets, each item appears in 3 sets on average. That is a practical minimum for HB. For larger pools (15+ items), increase rounds until each item appears at least 3 times.

Anchor questions. If you need a per-set importance or willingness-to-pay rating alongside the MaxDiff task, add a <number> or <float> element inside the <block> alongside the radio. Set randomize="1" on the block to control presentation order. Anchor data saves as Q1_md_block_1_youranchorlabel, Q1_md_block_2_youranchorlabel, etc.

Putting It All Together

Here is the complete, self-contained XML. Paste it into your Decipher survey file, swap the question title and row text for your items, and adjust pool, pickN, and rounds to match your design.

<exec when="init">
def q_json(value):
    return repr(value)
 
def q_parse_json(raw):
    if raw is None or raw == "":
        return []
    try:
        return eval(raw, {"__builtins__": None}, {})
    except Exception:
        return []
 
def q_assignment_slot(raw, slot):
    assigned = q_parse_json(raw)
    if slot < 0 or slot >= len(assigned):
        return []
    return assigned[slot]
 
def q_shuffle(items):
    out = list(items)
    shuffle(out)
    return out
 
def q_assign_balanced(pool, pick_n, rounds):
    values = [item.get("value", item) for item in pool]
    deck = []
    assigned = []
    for round_index in range(rounds):
        if len(deck) < pick_n:
            remaining = list(deck)
            refill = list(values)
            shuffle(refill)
            safe = [x for x in refill if x not in remaining]
            deck.extend(safe + [x for x in refill if x in remaining])
        assigned.append(deck[:pick_n])
        deck = deck[pick_n:]
    return assigned
 
def q_sample_assign(pool, pick_n, rounds, strategy):
    if strategy in ("balanced", "sparse", "rotational"):
        return q_assign_balanced(pool, pick_n, rounds)
    return [q_shuffle(pool)[:pick_n] for round_index in range(rounds)]
 
def q_setup_rows_by_labels(question, labels):
    wanted = [str(label) for label in labels]
    row_index = dict((str(r.o.label), r.index) for r in question.rows)
    for r in question.rows:
        r.disabled = str(r.o.label) not in wanted
    question.rows.order = [row_index[label] for label in wanted if label in row_index]
 
def q_setup_rows_by_value_map(question, values, value_to_row):
    labels = []
    for value in values:
        key = str(value)
        if key in value_to_row:
            labels.append(value_to_row[key])
    q_setup_rows_by_labels(question, labels)
</exec>
 
<!-- Hidden variable stores the per-respondent item assignment -->
<text label="claimSets" optional="1" where="execute">
  <title>HIDDEN: claimSets</title>
</text>
 
<!-- Generate balanced sets once when the respondent enters the survey -->
<exec>
pool = [
  {"value": 1, "weight": 1},  # fewer_meetings
  {"value": 2, "weight": 1},  # integrations
  {"value": 3, "weight": 1},  # easy_setup
  {"value": 4, "weight": 1},  # realtime_sync
  {"value": 5, "weight": 1},  # social_proof
  {"value": 6, "weight": 1},  # flexible
  {"value": 7, "weight": 1},  # security
  {"value": 8, "weight": 1},  # time_savings
]
claimSets.val = q_json(q_sample_assign(pool, 4, 6, "balanced"))
</exec>
 
<suspend/>
 
<loop label="Q1_md_loop" randomizeChildren="0" vars="task">
  <title>Q1 - MaxDiff Loop</title>
 
  <block label="Q1_md_block" randomize="1">
    <radio label="Q1_[loopvar: task]"
           adim="cols"
           grouping="cols"
           shuffle="rows"
           ss:questionClassNames="Q1_maxdiff"
           unique="1">
 
      <title>Which statement would most or least convince you to try a new project management tool?</title>
 
      <exec>
Q1_values = q_assignment_slot(claimSets.val, int([loopvar: task]) - 1)
q_setup_rows_by_value_map(
    Q1_[loopvar: task],
    Q1_values,
    {"1": "fewer_meetings", "2": "integrations",  "3": "easy_setup",  "4": "realtime_sync",
     "5": "social_proof",  "6": "flexible",        "7": "security",    "8": "time_savings"}
)
      </exec>
 
      <col label="best">Most Convincing</col>
      <col label="worst">Least Convincing</col>
 
      <row label="fewer_meetings">Cuts time spent in status meetings by automating progress updates</row>
      <row label="integrations">Works seamlessly with the tools your team already uses</row>
      <row label="easy_setup">Easy to set up — most teams are fully onboarded in under a day</row>
      <row label="realtime_sync">Updates sync instantly across devices and time zones</row>
      <row label="social_proof">Trusted by more than 10,000 teams across 80 countries</row>
      <row label="flexible">Flexible enough to match how your team actually works</row>
      <row label="security">Enterprise-grade security with SOC 2 Type II certification</row>
      <row label="time_savings">Teams report saving an average of 5 hours per week on coordination</row>
 
      <style mode="before" name="question.header"><![CDATA[
<style type="text/css">
.Q1_maxdiff tr.maxdiff-header-legend {
    background-color: transparent;
    border-bottom: 2px solid #d9d9d9;
}
.Q1_maxdiff tr.maxdiff-header-legend th.legend {
    background-color: transparent;
    border: none;
}
.Q1_maxdiff tr.maxdiff-row td.element {
    border-left: none;
    border-right: none;
    border-top: none;
    border-bottom: 1px solid #d9d9d9;
    text-align: center;
}
.Q1_maxdiff tr.maxdiff-row th.row-legend {
    background-color: transparent;
    border-left: none;
    border-right: none;
    border-top: none;
    border-bottom: 1px solid #d9d9d9;
    text-align: center;
}
</style>
      ]]></style>
 
      <style name="question.top-legend"><![CDATA[
\@if ec.simpleList
    $(legends)
\@else
    <$(tag) class="maxdiff-header-legend row row-col-legends row-col-legends-top ${"mobile-top-row-legend " if mobileOnly else ""}colCount-$(colCount)">
        ${"%s%s" % (legends.split("</th>")[0],"</th>")}
        $(left)
        ${"%s%s" % (legends.split("</th>")[1],"</th>")}
    </$(tag)>
    \@if not simple
  </tbody>
  <tbody>
    \@endif
\@endif
      ]]></style>
 
      <style name="question.row"><![CDATA[
\@if ec.simpleList
    $(elements)
\@else
    <$(tag) class="maxdiff-row row row-elements $(style) colCount-$(colCount)">
        ${"%s%s" % (elements.split("</td>")[0],"</td>")}
        $(left)
        ${"%s%s" % (elements.split("</td>")[1],"</td>")}
    </$(tag)>
\@endif
      ]]></style>
 
    </radio>
  </block>
 
  <looprow label="1"><loopvar name="task">1</loopvar></looprow>
  <looprow label="2"><loopvar name="task">2</loopvar></looprow>
  <looprow label="3"><loopvar name="task">3</loopvar></looprow>
  <looprow label="4"><loopvar name="task">4</loopvar></looprow>
  <looprow label="5"><loopvar name="task">5</loopvar></looprow>
  <looprow label="6"><loopvar name="task">6</loopvar></looprow>
</loop>

How This Differs from the Official Forsta Docs

Forsta's own documentation for the Indices Method (see Creating a MaxDiff Question — Indices Method) uses a different setup worth understanding before you decide which approach fits your project.

The key difference is a separate design.dat file — a tab-delimited file defining the exact item sets per version and task, generated offline using a tool like Sawtooth Software and uploaded to the project directory. The XML template opens this file at init via setupMaxDiffFile("design.dat"), builds a Python dict keyed by version and task ("v1_t1", "v1_t2", etc.), and uses a <quota> element plus p.markers to assign each respondent to one of the pre-computed versions. The row-disabling function (setupMaxDiffItemsI) then looks up that respondent's version-task key to get their item list.

The approach here diverges from that pattern deliberately, for two reasons.

All business logic lives in one file. With the official approach, the survey's behavior depends on an external artifact generated outside the project. When something goes wrong — items not showing correctly, a row not disabling — you are debugging across a file boundary, and the design file itself was produced by an external tool and is not readable inline. With the self-contained approach here, everything is in the XML: the pool definition, the assignment algorithm, the row-to-value mapping, and the loop. A programmer can read the file top to bottom and trace the full execution path without switching contexts.

Runtime generation is easier to maintain than a static design file. The official design.dat is a snapshot of a fixed experimental design. If the client adds an item, changes the number of tasks, or reorders the pool, the file has to be regenerated externally and re-uploaded. The q_assign_balanced function here takes pool, pick_n, and rounds as arguments in the XML itself — a design change is a one-line edit, and each respondent automatically gets a fresh balanced assignment.

There is a real tradeoff: the official approach assigns respondents to fixed pre-computed versions, which means you can audit the exact design matrix before fielding and verify pairwise co-occurrence properties offline. The runtime approach produces a unique near-balanced design per respondent, which averages out well across a full sample but cannot be inspected as a single design object. For most commercial MaxDiff work the runtime approach is the right call. If your methodology requires a fixed, auditable design — for example, a study following a strict BIBD or one where all respondents must share identical set compositions — the official design file pattern is more appropriate.

How Questra Handles This Automatically

The XML in this post is exactly what Questra generates when you program a MaxDiff survey. You describe the study in a simple definition — your items, how many to show per set, how many sets — and Questra's survey compiler produces the complete Decipher XML: the <exec when="init"> helper functions, the hidden assignment variable, the balanced sampling exec, the loop structure, the per-set row filtering, and the CSS layout overrides. The same code you just read through is what lands in your survey file.

This is the core of what Questra does for survey programming more broadly: researchers and programmers describe what a survey should do, and the platform handles the translation to platform-specific XML. For Decipher, that means generating the Indices Method pattern for MaxDiff, the correct loop structures for conjoint exercises, proper skip logic, and all the other boilerplate that experienced programmers have memorized but still have to type. The output is auditable, version-controlled XML that you own — not a black box.

If you are programming MaxDiff surveys in Decipher regularly, or working across multiple platforms and need consistent output, learn more about Questra at questra.ai.