Experiments and Reporting

ACI experiments use the same compiled-variant and immutable-release model as audience experiences. The base content is the control. Each variant arm adds an overlay, and ACI selects a stable assignment before returning the response.

Experiment resources

.content/
├── data/
│   ├── goals/{goal-id}/_index.json
│   └── experiments/{experiment-id}/_index.json
└── overlays/
    └── experiment/{experiment-id}/experimentArm/{arm-id}/
        └── pages/{page-id}/_index.json

Site configuration must include both experiment dimensions. Preserve any existing locale and audience dimensions:

{
  "overlayResolution": [
    "locale",
    "audience",
    "experiment",
    "experimentArm"
  ]
}

1. Define the goal

Create .content/data/goals/request-demo/_index.json:

{
  "$type": "goal",
  "props": {
    "id": "request-demo",
    "displayName": "Demo requested",
    "status": "enabled",
    "eventName": "custom.request-demo"
  }
}

The event name must match the canonical name captured by the browser collector. For example:

<button data-aci-event="request-demo">
  Request a demo
</button>

See Collection and Consent for JavaScript and markup tracking.

2. Define the experiment

Create .content/data/experiments/home-hero/_index.json:

{
  "$type": "experiment",
  "props": {
    "id": "home-hero",
    "displayName": "Homepage hero message",
    "status": "draft",
    "kind": "ab",
    "priority": 100,
    "allocationVersion": "v1",
    "salt": "home-hero-v1",
    "scope": {
      "routes": ["home"]
    },
    "primaryGoalRef": "$ref:data/goals/request-demo",
    "arms": [
      {
        "id": "control",
        "displayName": "Current message",
        "role": "control",
        "allocationBasisPoints": 5000
      },
      {
        "id": "outcome-led",
        "displayName": "Outcome-led message",
        "role": "variant",
        "allocationBasisPoints": 5000
      }
    ]
  }
}

Key rules:

ab and abn experiments require at least two arms and exactly one control;
allocation uses basis points and must total 10000;
scope.routes contains ACI page IDs, not public URL paths;
allocationVersion and salt keep randomized assignment stable; do not change them after an experiment starts; and
higher priority resolves conflicts when experiment scopes overlap.

The xt kind supports rule-based experience testing. Use it when arms have explicit request rules instead of randomized allocation.

3. Add a variant overlay

The control uses the base page. Create an overlay only for the variant at .content/overlays/experiment/home-hero/experimentArm/outcome-led/pages/home/_index.json:

{
  "$type": "page-overlay",
  "regions": {
    "main": {
      "hero": {
        "props": {
          "headline": "Move approved content from idea to production.",
          "description": "Build, review, personalize, and publish through one governed workflow."
        }
      }
    }
  }
}

The resolved arm must satisfy the same component contracts as the control.

4. Compile and preview both arms

npm run content:compile
npm run typecheck
npm test
npm run build

Push the draft to a branch, then use explicit dimensions on the ACI preview surface:

<preview-url>/?aci_experiment=home-hero&aci_experimentArm=control
<preview-url>/?aci_experiment=home-hero&aci_experimentArm=outcome-led

These selectors are preview controls. Live traffic follows the experiment definition and assignment policy. Review content, metadata, links, responsive layout, consent behavior, and the goal interaction on both arms before landing the branch.

5. Start the experiment

After the experiment and goal are approved on main, use the experiment API to transition the draft to running. Start with a dry run:

curl --request POST \
  "$ACI_API_URL/v1/experiments/home-hero/status" \
  --header "Authorization: Bearer $ACI_API_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{
    "status": "running",
    "operationId": "start-home-hero-v1",
    "dryRun": true
  }'

Repeat with "dryRun": false after reviewing the response. Then promote the experiment, goal, and overlay paths required for the run to live. Stage a release, review it, and activate it. The supported operational transitions are:

draft → running
running → paused
paused → running

Use a new operation ID for a new action. Reuse the same operation ID only when retrying that action.

6. Read the experiment report

curl \
  "$ACI_API_URL/v1/reports/experiments/home-hero" \
  --header "Authorization: Bearer $ACI_API_TOKEN"

The report groups exposure and conversion results by arm and includes the primary-goal statistics used for winner evaluation. Interpret a result only after checking:

exposure volume and data quality;
whether the sample is sufficient;
the control and variant conversion rates;
confidence information for the primary goal; and
whether consent or collection changes affected the measurement window.

Secondary goals are useful context, but choose and review the primary goal before starting the experiment.

7. Pause or promote

Pause through the status endpoint when traffic should stop entering the experiment. Existing release history remains available for investigation. When an arm has been approved as the winner, dry-run winner promotion:

curl --request POST \
  "$ACI_API_URL/v1/experiments/home-hero/promote-winner" \
  --header "Authorization: Bearer $ACI_API_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{
    "armId": "outcome-led",
    "operationId": "promote-home-hero-outcome-led",
    "dryRun": true,
    "note": "Approved after experiment review"
  }'

The dry run lists the base content that will change and the experiment overlays that will be removed. Repeat with "dryRun": false only after that plan is correct. Winner promotion requires write, promote, and publish permissions. It merges the selected arm into the base content, marks the experiment promoted, advances the affected paths to live, and publishes the resulting release as one operation.

Winner promotion changes live content. Record the active release before the operation and complete the same post-publish checks you use for a normal release.

Common problems

Compilation says the allocation is invalid

Confirm there is exactly one control, at least one variant, and all allocationBasisPoints values total 10000.

A preview arm looks identical to control

Confirm the overlay path contains the exact experiment and arm IDs, and that its block IDs exist in the base page. A control arm normally has no overlay.

The experiment cannot start

Confirm its scoped page or fragment exists, every arm compiles, the goal resolves, and both experiment dimensions appear in overlayResolution.

The report has no conversions

Confirm the goal’s canonical event name matches collection, the interaction occurs after assignment, and the relevant consent and collection path is working.

Next: Preview, Publish, and Roll Back →

​Experiment resources

​1. Define the goal

​2. Define the experiment

​3. Add a variant overlay

​4. Compile and preview both arms

​5. Start the experiment

​6. Read the experiment report

​7. Pause or promote

​Common problems

Experiment resources

1. Define the goal

2. Define the experiment

3. Add a variant overlay

4. Compile and preview both arms

5. Start the experiment

6. Read the experiment report

7. Pause or promote

Common problems