Deployment and experiments
Use agent versions, experiments, and traffic rules together to test a change on a controlled share of new web chat, email, and SMS conversations.
Current scope: Starting an experiment routes eligible new web chat, email, and SMS conversations immediately; it does not schedule a future start. Applied persists contact assignments and live-agent exposures and provides routing diagnostics. Outcome results remain labeled Legacy estimate and should be treated as directional rather than decision-grade statistics.
The three controls
Open an agent and select Deploy. The navigation follows the order in which you set up a test:
| Area | What it controls | What it does not do |
|---|---|---|
| Versions | Immutable snapshots of agent configuration | Does not split traffic |
| Experiments | Hypothesis, variants, allocation, primary metric, and guardrails | Does not decide who is eligible |
| Traffic rules | Audience, rollout gate, rule priority, and destination | Does not set the split between experiment variants |
The complete path is:
- Save the control and treatment as separate agent versions.
- Create a Draft experiment and define its hypothesis, variants, allocation, primary metric, and at least one guardrail.
- Create an active traffic rule that targets the experiment.
- Review the launch checklist, then select Start experiment when you are ready for eligible traffic to be routed.
- Check assignment and exposure diagnostics before reading the Legacy estimate, then stop the experiment and deploy the chosen version manually.
Rollout and allocation are different
Suppose a traffic rule has a 20% rollout and the experiment has a 50/50 variant allocation:
- The traffic rule first selects about 20% of contacts that match its conditions.
- The experiment then divides those eligible contacts between the two variants.
- Traffic that does not pass that rule continues to the next matching rule or the live-version fallback.
Changing the rule rollout changes how much traffic enters the experiment. Changing variant allocation changes how experiment traffic is divided.
What remains manual
Applied currently does not:
- schedule an experiment start;
- evaluate experiment guardrails or stop a test automatically;
- produce a decision-grade treatment-effect analysis or declare a winner;
- promote a variant, archive routing, or roll back automatically.
Record the planned observation window and stop conditions before launch. Watch operational guardrails in the relevant analytics pages, then follow the manual shutdown sequence in Create and Run an Experiment.
Start here
- Ask Applied Assistant to prepare a validated Draft when you already know the treatment, primary outcome, guardrail, allocation, and audience.
- Create Agent Versions
- Create and Run an Experiment
- Read Legacy Estimates