Customer Experience Lab

Overview

Customer Experience Lab is the layer between building an AI agent and deploying it to production. Most teams treat deployment as a single action: you make a change, you push it live, and you find out how well it works from your customers. CX Lab changes that by giving your organization a governed environment to version every change, validate it through simulation before any live customer sees it, compare versions side by side, and route every deployment through an approval workflow with a permanent audit trail.

The full AI change lifecycle in CX Lab is: create a configuration version, run a simulation to generate a scorecard and deployment recommendation, optionally compare it against a previous version, request a deployment approval, complete the review and approval workflow, deploy to production, and record the full deployment chain. Every version, simulation, approval, and override is permanently logged. Rollback to any previous version is a single action.

Requirements

To access Customer Experience Lab you need:

• Enterprise plan (CX Lab is not available on Plus, Pro, or any other plan)
• Owner, Admin, Developer, Manager, or Analyst role. Support and CoPilot User roles do not have access to CX Lab.
• At least one published AI agent in your ChatSpark account

Configuration Versions

A configuration version is a frozen, immutable snapshot of your agent's complete configuration at a specific point in time. Once created, it cannot be modified. It captures everything that defines how your agent behaves: system instructions, personality, training data references, automation rules, conversation routing, connected tools and AI Actions, and model settings.

Configuration versions are the core building block of CX Lab. Simulations, comparisons, and deployment approvals all reference a specific version. This is what makes results reproducible and audit trails complete.

Configuration versions are distinct from the AI Operator's change history. The AI Operator logs individual field edits with before and after values. A configuration version is a full snapshot of everything at a point in time, intended for simulation and deployment governance rather than granular change tracking.

Creating a Version

A configuration version is created in three ways:

• Running a simulation. CX Lab automatically creates a version snapshot before a simulation starts. If your agent has unsaved changes, you will be prompted to save them first.
• Requesting a deployment. A version is created as part of the deployment request if one has not been explicitly saved.
• Manual save. Use the Save Version button on any agent to capture the current configuration as a named version at any time.

Version Labels

Every version receives an auto-generated label based on the timestamp and a short hash. You can replace this with a custom label at creation time or afterward, for example "May 2026 pricing update" or "v3.1 escalation routing." Custom labels appear throughout the CX Lab UI and in reports.

Version Status

• Draft — created but not yet used in a simulation or deployment
• Published — the version that is currently live in production
• Archived — a previous published version that has been superseded

Running a Simulation

A simulation evaluates a configuration version against a set of personas and validation checks. The result is an AI Scorecard, a deployment recommendation, and a regression check report.

To run a simulation:

1. Go to CX Lab in your dashboard and click Simulations
2. Click New Simulation
3. Select the agent you want to validate
4. Select the configuration version to validate (defaults to the current version)
5. Choose a simulation type (see below)
6. Select one or more personas
7. Review the resource estimate (runtime, AI usage, conversation count)
8. Click Start Simulation

You can also start a simulation from the Validate in CX Lab button on the Agent Intelligence page and the agent editor, which pre-selects the agent for you.

Simulation Types

CX Lab uses a tiered evaluation engine with three layers. Different simulation types run different combinations of these layers.

• Deterministic checks. Fast, rule-based validation. Checks required phrases, prohibited claims, routing rules, escalation triggers, tool execution, lead capture fields, and compliance patterns. No model calls. Results are pass, fail, or warning. These always run.
• LLM evaluation. A model-as-judge layer that scores each category of the AI Scorecard. The judge reads each conversation turn and produces a numerical score with reasoning. This layer runs in Quick Validation and Full Synthetic modes.
• Full synthetic conversations. Real model calls using the selected personas as the simulated customer. The agent responds exactly as it would in production. These conversations are scored by the LLM evaluation layer. This layer runs only in Full Synthetic mode.

Use Quick Validation for fast feedback during active development. Use Full Synthetic before requesting a production deployment to get the most complete picture of how your agent will behave.

Resource Controls

Before starting, CX Lab shows you an estimate of the simulation's runtime, AI token usage, and total conversation count. Larger persona sets and higher conversation volumes increase both runtime and AI usage. You can adjust the conversation count per persona to control the scope.

You can cancel a running simulation at any time from the simulation detail page.

Simulation Status

• Pending — queued and waiting to start
• Running — actively processing conversations and checks
• Completed — all checks and evaluations finished successfully
• Cancelled — stopped manually. Partial results are available.
• Failed — an error prevented the simulation from completing

AI Scorecard

Every completed simulation produces an AI Scorecard. The overall score is a number from 0 to 100, calculated from seven weighted categories:

• Answer Accuracy — correctness and factual grounding
• Tone and Professionalism — alignment with your brand voice
• Completeness — whether questions are fully answered
• Empathy — handling of emotional or frustrated customer types
• Context Retention — maintaining context across a multi-turn conversation
• Conversation Flow — logical progression and coherence
• Resolution Quality — whether the conversation reached a successful resolution

The version delta view shows how each category score changed compared to the previous configuration version, so you can see exactly what improved and what regressed.

Below the scores, CX Lab generates an actionable items list. Each item maps to a low-scoring category and describes a specific change you can make to address it.

Deployment Recommendation

After a simulation completes, CX Lab generates a deployment recommendation. The recommendation is produced by synthesizing the overall scorecard, per-category scores, regression outcomes, and version delta. It appears at the top of the simulation detail page in plain language, with the reasoning included.

There are four recommendation states:

• Ready to Deploy. Scorecard is strong, regression checks pass, and no significant regressions from the previous version. Safe to proceed through your governance workflow.
• Deploy with Warnings. Scorecard is acceptable but one or more categories have notable gaps or minor regressions. Review the action items before deploying.
• Requires Review. One or more categories have significant score drops, regression warnings are present, or the version delta shows meaningful regression from the production version. Review and address before proceeding.
• Do Not Deploy. Regression checks have blocking failures, the scorecard has critical gaps, or there is severe regression from the current production version. Do not deploy this configuration version without addressing the issues flagged.

The recommendation is advisory. Your governance policy determines what happens next. Under Advisory mode you can deploy regardless of the recommendation. Under Required and Strict modes, regression failures block deployment.

Personas

Personas represent different types of customers your agent might encounter in production. Each persona has a name, a communication style, a buying intent level, urgency, sentiment, and decision authority. During a Full Synthetic simulation, the persona drives the simulated customer's side of the conversation.

CX Lab includes 16 built-in personas:

• First-Time Visitor — curious, low buying intent, early-stage exploration
• Frustrated Customer — high urgency, negative sentiment, looking for a fast resolution
• Enterprise Buyer — methodical, high decision authority, evaluating for a team
• Technical Evaluator — asks deep product and integration questions
• Price-Sensitive Shopper — focused on cost, comparing alternatives
• Returning Customer — familiar with the product, has a specific follow-up need
• High-Intent Buyer — ready to purchase, looking for confirmation or next steps
• Skeptical Prospect — challenges claims, needs evidence before engaging
• Support-Seeking User — has a problem and needs it resolved quickly
• Quiet Browser — minimal input, short responses, hard to qualify
• Urgent Escalator — immediately asks to speak with a human
• Compliance-Focused Buyer — asks about security, privacy, and regulatory requirements
• Onboarding User — recently purchased, looking for setup guidance
• Churning Customer — considering leaving, needs a reason to stay
• Internal Team Member — employee using a CoPilot-type internal agent
• Non-Native Speaker — communicates with imprecise phrasing, tests multilingual capability

To create a custom persona, go to CX Lab and click Personas, then click New Persona. Give it a name, a plain-language description of how this customer thinks and communicates, and configure the communication style, buying intent, urgency level, sentiment, and decision authority. Custom personas are reusable across all agents and all future simulations.

Regression Testing

Regression testing validates that a new configuration version does not break something that was working in the previous version. A regression suite is a saved set of deterministic checks that run automatically whenever a new version is created under Required or Strict governance, and can also be triggered manually at any time.

The types of checks you can include in a regression suite:

• Required phrase — the agent must include a specific phrase in its response
• Prohibited claim — the agent must not make a specific statement
• Routing rule — a conversation matching certain conditions must route to the correct destination
• Escalation trigger — specific inputs must trigger a handoff to a human agent
• Tool execution — a connected AI Action must be called in the expected scenario
• Lead capture — required lead capture fields must be collected in the defined scenarios
• Compliance check — regulated language and disclaimer requirements are met

Creating a Regression Suite

Go to CX Lab and click Regression, then clickNew Suite. Give the suite a name, select the agent it applies to, and add your checks. Each check has a type, a description, the input that triggers it, and the expected result. Save the suite and it is immediately available to run manually or automatically on version creation.

Each check produces one of four result states:

• Pass — the check completed successfully
• Fail — the check did not pass
• Warning — the result was borderline, review is recommended
• Blocked — a dependency check failed, so this check could not run

Running a Regression Suite

You can run a regression suite manually at any time from the Regression page. Under Required and Strict governance, regression suites attached to your agent run automatically when a new configuration version is saved. Results are linked to the version and appear in the deployment request.

Reading Results

Results appear in a grid on the regression run detail page. Each row shows the check, the result state, and a brief explanation. Under Required governance, a failing check blocks deployment unless an Owner or Admin overrides with a mandatory reason. Under Strict governance, all blocking checks must pass with no override path available.

Comparisons

Comparisons let you run two configuration versions against the same conversation set and compare the results directly. You can compare using synthetic conversations or historical production traffic. The result shows the scorecard delta, regression delta, and a business impact projection for each version.

To create a comparison:

1. Go to CX Lab and click Comparisons
2. Click New Comparison
3. Select the two configuration versions you want to compare
4. Choose synthetic conversations or historical traffic as the source
5. Click Run Comparison

Comparisons are a pre-deployment decision-making tool. Unlike production A/B testing, no live customers are exposed during a comparison. All traffic is simulated or drawn from recorded history.

Knowledge Coverage

Knowledge Coverage shows you a topic-level view of what your agent knows and where it has gaps. The coverage rate reflects how much of your training data overlaps with the topics that appear in real customer conversations.

The gap queue surfaces topics that appeared in real conversations but are not covered in your current training data. Each gap includes the questions customers asked, the frequency, and a direct link to Training Suggestions where you can address it by adding or updating your training sources.

Deployment Governance

Deployment Governance is a configurable policy that determines what validation and approval steps are required before a configuration change can go to production. There are three modes:

• Advisory. Validation is recommended but not required. You can deploy at any time. A non-blocking prompt appears when you attempt to deploy without a passing simulation, reminding you to validate first. You can dismiss it and proceed.
• Required. A simulation must complete and all regression checks must pass before a deployment proceeds. An Owner or Admin can override a failing check, but the override requires a written reason and is permanently logged in the audit trail.
• Strict. All blocking regression checks must pass and a designated approver who is different from the requestor must explicitly approve before any deployment proceeds. There is no override path. The four-eyes principle is enforced at the platform level.

To set your governance mode, go to CX Lab and click Settings. The default mode is Advisory. You can change it at any time.

Deployment Approval Workflow

When you are ready to deploy a validated configuration version to production, you submit a deployment approval request. The request includes the configuration version, the simulation results, a deployment note, and the governance policy in effect at the time of the request. Once a deployment completes, the full approval chain is immutable.

Requesting a Deployment

1. From the simulation detail page, click Request Deployment
2. The configuration version and simulation results are attached automatically
3. Write a deployment note describing the change and the intent
4. Click Submit Request

The request is visible to all Owners and Admins in the Deployments section of CX Lab. Notification delivery depends on your team notification settings.

Reviewing a Request

A reviewer opens the deployment request, reads the simulation results and deployment note, and adds a review note. Marking a request as reviewed signals that it is ready to move to the final approval step. In multi-approver workflows, marking as reviewed passes the request to the designated approver.

Approving or Rejecting

The approver reads the full request and review notes, then approves or rejects. Approval triggers the deployment to production. Rejection closes the request and requires the requestor to address the feedback and open a new request.

Under Strict governance, the approver must be a different person from the requestor. The platform enforces this and will not allow self-approval.

Bypassing Validation

In Advisory mode, an Owner or Admin can deploy directly without a simulation by confirming the bypass prompt. Every bypass creates a permanent audit record with the identity of the person who bypassed, the time, and the version deployed.

Deployment History and Rollback

Every deployment made from CX Lab is recorded in the deployment history for that agent. Each record shows the configuration version that was deployed, the simulation results at the time of deployment, the full approval chain (who requested, who reviewed, who approved), and the timestamp.

To roll back to a previous version, open the deployment record and clickPromote to Production. Rollback creates a new deployment record with the same governance requirements as any other deployment.

Reports

CX Lab includes five built-in report templates. All reports support PDF and CSV export.

• Executive Summary. Deployment frequency, override rate, CX score trend, and regression pass rate over a selected date range. Designed for leadership review.
• Operational. Full simulation run history, average time from simulation to approval, cancellation rate, and simulation volume by agent.
• Compliance. Full override log, approval chain records for every deployment, and a complete log of governance policy changes. Suitable for SOC 2 and internal audit review.
• Performance. Per-agent scorecard trends over time, category-level score movement, and knowledge coverage trend. Useful for tracking improvement across the team.
• Custom. Select your own metrics, agents, and date range to build a report tailored to your organization's needs.

Team and Permissions

CX Lab respects your existing ChatSpark team roles. Access is scoped by role as follows:

Plan and Limits

• Plan required: Enterprise
• Simulation conversation volume: 10 to 500 per run, capped server-side to protect your monthly message limit
• Usage billing: Each synthetic conversation counts against your plan monthly message limit. If your agent has a custom OpenAI API key configured, that key is used instead and no plan messages are consumed.
• Custom persona library: Unlimited
• Built-in personas: 16
• Configuration version history: Unlimited
• Deployment history: Unlimited
• Report export: PDF and CSV

FAQs

What is the difference between CX Lab Comparisons and production A/B testing?

Production A/B testing splits live customer traffic between two agent configurations and measures real-world outcomes over time. CX Lab Comparisons run both versions against synthetic conversations or recorded historical traffic with no live customer exposure. Comparisons are a pre-deployment decision tool. A/B testing measures production performance after deployment.

Can I run a simulation without creating a configuration version?

No. Every simulation requires a configuration version as its basis. If you start a simulation without explicitly saving a version first, CX Lab creates one automatically from your current agent configuration before the simulation begins.

What happens to a simulation if I cancel it while it is running?

Partial results are retained. The simulation is marked as incomplete and the results that were generated before cancellation are available to view. The scorecard will reflect only the conversations and checks that completed. You can run a new simulation at any time.

What does the Deployment Recommendation consider when generating a result?

The recommendation is generated by synthesizing the overall scorecard score, the per-category breakdown, the outcomes of all regression checks, and the version delta compared to the current production version. The reasoning is included in plain language on the simulation detail page.

Can a Developer approve their own deployment request?

No. The person who requests a deployment cannot be the same person who approves it. This applies in all governance modes. Under Strict governance, the platform enforces this at the API level and will reject a self-approval attempt. Under Advisory and Required modes, the same rule applies and is enforced in the approval UI.

What is the difference between Required and Strict governance modes?

In Required mode, all regression checks must pass before deployment, but an Owner or Admin can override a failing check by providing a written reason. The override is logged. In Strict mode, there is no override path. All blocking checks must pass and an independent approver must approve before deployment can proceed. Governance policy changes in Strict mode are also permanently audited.

Does CX Lab replace the AI Operator's change history?

No. The AI Operator logs individual field-level changes with before and after values. CX Lab configuration versions are full snapshots intended for simulation, comparison, and deployment governance. They serve different purposes and both are available to you.

Which roles can access CX Lab?

Owner, Admin, Developer, Manager, and Analyst roles can access CX Lab. Support and CoPilot User roles do not have CX Lab access. Specific capabilities within CX Lab vary by role. See the Team and Permissions table above for the full breakdown.