Intercom Evals and Releases Brings Controlled Testing to Fin Deployments.

Intercom is giving support teams a way to test Fin before putting changes in front of customers. The company’s new Evals and Releases system combines simulated testing, controlled deployment and live-conversation evaluation, addressing one of the harder problems in operating an AI customer support agent at scale.

The company announced Evals and Releases on August 13, positioning it as a complete evaluation system for Fin. Rather than relying on a handful of manually tested prompts, teams can use Evals to assess Fin’s behavior across simulated scenarios built from their own customer conversations.

Those simulations can be grouped into an Eval around a specific theme. Intercom gives refund requests, escalation rules and tone of voice as examples. That structure should let support leaders test individual policies or behavioral changes without treating the AI agent as a single, opaque system.

AI support systems can behave differently when policies, knowledge sources or instructions change. A modification intended to improve one type of response may create problems elsewhere, particularly when customer questions are ambiguous or combine several issues.

Evals is designed to expose those problems before deployment. Its use of scenarios derived from a company’s own conversations is notable because generic benchmarks often fail to reflect the language, edge cases and operational rules found in a real support queue. Teams could, for example, organize separate evaluations around billing questions and escalation behavior, then assess a proposed change against both areas.

Releases adds a deployment layer after testing. Intercom says teams can roll out changes with control, although its announcement does not specify the available release mechanisms or whether teams can define percentages, audiences or other rollout conditions. The practical goal is clear: separate making a change from exposing every customer to it immediately.

Together, the features outline a more deliberate operating cycle:

• Build simulated scenarios from existing conversations.
• Group related scenarios into an Eval.
• Test changes against the relevant customer-support behaviors.
• Release approved changes in a controlled way.
• Evaluate performance across live conversations.

This is closer to software release management than the prompt-and-publish workflow that has characterized many early AI agent deployments.

Intercom is pairing Evals and Releases with Monitors, its system for evaluating Fin’s live conversations. Evals covers expected behavior before a change reaches customers; Releases governs deployment; Monitors provides the production feedback loop afterward.

The combination matters because pre-release tests cannot anticipate every customer interaction. Live evaluation can identify issues that only emerge through unusual phrasing, incomplete account information or combinations of policies that were not represented in the original scenarios. Intercom says the system can evaluate every live conversation, moving quality review beyond the small samples that human managers can inspect manually.

For smaller businesses, the main benefit may be operational confidence rather than raw automation. A lean support team cannot review every AI response, but it still needs a way to detect regressions and verify that Fin follows escalation and tone requirements. The usefulness of the system will depend on how easily teams can create meaningful evaluations and act on the resulting findings.

Evals and Releases also fits Intercom’s broader effort to build management infrastructure around AI support. It follows the company’s introduction of Operator, an AI agent intended to manage customer support operations, and its work on a helpdesk designed for collaboration between AI and human agents.

Intercom has not disclosed separate pricing, plan requirements or detailed release controls in the announcement. Those omissions leave open important implementation questions, but the direction is significant: Fin is gaining the testing, deployment and monitoring layers needed to treat an AI agent as a production system rather than a standalone chatbot.

What is Intercom Evals and Releases?

Intercom Evals and Releases is an evaluation and deployment system for Fin. It lets support teams test Fin using simulated scenarios, roll out changes with greater control and evaluate how the AI agent performs in live customer conversations.

How does Evals test Fin?

Evals uses simulated scenarios built from a company’s own conversations. Teams group those simulated conversations around themes such as refunds, escalation rules or tone of voice, then use them to assess Fin’s behavior at scale.

What does Intercom Evals and Releases cost?

Intercom’s announcement does not specify pricing or plan requirements for Evals and Releases. Existing or prospective customers will need to check the product page or contact Intercom for current availability and commercial terms.