Skip to content
WorkshopOnlinePast event

AI Evals and Observability Strategies

Event artwork for AI Evals and Observability Strategies

This event has ended

This listing is kept as an archive. Explore upcoming events to find your next one.

See upcoming events
About this event

What to expect

Build a practical, evidence-based strategy for evaluating AI agents in this 75-minute online workshop with Supreet Kaur on 18 September 2026.

Creating an AI agent is only the first step. Once it begins planning, selecting tools and making decisions across multiple stages, you need reliable ways to determine whether it works,and understand why it fails. AI Evals and Observability Strategies focuses on the methods teams can use to measure agent quality, diagnose problems and move agentic applications from promising prototypes towards production-ready systems.

Running from 2:00pm to 3:15pm UTC, this online workshop is presented by Packt Publishing Limited and led by Supreet Kaur. The session moves beyond subjective judgements such as “the agent seems to work”, replacing them with a repeatable approach grounded in test cases, evaluation criteria, traces and production signals.

What the workshop covers

The agenda begins by examining why agent evaluations differ from conventional software testing and what teams should evaluate. You’ll then explore how to:

  • Design meaningful test cases and agent quality criteria
  • Run offline evaluations before deploying changes
  • Assess tool use, reasoning paths and multi-step execution
  • Evaluate agent trajectories and intermediate decisions
  • Use traces and production telemetry to investigate failures
  • Introduce online evaluations for continuous quality monitoring
  • Combine offline and online evaluation into an evolving strategy

Who should attend?

This workshop is designed for web and software developers building agentic applications, AI engineers, platform and backend engineers, technical leads, architects, and teams taking AI agents from prototype to production. It will be particularly relevant if you already have an agent or agentic workflow and want a more systematic way to measure, debug and improve its behaviour.

Attendees will leave with a practical evaluation framework, guidance on selecting useful metrics and a clearer understanding of observability for production agents. The session also aims to give you the vocabulary and intuition needed to discuss agent behaviour more precisely, investigate failures effectively and make better product decisions.…

Register for this event

Running an AI event?

List it here, free.

Meetups, workshops and conferences go in front of thousands of builders after a quick review.

List your event