Skip to content
In PersonIn personPast event

Observability, Testing, and Trust for Agentic AI with MLflow

Tools & platforms

Event artwork for Observability, Testing, and Trust for Agentic AI with MLflow

This event has ended

This listing is kept as an archive. Explore upcoming events to find your next one.

See upcoming events in Washington

Supporting this event

Sponsors

Booz AllenGWU MS in Business AnalyticsDC Tech LiveInbox America
About this event

What to expect

Can you really trust an AI agent that scaffolds an entire application from a single prompt? When your coding assistant makes fifty tool calls, rewrites three files, and quietly introduces a security vulnerability, debugging becomes a nightmare. This Data Science DC meetup tackles one of the most critical challenges in agentic AI: building systems you can actually observe, test, and trust in production.

Join Tim Lortz, AI Product Specialist at Databricks, for a hands-on exploration of MLflow's tracing and evaluation framework. This isn't theoretical — you'll follow a complete workflow from development through production monitoring. The session demonstrates how to build an AI agent using Claude Code as a coding assistant, trace both the assistant's work and the agent it produces, then evaluate quality using LLM judges. You'll see how to close the loop from initial development through systematic testing and production observability.

What You'll Learn

  • Practical observability techniques for agentic AI systems that go beyond "it works on my machine"
  • MLflow tracing framework for tracking complex multi-step agent workflows
  • Systematic testing approaches using LLM judges for quality evaluation
  • Production monitoring strategies for mission-critical AI systems
  • Open-source toolkit with no vendor lock-in required

This session is perfect for data scientists, ML engineers, and AI practitioners working on production systems that require compliance, reliability, and auditability. Whether you're building coding agents, customer support bots, or any autonomous AI system, you'll gain concrete strategies for transforming black-box agent behaviour into inspectable, measurable workflows.

About the Speaker

Tim Lortz serves as the technical lead for Databricks' go-to-market efforts for AI in regulated industries, focusing on enterprise-grade governance for AI platforms. With a background spanning data science practitioner and leadership roles in Federal contracting, plus a PhD in Industrial & Operations Engineering from the University of Michigan, Tim brings deep expertise in building trustworthy AI systems for high-stakes environments.

The evening begins at 6:00 PM with networking at Prefect's Washington DC office, followed by introductions at 6:30 PM and the main presentation at 6:45 PM. This in-person event offers valuable face-time with the DC data science community and sponsors including Booz Alle

Running an AI event in Washington?

List it here, free.

Meetups, workshops and conferences go in front of thousands of builders after a quick review.

List your event