Skip to content
VirtualOnlinePast event

LLM evaluation: A live demo

Event artwork for LLM evaluation: A live demo

This event has ended

This listing is kept as an archive. Explore upcoming events to find your next one.

See upcoming events

Supporting this event

Sponsors

Sauce LabsTUITrendig
About this event

What to expect

Discover how to evaluate large language models in this hands-on online demonstration that tackles one of the biggest challenges facing test automation engineers today: testing non-deterministic AI systems. Hosted by CTM Continuous Testing Meetup Berlin, this session bridges the gap between traditional quality assurance practices and the emerging world of LLM testing.

What You'll Learn

Anupam Krishnamurthy will walk you through the complete evaluation process for a Retrieval Augmented Generation (RAG) model using an LLM-as-judge approach. You'll see exactly how one large language model can systematically assess the outputs of another — a critical capability for anyone building or testing AI-powered applications.

The live demonstration covers:

  • RAG model internals: Understanding how Retrieval Augmented Generation systems actually work under the hood
  • Automated evaluation setup: Implementing LLM-as-a-judge frameworks to assess model performance
  • Debugging evaluation results: Examining stack traces and investigating unexpected outcomes from your evaluation framework
  • Security testing: Conducting an indirect injection attack to expose potential vulnerabilities in the RAG model

Why This Matters

For years, test automation engineers have built their craft on eliminating non-deterministic behaviour from systems. Now we're testing software that is fundamentally non-deterministic by design. This session addresses that paradigm shift head-on, showing practical techniques for bringing rigour and consistency to LLM evaluation.

Whether you're currently working with AI agents, planning to integrate LLMs into your products, or simply curious about quality assurance in the age of generative AI, this demonstration provides concrete strategies you can apply immediately.

Format & Audience

This 75-minute online event includes a 50-minute live demonstration followed by 15 minutes of Q&A and open discussion. The session is ideal for QA engineers, test automation specialists, software developers working with LLMs, and anyone involved in building or evaluating AI-powered systems.

The event is sponsored by Sauce Labs, TUI, and Trendig, who support the continuous testing community in developing best practices for emerging technologies.

[Register for this event](https://www.meetup.com/continuous-testing-meetup-berlin/events/314193475/?eventOrigin=gro

Running an AI event?

List it here, free.

Meetups, workshops and conferences go in front of thousands of builders after a quick review.

List your event