Skip to content
MeetupIn personPast event

Your Evals Are Bad: Evaluation and the Model Development Lifecycle

Event artwork for Your Evals Are Bad: Evaluation and the Model Development Lifecycle

This event has ended

This listing is kept as an archive. Explore upcoming events to find your next one.

See upcoming events in Washington

Supporting this event

Sponsors

Booz AllenGWU MS in Business AnalyticsDC Tech LiveInbox America
About this event

What to expect

If you've ever shipped a machine learning model, celebrated improved metrics, only to discover through user feedback that something was fundamentally wrong, this event is for you. Join Data Science DC for an essential talk on model evaluation by Mary Gibbs, Senior Applied Scientist at Relativity, who will explain why your evaluation metrics may have been misleading all along—you just didn't know it yet.

Why Your Evaluation Strategy Matters

This session tackles a critical but often overlooked problem in the model development lifecycle: the gap between what your metrics say and what your users experience. Mary will break down the three core components of any evaluation—benchmarks, scorers, and claims—and expose how each can fail in ways that compound throughout your development process. When teams optimize directly against flawed metrics, measurement problems evolve into model problems that impact real users.

What You'll Learn

  • Benchmark Weaknesses: How narrow coverage, contamination, and saturation undermine your evaluation datasets
  • Scorer Selection Pitfalls: Why choosing scorers for automation convenience rather than user outcome alignment leads to blind spots
  • The Missing Link: How implicit claims about what scores represent create disconnects between metrics and reality
  • Lifecycle Consequences: How these evaluation gaps compound as teams iterate and optimize
  • Practical Solutions: Actionable advice for building more robust evaluation frameworks that align with actual user needs

About the Speaker

Mary Gibbs brings a unique perspective shaped by diverse data science experience across e-discovery, legal tech, career progression, cybersecurity, and government consulting. Before transitioning to data science, she conducted neurodevelopment research at the National Institutes of Health. She holds an M.S. in Data Science from The George Washington University and a B.A. in Biological Sciences from Cornell University. Beyond her work at Relativity, Mary is an active organizer for Women and Gender eXpansive Coders DC, supporting women and nonbinary individuals in tech careers.

Event Format

The evening begins with networking from 6:00-6:30 PM, followed by introductions. Mary's talk runs from 6:45-7:30 PM, with wrap-up and discussion until 8:00 PM. This in-person gathering takes place at Prefect in Washington, providing an excellent opportunity to connect with the local data

Running an AI event in Washington?

List it here, free.

Meetups, workshops and conferences go in front of thousands of builders after a quick review.

List your event