PostHog Launches Automatic Clustering to Surface Patterns in LLM Traces.

PostHog has released automatic clustering for LLM traces, a new feature that analyzes large volumes of AI interactions to surface usage patterns, outliers, and problem areas without manual inspection.

The open-source analytics platform processes over 2.5 million traces per week from its own AI-powered features. That volume makes manual review impossible, a challenge facing any company building AI products at scale. The new clustering pipeline ingests traces as PostHog events, converts them to readable text, samples them hourly, and uses LLM-powered summarization to extract meaning before clustering the results.

The system combines several techniques to turn raw trace data into actionable insights. After ingesting traces and generations as events, PostHog converts each trace to a readable text format and samples them every hour. Those samples go through structured LLM summarization to generate embedding vectors.

The clustering itself relies on UMAP for dimensionality reduction and HDBSCAN for identifying groups. An AI agent then names and describes each cluster. The results appear in a new clusters tab within PostHog, displaying both scatter plots and distribution charts that show how users actually interact with AI features.

This matters because usage metrics alone don't tell you whether an AI feature works. Someone might trigger your AI chatbot dozens of times, but if the responses are incorrect or unhelpful, the feature is failing despite high engagement numbers.

PostHog previously launched LLM analytics for tracing AI agent runs, giving teams visibility into individual interactions. Clustering extends that capability by analyzing traces in aggregate, identifying patterns that only emerge when looking at thousands of interactions together.

The company detailed the full technical implementation in a blog post by engineer Andy Vandervell, covering everything from the embedding model choices to the clustering algorithm parameters. For teams already using PostHog, the feature requires no additional setup, since all trace data already flows through the platform's event pipeline.

The release addresses a gap in the AI development toolchain. Most observability platforms focus on logs and performance metrics, useful for debugging but less helpful for understanding whether an AI feature delivers value. Clustering helps product teams answer questions like which conversation patterns lead to successful outcomes, where users get stuck, and which edge cases appear most frequently.

PostHog's decision to build this into its core platform rather than spinning up a separate AI observability tool reflects a broader trend. As AI features become standard in software products, the line between traditional product analytics and AI-specific monitoring blurs. Teams need both the quantitative metrics (how often is this used) and the qualitative understanding (is it actually working) in one place.

The feature is available now to PostHog users ingesting LLM traces through the platform's event tracking.