LLM Evaluation in Production: Building the Eval Pipeline That Runs on Every Deploy

Everyone ships the RAG system. Almost nobody ships the eval system that tells them when the RAG system starts lying. You updated the embedding model. Tweaked the system prompt. Swapped the re-ranker. Metrics look fine. Three weeks later, support tickets arrive — the system is drawing inferences the source documents never made. No alarm fired. No test failed. The system drifted silently. This is not a model quality problem. It is an evaluation infrastructure problem. The Four Metri...