Semantic Search at Scale: What I Learned Building RAG Infrastructure at Microsoft Copilot

I work on Microsoft Copilot's Search Infrastructure team, where I focus on semantic indexing and RAG (Retrieval-Augmented Generation). The challenges of building search at scale are fundamentally different from what you encounter in tutorials. Here's what building production RAG taught me — and how I applied those lessons to my open-source projects. The Tutorial vs. Production Gap Most RAG tutorials show this: # Tutorial RAG chunks = split_document(document) embeddings = embed(...