LSM Trees: Why Your Database Writes Are Fast and Your Reads Are Lying to You

TL;DR: The thing that broke my comfortable ignorance about storage engines was a pipeline ingesting sensor telemetry — about 50,000 inserts per second into a PostgreSQL 15 cluster. The hardware wasn't cheap: NVMe drives, 32 cores, 128GB RAM. 📖 Reading time: ~42 min What's in this article The Problem That Made Me Actually Care About Storage Engines What an LSM Tree Actually Does (Without the Textbook Nonsense) The Write Path Step by Step The Read Path — Why It's More Expensive Th...