I spent a week fixing my chatbot's memory — here's what worked

Two months ago, I shipped a customer support chatbot for my SaaS product. It worked great for the first three messages. Then it started forgetting what the user said earlier, repeating itself, and giving contradictory advice. Users noticed. One wrote: "Your bot has the memory of a goldfish." I had hit the classic LLM context window wall. My initial implementation just stuffed the entire conversation history into the prompt. That worked until conversations grew beyond 4k tokens. Then I tried tru...