An LLM is not an AI agent. An LLM predicts useful next tokens. An agent is the system around it: state, retrieval, tools, permissions, retries, and observability. Treat the model as a decision engine—not your whole application.

This piece delves into the nuanced distinction between large language models (LLMs) and AI agents. It clarifies that while LLMs excel at predicting useful next tokens, they lack the comprehensive systems, tools, and permissions that define an AI agent. The article emphasizes treating LLMs as decision engines rather than standalone applications, stressing the importance of surrounding them with the right state, retrieval mechanisms, and observability for effective use. This distinction matters because it helps developers avoid overestimating the capabilities of LLMs and instead focus on building robust, integrated systems around them.