Context Engineering for Agentic Workflows: Task-Aware Token Budget Forecasting

The first time I looked closely at a running agent session, I was not worried about accuracy. Accuracy was fine. What bothered me was the shape of the token curve.The agent was doing something ordinary: answering a data question by planning a query, running it, checking the result, and correcting itself when the result looked wrong. Nine model calls in total. By call six, most of what we were sending to the model was history the model no longer needed. Schema definitions it had already used. Tool output it had already summarized. Instructions it had already followed twice. We were paying full price on every call to re-send a conversation the agent had mentally closed several steps earlier. I run platform engineering for AI and data infrastructure, so my instinct was not to go fix that one agent. My instinct was to assume every agent we onboarded would do the same thing, and that none of the teams building them would notice until the invoice arrived. That turned out to be right. What came out of it is the thing I want to describe here: a context engineering layer that sits underneath agents rather than inside them, and a habit of forecasting a token budget per task before the agent runs instead of discovering the cost afterward. Why I stopped treating this as an agent problem The obvious fix is to tell each team to manage their own context. Trim your history. Summarize your tool outputs. Do not paste the whole schema. I tried that. It does not hold. It does not hold because context management is invisible when it works and painful when it does not, so it always loses to whatever feature the team is shipping this week. It also does not hold because every team solves it slightly differently, and a platform with forty different context strategies is a platform with no context strategy. When a model version changes, or pricing changes, or someone finds a better summarization approach, you want one place to change it. Not forty. So I moved it down a layer. The agent decides what it wants to do. Something below the agent decides how much of the world it gets to see while doing it. That split matters more than any individual technique I will describe below. If context handling lives inside agent logic, improving it means asking every team to refactor. If it lives underneath, improving it means shipping a version bump. Token spend is more predictable than it looks Here is the observation the whole approach rests on. Agent tasks are not unique snowflakes. They fall into a small number of classes, and within a class, the context they need is fairly stable. A question that requires looking at one table and returning a number needs schema for that table, a couple of examples, and almost nothing else. A failure investigation across a build system needs logs, and logs are long, but the useful part is usually a small window around the first error. A multi step operational workflow needs procedure documentation plus current state, and the procedure documentation barely changes week to week. Once you can name the class, you can forecast the budget. Not exactly. Roughly. Roughly is enough. In practice the forecast is simple. For a given task class I estimate how much of the window should go to instructions, how much to retrieved knowledge, how much to conversation history, and how much I need to reserve for the model's own output. Those become caps, not suggestions. The layer's job is to fit the actual context into those caps before the call goes out. This inverts the usual order of operations. Most agent code assembles whatever context it has and sends it, then someone looks at the bill later. Budget forecasting decides the ceiling first and then fits the content to it. That single reversal did more for our cost curve than any clever compression trick. It also does something less obvious. When you cap the history budget, you find out fast which agents are hoarding history because they need it and which ones are hoarding it because nobody wrote the code to stop. The second group is much larger than I expected. The four layers, in the order they run The framework has four stages. They run in a specific sequence, and the sequence is doing real work. Selection comes first Selection answers a narrow question: of everything available, what is even a candidate for this call? This is the cheapest stage and the one with the most leverage, because anything selection removes never has to be compressed, ranked, or clustered. If the task is scoped to one data domain, the other domains never enter the pipeline. If the current step is a validation step, the planning instructions from three steps ago do not come along. I think of selection as the bouncer. It is not smart. It works off task class, scope, and recency. It just needs to be right about the obvious cases, and the obvious cases are most of them. Reranking sorts what survived Everything that made it past selection is now competing for a limited budget. Reranking scores each piece by how relevant it is to the current step rather than to the overall task. The distinction between step relevance and task relevance is where most naive retrieval goes wrong. An agent halfway through a task is not still working on the original request. It is working on a sub-goal that may have very little to do with the words in the user's original message. Rank against the sub-goal, and the top results change substantially. Reranking is also where I get honest about a tradeoff. A good reranker costs something to run, sometimes a small model call of its own. If the budget you are protecting is small, reranking can cost more than it saves. I gate it by task class for exactly that reason. Short tasks skip it. Clustering removes the duplicates By this point, the surviving candidates are relevant, and a surprising number of them are relevant in the same way. Retrieval systems love near duplicates. Three chunks of documentation describing the same procedure with slightly different wording. Four log lines that are the same error repeated. Two summaries of the same tool output written at different points in the run. Clustering groups things that carry the same information and keeps one representative from each group. It is quiet, unglamorous, and it consistently reclaimed more budget than I expected. Redundancy in agent context is not an occasional accident. It is the default state, because agents accumulate context by appending and nothing ever cleans up behind them. Compression fits what remains Only now do I compress, and only what is still standing. Compression is last on purpose. Summarizing something you were about to throw away is wasted effort, and it is worse than wasted if the summary loses a detail you needed. Running it last means everything it touches has already earned its place. The important design choice here is that compression is not uniform. Old conversation turns can be summarized aggressively, because what matters is the decision that was reached, not the wording. Tool output can be compressed structurally, keeping the shape and dropping the volume. Current instructions and anything the agent is actively reasoning over do not get compressed at all. Compressing the wrong thing is how you build an agent that becomes vaguely worse over long sessions in ways nobody can reproduce. On a real multi step session, the four layers together removed roughly three quarters of the input tokens per call, with no change to the agent's logic and no change to its answers. The savings were not evenly distributed across the layers, and they were not evenly distributed across calls either. Early calls barely changed. Late calls, the ones dragging the most history, changed enormously. Why a drop-in layer, and what it costs The framework attaches to an agent without the agent knowing. Same interfaces, same call patterns. A team turns it on with configuration rather than a rewrite. I want to be straightforward about the tradeoffs, because a drop-in design is not free. The layer knows less than the agent does. It works off task class and signals rather than genuine understanding of intent, so it will occasionally drop something a smarter, agent-aware implementation would have kept. I accepted that. A framework that every team gets for free and improves centrally beats a better framework that three teams implement well and the rest never adopt. Debugging gets harder. When an agent behaves oddly, there is now a layer between what the agent thinks it sent and what actually went out. This is a genuine problem and the only real answer is visibility. Every stage logs what it dropped and why. Without that, the layer becomes a black box people work around instead of a tool people trust. Budget caps can bite. If a task genuinely needs more context than its class allows, capping it produces a worse answer rather than a more expensive one. Failing loudly matters here. When the layer cannot fit context into budget, I would much rather it says so than silently truncates and lets the agent produce a confidently wrong result. Where the paved path comes in None of this matters if teams have to wire it up themselves. Alongside the context framework, we built a self-service path for starting an agent. A developer fills out a short form in an internal portal and gets back a repository that is already wired: agent scaffolding, local memory, tracing, evaluation tracking, and the context layer, configured and connected. What used to be several days of setup, most of it copied from whoever built an agent most recently, became a single request. The design decision I care most about there is that the generated repository runs locally first. Not in a shared development cluster. On the developer's machine, with local memory and local tracing, before any production resource is involved. This was contested internally. Local-first means maintaining local equivalents of things that exist as managed services in production, and the two drift if you are not careful. The argument that won was about iteration speed. An agent developer needs to run the same task twenty times in an hour while tuning a prompt. If each run requires a deployment, they will run it three times and guess at the rest. The quality difference between twenty iterations and three is not subtle. Promotion to production is a separate automated step that provisions the real resources and wires up identity based access, so nothing is holding static credentials. But by the time an agent gets there, it has already been tested, traced, and evaluated by the person who wrote it. The context framework benefits from this arrangement in a way I did not anticipate. Because tracing and evaluation are in the generated repository from the first commit, every agent produces token and quality telemetry by default. That telemetry is what lets me tune budget forecasts per task class using evidence instead of intuition. The paved path and the context layer ended up feeding each other. What I would tell someone starting this Measure a real multi step session before you build anything. Not a single call. A full task with tool use and self correction, because that is where the waste hides. The shape of that curve will tell you which layer to build first, and it may not be the one you expected. Name your task classes early, even if the names are rough. You cannot forecast a budget for a category you have not defined, and the categories are usually already visible in what teams are asking for. Put it underneath the agents, not inside them. This is the decision that determines whether you are running a platform or maintaining forty variations of the same idea. And log every drop. The first time someone accuses your layer of breaking their agent, you will either have the answer in thirty seconds or you will spend a week defending something you cannot see into. I have done both.