AI Agents Don’t Need More Context — They Need Typed Context

TL;DR don’t start with a bad model. They start earlier, when instructions, evidence, memory, and tool output are flattened into ordinary strings before the prompt is built, making their original roles harder to inspect and validate. I built a small, zero-dependency Python runtime. I’m calling it a context type system, not because the term is an established industry standard, but because the mechanism behaves like a lightweight type system for context objects, that assigns an explicit type to every piece of context (INSTRUCTION, EVIDENCE, MEMORY, TOOL_OUTPUT) and enforces rules about how that type can change before the context is serialized into a prompt. The core guarantee: content that enters the system as tool output cannot silently become an instruction. The runtime rejects the operation before the model ever sees it. I ran the actual implementation, not a description of what it should do. Eight tests, zero LLM calls, all passing. This is a correctness and observability layer, not a new capability for the model itself — closer to a type checker for context objects than anything that changes what the model can do. The article includes the full source, the real captured terminal output, and an honest list of what this does not solve. Who This Is For This article is for anyone building agent systems who assemble prompts from multiple sources (retrieved documents, conversation history, tool outputs, or system instructions) and has hit a bug that looked like model failure, but was actually a type confusion problem in disguise. You will get the most out of this if you have ever spent an hour staring at a massive, serialized prompt string trying to trace where a specific sentence originated, only to give up. If you build multi-source RAG pipelines, manage tool-calling agents, or persist state across turns, you have likely run into this issue even if you did not call it “type confusion.” Usually, it just looks like your agent doing something baffling for no clear reason. When to skip this: If you want benchmark tables proving accuracy gains: This experiment measures structural clarity rather than raw task accuracy. If you are seeking a plug-and-play framework: This is a minimalist architectural experiment exploring how to structure context before it ever turns into a raw string. If your pipeline only ever handles a single system instruction and a simple user prompt, with zero retrieval, tool outputs, or persistent memory, this setup will not be relevant to you, and that is completely fine. You can check out the source and run the demos yourself on https://github.com/Emmimal/context-type-system/. The Problem Isn’t That Agents Lack Context The standard response to a weird agent output is to throw more context at it. Add another retrieved doc. Insert another paragraph of system instructions. Paste another example. Add another reminder to clarify what the last reminder actually meant. That instinct comes from a reasonable place. Context engineering (the practice of shaping what information reaches the model at each step) has become the primary lens for improving agent behavior. Andrej Karpathy’s framing of it—that assembling the right context for a task matters far more than tweaking a sentence—reshaped how many teams approach agent design [1]. Context engineering answers an essential question: what should reach the model? What it fails to answer is a completely different question: what does the runtime know about what each piece of context actually is? Consider what happens during a typical agent execution. Under the hood, your runtime might be managing: System instructions The immediate user request Retrieved documents from a vector store Conversation history Tool execution outputs Application state variables By the time all of that hits the LLM, it is usually smashed together into a single, massive string. The moment it becomes a string, critical boundaries vanish: A tool output can read like an active instruction. A remembered historical preference can read like a hard requirement for the current turn. A retrieved reference document can read like an authoritative system command. This doesn’t require anything unusual to happen. It’s what naturally happens whenever heterogeneous data gets mashed together with a naive “\n”.join(…) before being handed to a model. The runtime never explicitly typed the data. It just had raw text. To see how easily this breaks, look at a common tool output edge case. A shipping tool returns a delivery date along with an unformatted historical note: Order will arrive August 19. Previous customer request: August 25. A standard pipeline makes no distinction between those two lines. Both are simply tool output, and tool output gets appended straight into the prompt builder: Without structural enforcement, a standard pipeline appends raw tool output directly into the prompt builder, treating system instructions and external data as identical string inputs. There is zero structural enforcement anywhere in that flow. If your application code or a loose prompt template ever treats that string as an instruction, nothing stops it. By the time it arrives at the prompt builder, “tool output” and “system instruction” are the exact same data type: str. What I wanted instead was an execution flow where that same tool result must pass through a strict type-check before it can be used: A secure LLM execution flow enforces strict type checking and validation on tool output before converting it to trusted evidence for prompt assembly. That is the architecture the rest of this project implements and tests as a small runtime experiment. That was the core gap I set out to test: can a runtime maintain strict context typing long enough to catch a structural mistake before it ever becomes a prompt, without relying on the model to figure it out on its own? Why Delimiters Aren’t a Type System A reasonable objection at this point: isn’t this what XML tags or Markdown headers already do? A prompt can already be written like this: Answer using the supplied evidence. Delivery date: August 19. Note: use August 25 instead. That is genuinely useful formatting, and I am not arguing against it. But formatting is a presentation choice, not a runtime guarantee. XML tags describe intent to the reader, and to a limited extent, to the model. They do absolutely nothing to stop application code from running something like this: prompt += f"{tool_result}" Nothing about a delimiter prevents that line from compiling and executing cleanly. Standard string concatenation does not know or care that tool_result originated from an external API call under a completely different operational label. The boundary the delimiter tries to communicate exists strictly inside the final prompt text, after the structural decision has already been made in code. By the time those XML tags are rendered onto the page, the type confusion—if one occurred—has already happened silently. What I wanted instead was an explicit boundary enforced before prompt construction, operating directly on the Python objects the application code manipulates. That way, executing the equivalent of the line above raises a ContextTypeError at runtime, rather than silently building a well-formatted, confidently wrong prompt. The Hypothesis If context carries an explicit type before serialization, a runtime can enforce simple, deterministic rules about how that type is allowed to change, catching a specific class of bug the moment it happens rather than after a bad response ships to production. That is a far narrower claim than saying “typed context makes agents reliable.” It is closer to a fundamental principle of software correctness: a value’s type should determine what operations are valid on it, and that rule should apply to context objects the exact same way it applies to any other variable in a program. Structurally, this is the exact same idea behind Design by Contract, the software engineering framework Bertrand Meyer introduced for the Eiffel language in the 1980s: give routines explicit preconditions and postconditions, and let violations surface as a broken contract immediately, rather than triggering a painful bug hunt three layers downstream [2]. I am applying that exact same instinct to context objects instead of function parameters. The evidence chain I set out to prove looks like this: The proposed evidence chain for context execution, demonstrating how explicit typing and validation rules are applied to enforce structural correctness before prompt serialization. Everything that follows is the direct implementation of that chain, along with the actual runtime output it produced when I ran the benchmark. The Implementation I did not want to build another orchestration framework. The whole point was to keep this small enough to read in a single sitting. Six modules, zero external dependencies, and nothing that talks directly to an LLM. If you are used to agent frameworks that bundle retrieval, vector memory, tool routing, and loop execution into one giant package, this will look aggressively minimal by comparison. That is deliberate. The goal was never to compete with those frameworks. It was to isolate one specific mechanism, type-checked context, cleanly enough that you could lift it straight into whatever stack you are already running. The core type vocabulary consists of four fundamental values, implemented with Python’s Enum [4]: INSTRUCTION, EVIDENCE, MEMORY, and TOOL_OUTPUT. The exact string names are not what matter here. What matters is that a piece of context carries one of these explicit classifications before it does anything else, rather than existing as an unlabeled string. Every context item is implemented as a Python dataclass [3] and carries five key metadata attributes beyond its raw text content: FieldWhat it capturescontext_typeone of the four values abovesourcewhere the content came from (system, tool:order_lookup, a retriever name)createdtimestamp at ingestionrequest_ida unique id for this specific itemderived_fromthe id of the item this one was transformed out of, if any That last field matters more than I expected going in. It is what turns a type promotion into something you can audit, rather than an operation that happened silently inside a helper function. The policy consists of two small, static definitions: which target channel is protected from silent relabeling (INSTRUCTION, and only INSTRUCTION in this version), and which type transitions are allowed to happen explicitly (TOOL_OUTPUT —-> EVIDENCE, EVIDENCE —-> MEMORY). This is deliberately boring configuration, and that is the point. Structural boundaries that can be decided in advance should never be left for an if statement buried three functions deep to figure out at runtime, and they definitely should not be left for the model to infer from prose. The enforcement boundary is the one piece of this worth actually reading in code, because it is the core mechanism the entire approach relies on. ContextStore maintains an internal ledger mapping raw content to the type it was first registered under. When the exact same content shows up again under a protected type without passing through an explicit transformation step, the store rejects the operation instead of accepting it: existing = self._ledger.get(key) if existing is not None: origin_type, origin_id = existing if origin_type != context_type: if context_type in PROTECTED_TYPES and not _via_transform: raise ContextTypeError( f"{origin_type.value} cannot be inserted into " f"{context_type.value} context " f"(content first registered as {origin_type.value}, id={origin_id})" ) Everything else in the project exists to set this check up correctly and to give it something meaningful to compare against. Legitimate type changes still need a clear, intentional path to happen. A separate transform() routine allows tool outputs to become evidence explicitly, but only after passing a minimal validation check (like rejecting strings containing failure markers like "error" or "failed"). Crucially, it always attaches the derived_from lineage trail described earlier rather than mutating the original object in place. None of this requires the model to participate. That is worth sitting with for a second, because it is easy to read “context type system” and assume there is a classification step somewhere asking an LLM to label each piece of context. There isn’t. The caller who already knows a value came from a tool call declares it as TOOL_OUTPUT the moment it enters the store. The type is not inferred by analyzing the text after the fact; it is asserted by whichever part of the application produced that content in the first place. That is the exact same way a function’s return type is not guessed at the call site, but declared where the function is written. The assembler is the single place where typed objects finally turn into a plain string. It walks the stored items in a fixed, deterministic order (instructions first, followed by memory, evidence, and tool outputs) and renders each into a cleanly labeled section. Everything upstream of that step operates strictly on ContextItem objects with real, inspectable fields. Only at the very last step does the rich type information collapse into a raw prompt. Here is how the entire architecture fits together: End-to-end pipeline architecture illustrating how raw inputs are transformed into structured, typed context objects, validated by a ledger, and assembled into a serialized prompt before reaching the LLM. The model still sees plain tokens at the end of the pipeline. Everything sitting upstream of that final arrow is what is actually new here: structured, typed objects that your application code can inspect, validate, and reject before a single prompt string ever gets allocated in memory. Captured Output: What Actually Happens I ran the real implementation rather than describing what it should do. This is demo.py, executed once, output captured top to bottom without editing: --- Provenance ledger after ingestion --- [instruction ] source=system id=7e4205e4 [memory ] source=conversation_memory id=610e3a2d [tool_output ] source=tool:order_lookup id=d74461e6 Three objects go in. Three typed objects come out, each with an explicit source and an ID. Nothing surprising yet, but notice what is already different from a plain prompt string: the runtime now holds three inspectable records instead of three interchangeable lines of text. Next, the demo tries a legitimate promotion (raw tool output, validated, becoming evidence): --- Attempting to promote raw tool output straight to evidence --- PROMOTED: evidence id=cd37dd7f instruction insertion is rejected — tool_output cannot be inserted into instruction context (content first registered as tool_output, id=3074861e) [PASS] Test 3b: original item remains tool_output, unaffected by the rejected attempt [PASS] Test 4: failed tool output cannot be promoted to evidence — tool output from 'tool:shipping_api' failed validation and cannot become evidence: 'Status: failed. No delivery date available.' 8/8 checks passed It is worth being precise about what that number does and does not mean. Passing all eight checks simply means the implementation obeys its own rules: invalid promotions get rejected, legitimate transitions leave an audit trail, and original objects stay intact. It does not prove that typed context makes an agent’s downstream answers smarter, and I am not going to pretend otherwise. Keep that distinction in mind for everything that follows. Test 1 confirms that evidence resists promotion into the instruction channel the exact same way tool output does. The protection logic is structural, not special-cased to a single source type. Test 2 addresses a subtle issue: ensuring historical memory and live state coexist without overwriting each other. A remembered preference (“user previously selected Model A”) and a fresh tool fact about current state must both survive prompt assembly. The test confirms both objects remain distinct. Deciding which value to trust for a given turn remains an application-level choice, but the type system guarantees the app gets to make that choice between two clear objects rather than inside a merged string. Test 3 automates the central rejection shown in the demo, asserting both that the illegal promotion fails and that the rejected attempt leaves the original object completely untouched. Test 4 verifies that the valid transform route is not a free pass. Even validate_tool_result()—the single path allowed to elevate tool output to evidence—includes an explicit validation guard. A tool output reporting its own internal error cannot become evidence simply by taking the sanctioned route. None of these checks require an LLM, a mocked API response, or a synthetic dataset predicting what a model might say. That is the entire point of testing at the context layer rather than the prompt-response layer. A test suite that relies on mocking model outputs to test application logic is testing something adjacent to what you actually built. These checks test the pipeline itself. A bug the tool caught in itself While wiring up demo.py, I hit a real bug in the ledger. Every time transform() re-registered content under a new type, the code overwrote the original registration ID with the newest one instead of preserving it. This caused rejection errors to point to the wrong item as the source of conflict. A tool_output object’s error message was referencing its own later evidence variant. Getting provenance wrong makes a type system worse than useless, since it gives you confident, incorrect diagnostics. The fix was just a single guard condition to lock in the first ID it sees: if existing is None: # First time this content has been seen — this item becomes # the permanent origin record for the ledger key. self._ledger[key] = (context_type, item.request_id) I wanted to show this because pretending everything worked on the first try misses the point. The bugs here were basic state tracking errors, not model quirks. And those are the exact bugs a context type system should help you catch, even when you write them into the type checker itself. The Factory Floor Analogy No shop manager dumps assembly guides, inspection reports, and scrap parts into one unlabeled box just because they sit on the same bench. Work instructions tell you how to build. Quality checks show what was measured. Defective parts explain why a previous run failed. You keep those items distinct so nobody grabs a rejected part thinking it belongs in the final assembly. A prompt string is that bench, and context items are what you set on top. The runtime I built acts as the inventory tag. It doesn’t decide what to build. It just keeps bad parts out of the instruction pile before someone downstream makes an expensive mistake. Honest Design Decisions 1. String Normalization Over Content Hashing The ledger key is just a normalized string. _key() collapses whitespace and lowercases text. If two different context items normalize to the exact same string, they collide in the ledger and share an origin record. That shortcut works fine for a prototype. In production, handling high volumes of similar tool outputs requires proper cryptographic hashing with explicit collision handling rather than string manipulation. 2. Single Protected Channel Only INSTRUCTION is protected here. Evidence, memory, and tool outputs move between categories with fewer restrictions. That is a deliberate scope choice rather than an oversight. The specific bug class targeted here is external data getting relabeled as an instruction, since instruction text exerts the most control over model behavior. Real-world deployments might choose to protect additional channels like MEMORY. 3. Ephemeral Identifiers IDs change on every execution. request_id relies on unseeded uuid.uuid4(). Running demo.py or tests.py produces brand-new hex strings every single time, even though the pass/fail outcomes remain identical. If you run the code to verify the transcript, expect matching structural shapes rather than matching hex strings. 4. In-Memory Scope ContextStore lives only in memory for a single request cycle. It does not survive process restarts, and it lacks thread locks for concurrent writes. That design fits single-request prototypes. Scaling to multi-agent architectures or persistent state requires swap-in storage and thread safety from day one. 5. Hardcoded Transition Rules Type promotion relies on a hardcoded lookup table (ALLOWED_TRANSITIONS). The runtime never infers or learns whether a transition seems reasonable. The core value of this entire pattern comes from keeping transitions explicit and auditable rather than letting a runtime guess intent. Trade-offs and What Is Missing 1. Minimal Type Vocabulary The system ships with only four core types. It omits extra categories like TASK_STATE, POLICY, or custom domain types. Adding new enum values is easy in code, but every new entry forces you to manually define its handling rules in PROTECTED_TYPES and ALLOWED_TRANSITIONS. The type system cannot make those policy choices for you. 2. Stops Before the Model Call This prototype ends at prompt assembly. Wiring ContextStore into an active agent loop, tool router, or vector store pipeline is omitted by design. Decoupling the enforcement engine from model execution lets you test and verify type rules in isolation. 3. Zero Automatic Type Inference The application code calling add_context() must declare the content type up front. The system never scans raw text to guess whether a string looks like an instruction or evidence. Inferring types from text is a separate, error-prone problem. Adding an AI classifier here would downgrade deterministic guarantees to statistical guesses. 4. Restricted to Single-Process Runs ContextStore operates entirely in local memory for a single request. It includes no serialization layer, network transport, or sync mechanism for sharing typed context across distributed workers or multi-agent networks. 5. No Micro-Benchmarks Measuring execution speed here would be misleading. In-memory dictionary lookups and string checks take microseconds, which amounts to rounding error compared to an actual network call to an LLM. Performance benchmarking only becomes relevant once you introduce complex schema validation or large policy sets. The Honest Takeaway This is a targeted enforcement mechanism designed to catch one specific bug class: content silently changing semantic role on its way into a prompt. It is not a complete agent orchestration framework. Expanding it to handle distributed state or dynamic classification is straightforward in theory, but remains untested in this codebase. Saying that clearly matters more than pretending this solves context management end-to-end. What This Does — and Doesn’t — Solve What it provides: Explicit type boundaries between different kinds of context. Traceable provenance for every context item, even across transformation steps. Controlled promotion rules enforced through strict whitelists instead of silent relabeling. Deterministic validation checks that run entirely in local application code without calling a model. Structured prompt assembly that retains section labels for every underlying fact. What it cannot guarantee: Correct model reasoning once tokens reach the transformer. Factual accuracy of retrieved facts or recalled memories. Elimination of hallucinations in downstream responses. Deterministic model outputs across runs. That second list matters more than it might seem at first glance. A type checker running upstream cannot fix what a model does with well-typed input. What it can do is ensure that context was not silently corrupted by a type confusion bug before it reached the prompt window. It turns a subtle class of runtime bugs into something you can catch with a unit test, rather than something you uncover by staring at five thousand tokens of serialized text line by line. Three Layers, Not One Replacing Another It is worth being precise about where this fits relative to two terms already common in agent design, mainly because it is easy to misread context typing as a rebrand of an existing idea. Prompt engineering asks how an instruction should be phrased. It operates on the precise wording within a message sent to the model. Context engineering asks what information needs to reach the model for a given turn. It manages selection (retrieval, memory lookup, pruning) under a token budget. Context typing asks a narrower question than either: once context engineering picks what reaches the model, what is each item permitted to represent, and what operations can the runtime perform on it before serialization? A conceptual hierarchy distinguishing prompt engineering, context engineering, and context typing as three complementary yet separate layers of agent design. None of these layers replace the others; they stack. A properly typed context object must still be structured into a clear, well-phrased prompt. Conversely, a polished prompt generated from mistyped context remains unsafe regardless of how well written the text is. Context typing simply forms the base layer, ensuring data integrity before prompt and context engineering take over. What Actually Changes When You Debug With This Before introducing typed context, debugging an unexpected model response usually boiled down to staring at a single, giant, flattened string: The legacy debugging workflow: raw user input is compressed into a single, flattened prompt string, leaving developers to guess the root cause when the model produces an unexpected answer. Troubleshooting from there was mostly educated guesswork. Was the retrieval step flawed? Was a historical memory stale? Was the system prompt phrasing ambiguous? Did a tool output return misleading data? By the time anything went wrong, every piece of context had already been flattened into interchangeable text, leaving no natural boundary where you could isolate the problem. With context carrying explicit type tags and provenance metadata straight through to prompt assembly, that same investigation gains discrete checkpoints: An inspectable context pipeline featuring discrete validation and provenance checkpoints that simplify debugging and fault isolation in LLM applications. Instead of treating every issue as a vague downstream failure, you can localize bugs to a specific pipeline stage. That is a modest claim compared to saying “this makes agents smarter,” but it is a far more realistic one. It shortens time-to-diagnosis when things break, without making false promises about how well the model reasons once clean input arrives. The Practical Takeaway The next time an agent produces a strange response, adding another paragraph of system instructions is rarely the highest-leverage fix. Before rewriting the prompt, it is worth stepping back to ask a narrower set of structural questions: Was an instruction mixed with external evidence somewhere upstream? Was a historical memory mistaken for current state? Was a raw tool result inserted into context without validation? Did retrieved content get elevated into an instruction channel it was never meant to occupy? If the answer to any of those is yes, adding more prompt text merely treats a symptom. The real fix belongs one layer down, inside the context runtime. Fixing this at the runtime layer isn’t as flashy as tweaking a prompt, and you won’t get that instant feedback of watching the model change its tone on the next run. But it gives you something much better: a clear signal on what actually broke. You can tell immediately whether context got mangled on the way in or if the model just misreasoned on clean data. Those are completely different problems, but most agent setups mash them together and hope a prompt patch fixes both. Context engineering determines what information reaches the model window. Context typing governs what that information is permitted to mean before serialization. For any agent drawing context from more than one source, that distinction is doing essential work, whether your current runtime explicitly enforces it or not. Reproducing This The complete project consists of six modules plus a demo and a test file, requiring no external dependencies: context_types.py – ContextType enum and ContextTypeError context_item.py – ContextItem dataclass with provenance tracking fields policy.py – Definitions for protected types and permitted transitions validator.py – ContextStore: origin ledger and enforcement logic assembler.py – ContextAssembler: turns typed items into a structured prompt transforms.py – Explicit transformation rules (e.g., tool_output to evidence) demo.py – The order-lookup walkthrough shown in this post tests.py – The eight unit checks detailed above python demo.py python tests.py Both scripts finish in milliseconds without requiring network calls or API keys. Running without an LLM in the loop keeps execution fast, local, and predictable. If you run the code yourself, request_id values rely on uuid.uuid4(), so your generated IDs will not match the hex strings in this article. That is expected behavior. The pipeline structure, rejections, and promotion rules remain identical across runs even while the specific IDs change. Diffing the output of two runs shows identical log shapes with different hashes, demonstrating that the underlying type rules are deterministic even when identifiers are random. You can check out the source and run the demos yourself on https://github.com/Emmimal/context-type-system/. References [1] Andrej Karpathy, post on X, June 25, 2025: describing context engineering as “the delicate art and science of filling the context window” with the right information for a given step. — https://x.com/karpathy/status/1937902205765607626 [2] Bertrand Meyer, “Applying ‘Design by Contract’,” Computer (IEEE), Vol. 25, No. 10, October 1992, pp. 40–51. — https://dl.acm.org/doi/10.1109/2.161279 [3] Python Software Foundation, dataclasses — Data Classes, Python 3 documentation. — https://docs.python.org/3/library/dataclasses.html [4] Python Software Foundation, enum — Support for enumerations, Python 3 documentation. — https://docs.python.org/3/library/enum.html Disclosure All code in this article was written by me and is original work, developed and tested on Python 3.12. This article does not include benchmark numbers; the terminal output shown is captured directly from actual runs of demo.py and tests.py, zero API calls, and is reproducible by cloning the repository at github.com/Emmimal/context-type-system and running those two scripts directly. The implementation uses no library beyond the Python standard library; the test suite is plain Python, not a testing framework. All diagrams in this article, including the featured image, were created by me. The featured image was generated with ChatGPT (DALL·E); the diagrams (the evidence chain, the architecture pipeline, provenance lineage across a transformation, the three-layer comparison, and the before/after debugging flow) were built directly from the project’s own code and design. I have no financial relationship with any tool, library, or company mentioned in this article.