RAG Isn't an Agent — I Built the Layer Between Retrieval and Action

TL;DRI built the whole thing in pure Python and ran the tests myself, with the results shown as pass or fail.RAG can find information. An agent can take action. I wanted to see what actually happens when you put them together instead of assuming both are needed.I built three versions: retrieval only, a deterministic action planner, and a hybrid system that connects retrieval with actions. Then I ran the same nine tasks through all three.I found two real bugs while building it. I show both bugs, along with the terminal output before and after fixing them.This is not just a conceptual comparison. It is a small working implementation with measured results, plus a separate ground-truth check that caught a parser bug before I ran the final tests.The Assumption I Kept Running IntoI first built a RAG system to answer questions from my own documentation. The knowledge part worked. Then I wanted it to change something after finding the answer.It couldn't.Take a support ticket. Someone submits a ticket and it needs a category. The correct category is based on information in internal documentation. A RAG system can find that information, but it doesn't automatically update the ticket.An action system has the other half. It can update the ticket, but it needs to know what category to use.This was the part I wanted to test. What happens when I keep retrieval and action separate? What happens when I connect them?I kept coming across the term "agentic RAG" while working on this. In some examples, the system retrieved documents and generated an answer, but there was no real action after retrieval.I also found systems called agents because they could call tools and change records. But the information needed to make those changes was already given to them in the prompt.I didn't want to argue about which definition was right. I wanted to run the systems and see what happened.So I built three versions:A retrieval-only system.A deterministic action-only planner.A hybrid system that could retrieve information and use it to choose an action.Then I gave all three the same nine tasks.I also decided not to use an agent framework. I wanted to see the retrieval, decision, and action code myself. That meant standard Python only. No external LLM, no vector database, and no embedding API.This also made debugging easier. When something failed, I could trace it back to code I had written.All the results below come from actual runs using Python 3.12, CPU-only and the standard library. I mark anything that was calculated separately rather than produced by a run.Complete code: https://github.com/Emmimal/rag-vs-agent/What This Actually TestsBefore getting into the tests, here is what I mean by each term.RAG (Retrieval-Augmented Generation) looks for relevant information in an external source and uses that information to answer a question [1].In this experiment, that means finding the information needed for a task. It does not update a ticket, change a status, or modify a record.An agent, in the way I use the term here, can take an action. It might update a ticket, assign an owner, or change a status. But being able to take an action does not mean it knows which action to take. It still needs the information required to make that decision.RAG + Agent is where those two parts are connected. The system retrieves information and then passes that information to the part that can take an action.That connection is what I wanted to test.Does retrieval actually help the action system make the right decision? Does the action system work without retrieval? And what happens when I remove one of the two?The tests below are meant to answer those questions with the same tasks and the same data, rather than assuming that combining the two automatically makes the system better.Full System ArchitectureHow RAG, standalone agents, and a hybrid RAG agent use the same task through different retrieval and action paths.Comparison of three AI system architectures: a RAG system that retrieves information from a 334-chunk corpus but cannot access the environment, a standalone agent that can act on a ticket store but has no corpus access, and a hybrid RAG plus agent system that passes retrieved context directly to the AgentPlanner before taking action.I split the project into five files. Each file has one main job.retrieval.py contains the document corpus and the retrieval code. Nothing else.environment.py contains the ticket data and the current state of each ticket.agent_system.py handles the action logic. It can use the environment, but it does not import the retriever.rag_system.py handles retrieval. It imports the retriever, but it does not know anything about the ticket environment.Then there is hybrid_system.py. This is the only file that imports both sides.That separation matters for the test. In the RAG system, retrieval happens without access to the environment. In the agent system, actions happen without access to retrieval. The hybrid system is where I connect the two.So when the hybrid system passes a task, I can see exactly where the retrieved information is being used to make an action decision.Component 1: The RetrieverI kept the retriever simple. There are no embeddings and no approximate nearest-neighbor index. I used term frequency-inverse document frequency with cosine similarity [2].The final corpus has 334 chunks from 22 of my articles, with 59,406 words covering RAG, AI agents, data science, and Python.On my machine, one retrieval query took about 1.4 ms on average. I measured this from the actual runs rather than calculating it from the corpus size. The performance details are in the section below.I first tried a couple of queries to make sure the retriever was finding the right articles.QueryResultScoreArticle"How should I handle missing values?"Correct0.582How to Handle Missing Values in Data Science"how do agents coordinate tasks"Correct0.482How to Design and Implement a Multi-Agent SystemBoth queries returned the article I expected as the top result.That was enough for this first check. I wasn't trying to prove that the retriever was good with two queries. I just wanted to make sure it was actually finding relevant content before using it in the larger test.The harder cases came later. The correct article was not always the top result, and that mattered once the retrieved information was being used for an action.The tokenization is basic too. I convert everything to lowercase, remove characters that are not letters, remove a small list of stopwords, and ignore words shorter than three characters.There is no stemming or lemmatization.I wanted to keep this part easy to inspect. If retrieval gives me a bad result later, I can check exactly what happened instead of looking through a large retrieval stack.One of these simple choices did cause a problem later.Component 2: The EnvironmentThe environment is just the ticket store in this experiment.I started with four tickets and four actions:change_prioritychange_statusassign_ticketset_categoryFor example, this is the priority update:There isn't much hidden here. It checks that the ticket exists, checks that the new priority is valid, and then changes the value.The other three actions follow the same idea. A valid call changes the ticket and returns True. A bad call returns False.I also keep a small action log. For every call I store the action name, whether it worked, and a short description.I needed that because the agent's output is not the final answer for these tests.If it says:Action succeededI still check the ticket.That distinction turned out to matter. One of the bugs I found later looked fine from the system's own output. The ticket state showed something different.That is why the final checks use both the action log and the actual ticket data.Component 3: The Agent PlannerI kept the agent planner intentionally dead-simple—no ReAct-style loops [3], no LLM-driven tool calling. It doesn't use an LLM to decide actions at all. It just regexes the task description for a ticket ID and fields like status or assignee, then hits the environment directly. Keeping it that predictable was intentional. If an LLM is steering the actions, debugging a failure becomes a nightmare: is it a bad prompt, a weird model choice, a regex parsing error, or a broken environment hook? By keeping the decision logic strictly in code, I eliminated those moving parts.There’s a strict boundary here, too: the planner doesn't import the retriever, nor does it touch the article corpus. If a task requires external documentation, the agent can't go looking for it—it has to rely purely on what's explicitly handed to it in the prompt.I needed this restriction to make the upcoming system comparisons fair. The agent-only setup has to genuinely operate without retrieval so I can isolate how well the hybrid version performs when retrieval is actually bridged to the action layer. Plus, it keeps agent_system.py clean and auditable at a glance; I can open the file and immediately see its exact boundaries.Component 4: RAG-Only and Agent-OnlyThe RAG setup is strictly limited to the retriever—it doesn’t even import the environment module. If someone hands it a task that's clearly an action request, it won't just fail silently; it explicitly pushes back:The standalone agent is the exact opposite. It has full run-access to the environment, but zero visibility into the article corpus. If you ask it a pure knowledge question, it bombs out immediately, which is intentional rather than an accident.Component 5: The Hybrid ConnectorWiring the retriever and planner together.I didn't want any hidden magic here, so the hybrid loop explicitly calls search first, then hands the resulting chunks down to the planner argument:Planner itself remains completely blind to the corpus. It can only see whatever retrieved_context gets passed in by the hybrid runner.To map those chunks into a valid ticket category, I just look at the chunk's group tag and tally them up:If 3 chunks hit rag and 2 hit agents, rag wins.Zero LLM involvement. No fuzzy text matching either. Just simple counting over a dictionary map.I built it this way so I could log and inspect what retrieval handed over before the action gets triggered—which saved me when debugging why certain tasks blew up.The Nine TasksSplit into three groups of three. I froze the expected final states before executing a single line, meaning I defined the exact ticket fields required for a pass beforehand, not just vague prompts.CategoryExample taskWhat it requiresKnowledge-only (A1–A3)"What are the main chunking strategies discussed in the RAG material?"Corpus answer, zero ticket changesAction-only (B1–B3)"Assign T101 to Alice and set its priority to high."Two field updates, no doc lookupsKnowledge + action (C1–C3)"T101 is returning irrelevant chunks. Determine the appropriate category from the knowledge base and update the ticket."Search docs first, then write out the resolved categoryEvery single task spins up on a fresh clone of the environment. No state leakage from previous runs messing up later scores.What Happens When You Actually Run It: Bug OneThe initial hybrid planner had a glaring blind spot: it scanned for a ticket ID right out of the gate. If it didn't find one, it bailed out instantly—even if retrieval had already pulled back something useful. For pure knowledge questions, that logic was completely inverted.Here is how the first run looked:SystemRetrievedAction requiredResultRAGCorrect chunks found—PASSHybridCorrect chunks foundAssumed yes (no check)FAIL — "no ticket id found in task"Plain RAG nailed it. Meanwhile, the hybrid setup—which technically had more context and capabilities: completely choked, simply because I'd hardcoded the planner to assume every prompt was a ticket operation. Merging retrieval into a planner actually made it worse at something the standalone retriever could already handle easily.The fix was simple: check if an action is even needed before running the workflow.After patching that logic:SystemRetrievedAction requiredResultHybridCorrect chunks foundNoPASSThe takeaway here isn't just "write a conditional check." It’s a reminder that awkwardly welding retrieval onto a planner doesn't automatically create a superior system, it often just introduces a brand-new failure mode that neither component suffered from on its own.Bug TwoTwo action-only tasks failed next.At first, I thought I had found another problem with the way the systems were connected. Both the standalone agent and the hybrid system failed on:"Assign T101 to Alice and set its priority to high."I checked the environment. The priority had changed, but Alice had not been assigned.Then I checked the parser.My assignment regex was looking for text in the form "assign to Alice." But the task said "assign T101 to Alice." The ticket ID was between assign and to, so the regex did not match it.What made this more confusing was the result reported by the planner:CheckResultSelf-reportedPASSActual ticket stateassigned_to: None (expected Alice)VerdictMISMATCH — self-report disagrees with ground truthThe planner had reported success because the actions it did run had succeeded. It wasn't checking whether it had found every instruction in the task.I found the same kind of problem with another task. That one used "in_progress status" where my parser was expecting "status to in_progress."I changed the two regexes to handle the wording used in the tasks:I didn't consider the two failing tasks enough evidence that the fix was done.After changing the parser, I reset the environment and ran all nine tasks through all three systems again. That way the final results came from the same version of the code rather than mixing runs from before and after the fix.Running It YourselfIf you want to run the experiment, there isn't much setup.From the project folder:The script runs the nine tasks against the three systems. After each run, the result is compared with the expected ticket state.I don't use the system's own response as the final result. The ticket state is checked separately.There is also a smaller script:It runs three tasks, one from each category, so you can see how the experiment works without going through all 27 executions.The project uses Python's standard library. There is no pip install, no API key, and no network call needed once the corpus has been created.Measuring What It Actually Buys YouHere is the final run. All 27 executions were checked against the expected ticket state.IDCategoryRAGAgentHybridA1–A3KnowledgePASSFAILPASSB1–B3ActionFAILPASSPASSC1–C3Knowledge + actionFAILFAILPASSAfter fixing the two parser problems, I ran the full set again from a clean environment. This time the system's reported result matched the actual ticket state for all 27 executions.The RAG system passes the knowledge tasks, but it cannot pass the action tasks in this implementation. There is simply no ticket-changing code in rag_system.py.The agent has the opposite result. It can change the ticket, but it does not have access to the documentation, so the knowledge tasks fail.The combined tasks are where the difference shows up. Both pieces are needed there. The hybrid system is the only one that has access to the retrieved information and the ticket actions in the same execution.That is what I wanted the experiment to show. Not that RAG is better than an agent, or that adding an agent automatically improves RAG. The result depends on what the task actually requires.What Surprised Me: Top-1 Retrieval Wasn't EnoughI expected the knowledge + action tasks to be the difficult ones. They were, but there was something else going on in the retrieval results.The task itself contains the words "knowledge base" and "appropriate category." Those words also appear a lot in my RAG articles. So the retriever sometimes paid more attention to the instruction than to the actual ticket problem.Here are the top three results I got:TaskTicket topicTop-3 retrievedCategory resolvedC1RAG/chunking issueRAG, RAG, Missing ValuesretrievalC2Missing-values issueRAG (noise), Missing Values, Exploratory Data Analysisdata-qualityC3Exception-handling issueRAG (noise), Python Modules, Exception Handlingcode-qualityC2 and C3 are the interesting ones.The first result was wrong in both cases. The same RAG article, "What is Retrieval-Augmented Generation?", came back because of the wording around the task.The second and third results were more useful. For C2, two of the three results pointed to the data-quality group. For C3, two pointed to code-quality.The planner uses all three results when resolving the category, so the wrong first result didn't change the final category.I had already written the category resolver this way before running these tasks. I wasn't expecting these particular retrieval results and then adding majority voting to make them pass.I also don't want to make a bigger claim from three tasks. This only shows what happened with this corpus and these queries.The reason for the bad top result is fairly easy to see after looking at the queries. Each knowledge + action task contains a phrase about determining the category from the knowledge base. My corpus contains a lot of RAG material, so terms such as "knowledge base" and "retrieval" have a lot of weight.The retriever doesn't know that those words belong to the instruction around the task. It sees the complete text as a query and counts the matching terms.That means the wording of the task itself can affect retrieval.In a larger system, I would want to deal with that explicitly. One option would be to remove the instruction part before retrieval. Another would be to avoid treating the first retrieved result as sufficient evidence for an action.In this experiment, the top-three majority was enough to handle the two noisy results.Performance CharacteristicsI measured the main parts of the system on my machine.The test was run with Python 3.12.3, CPU only, using the 334-chunk corpus. Each number below is the mean from 200 runs.OperationLatencyNotesRetriever init (load + build TF-IDF index)~29.4 msIncludes loading the chunks and building the indexRetrieval query (index already built)~1.4 msOne search() callEnvironment action~0.001 msDictionary lookup and assignmentAgent planner (regex parse + 2 actions)~0.009 msDoes not use the corpusRAG system full run~1.3 msRetrieval onlyHybrid full run, index reused~1.5 msRetrieval + category resolution + ticket updateThe number that caught my attention was the retriever startup time.A new HybridSystem takes roughly 31 ms on this corpus. Most of that is the retriever loading the chunks and building the TF-IDF index. Once the index has been built, another retrieval call is only about 1.4 ms.The experiment creates a new system for every task. I did that so every execution starts with the tickets in the same state. Otherwise, an earlier task could change a ticket and affect the result of a later task.That also means the index gets rebuilt during the test runs.For a normal application, there is no reason to do that for every request. The index can stay loaded and the same retriever can handle subsequent queries.So there are really two different timings here. Starting a new hybrid system is around 31 ms on my machine. Running the hybrid system with its index already loaded is around 1.5 ms.These numbers are from this implementation and this corpus. I am not treating them as a general benchmark for RAG or agent systems. The point here is simply to show what this particular implementation costs to run.Limitations and What I'd ChangeThere are a few things I would change before using this outside the experiment.TF-IDF instead of semantic retrieval. The retriever only works with the words it sees. For example, "unhandled exception" and "uncaught error" don't have much connection from its point of view. The noisy top-1 results in C2 and C3 are another example of this.I used TF-IDF because I wanted to keep the retrieval code small and readable. There is no embedding model hiding behind it. If I changed this part, I would probably try a small embedding model next. That would also mean giving up the zero-dependency setup used in this experiment.Regex parsing. The planner is also limited by the text patterns I wrote."Assign T101 to Alice" works after the parser fix. Something like "Please give this one to Alice" would not.I only tested a small number of sentence patterns, so I don't know how the parser would behave with wording I haven't included. A structured-output model could handle much more varied input, but then another part of the system would become harder to inspect when something went wrong.No confidence check for category selection. At the moment, the category resolver just uses the most common category in the retrieved results.A 2-1 split and a 3-0 split both produce an answer. There is no extra check for how strong that answer is.If I were taking this further, I would want the system to flag uncertain cases instead of changing a ticket when the retrieved evidence is weak.The test set is small. There are nine tasks, one corpus, and all of the documents came from my own articles.That was enough to expose the separation between retrieval and action, and it also exposed two implementation bugs. It isn't enough to make broader claims about how these systems would behave on a much larger set of tasks.The majority vote worked for the noisy results I saw in C2 and C3. I wouldn't assume the same thing would happen with a different corpus or much noisier retrieval results.ClosingThe experiment ended up being simpler than the terminology around it.The RAG system could find the information, but it couldn't change the ticket. The agent could change the ticket, but it didn't have the information needed for the knowledge-based tasks.The hybrid system had both.The C2 and C3 tasks made this easiest to see. The ticket description wasn't enough to choose the category. The system had to look at the documentation first and then use that result when making the ticket update.Even then, getting both parts into the same system wasn't enough by itself. I had to fix the parser twice before the final run matched the actual ticket state.That's probably the part I will remember from this build. The retrieval code and the action code were both working on their own. The bugs showed up when I started passing information from one part to the other.The final nine-task run gave me the result I was looking for: the three systems behaved differently because they had different access to information and actions. The code makes that difference visible.Complete code: https://github.com/Emmimal/rag-vs-agent/References[1] Lewis, P., Perez, E., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 33, 9459–9474. https://arxiv.org/abs/2005.11401[2] Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5), 513–523.[3] Yao, S., Zhao, J., et al. (2022). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629. https://arxiv.org/abs/2210.03629EmiTechLogic articles used to build the corpus for this experiment (22 real, independently published articles — not written for this experiment):Retrieval-Augmented Generation: What is RAG? · Top Chunking Strategies · Agentic RAG · AI-Powered Tutor with RAG · LightRAG vs GraphRAG · Automated Knowledge Graphs with LLMs · ChatGPT for PDF with Python · Build Your Own AI Virtual Assistant · Multilingual Chatbot with LLMsAI agents: AI Agents and Agentic AI · Design and Implement a Multi-Agent System · GitHub Copilot Guide · Resolve GitHub Issues with LLM and RedisData science: Exploratory Data Analysis: Top 10 Python Libraries · Handling Missing Values · What is Data Wrangling? · Data Analysis Using Pandas · Automating Data Cleaning with PyCaretPython: Python Dictionaries · Define and Call Functions · Exception and Error Handling · Creating Python ModulesDisclosureAll code in this article was written by me and is original work, developed and tested on Python 3.12. All performance numbers are from actual measured runs on my local machine (CPU only, standard library only — no external APIs, embedding services, or vector databases), reproducible by cloning the repository and running run_experiment.py, except where explicitly noted otherwise. The knowledge base is built from my own site, EmiTechLogic, specifically so retrieval had to work against real, independently written content rather than text authored to make the demo succeed. I have no financial relationship with any tool, library, or company mentioned in this article.