Ask three vendors for "an AI assistant for our documents" and you may receive three proposals that use the same words and describe different things. One is a chatbot with a friendly prompt. One is a retrieval-augmented generation system that reads your documents before it answers. One is that system connected to your other software through the Model Context Protocol, so it can check a record or raise a ticket as well as answer. Each is the right choice for some problems and the wrong choice for others. The difference is not how clever the model is. It is where the answer comes from and what the system is allowed to do with it.
To keep the comparison concrete, we use one question throughout: "Can we pay Northgate Fabrications invoice INV-2041?" The supplier and invoice are fictional, borrowed from our invoice OCR article. The behaviour of each system is what you would see in practice.
What is a chatbot, and what does it actually know?
A chatbot in the current sense is a large language model behind a chat window, with a prompt that sets its tone and role. It knows two things: what was in its training data, which stops at a date months or years ago, and what is in the current conversation. That is all. It does not know your invoices, your policies, or who is allowed to see them, unless someone pastes them into the chat.
Asked our question, a chatbot answers fluently. It will explain that invoices should be matched against a purchase order and goods receipt before payment, and it may add plausible detail about approval thresholds. None of it refers to Northgate Fabrications or INV-2041, because it cannot see them. If pressed, it may invent a policy or a status. That is not a defect in the model; it is the design. A chatbot is excellent for drafting, summarising text you give it, and general questions, and it is the wrong tool for any question whose answer lives in your records.
What does RAG add?
Retrieval-augmented generation puts a step before the model. When a question arrives, a retriever searches an index of your documents, selects the passages most likely to answer it, and hands those passages to the model with the instruction to answer only from them and to cite them. Our article on hybrid search covers how that retrieval step is built so it finds both exact references and plain-language questions.
Asked our question, a well-built RAG system retrieves the accounts payable policy and quotes the relevant clause, for example that invoices over a threshold require a three-way match. If the invoice record and the purchase order have been indexed, it finds them too, shows the amounts, and cites each source so the reader can check. It does all this within the permissions of the person asking: a user who may not open supplier bank details will not receive an answer containing them, because the filter is applied before ranking, not after.
Then it stops. RAG answers questions; it does not take actions. If the goods receipt lives in the ERP and was never indexed, the system can only say what the policy requires and what documents it found. It cannot look the receipt up. That boundary is a feature when the goal is trustworthy answers, and it is the limit you hit when the goal is to complete a process.
RAG also brings obligations a chatbot does not have. The index must be kept current. Chunking, retrieval quality, and citation accuracy must be measured on your own documents before go-live and after every change. Permissions must be synchronised from the source systems. Our RAG development services page sets out that work in detail.
What is MCP, and what does it add to RAG?
The Model Context Protocol is an open standard for connecting AI applications to tools and data sources. It was published by Anthropic in late 2024 and has since been adopted by most major model providers and developer tools. In plain terms, it defines how a model-driven application asks an external system for something, what a "tool" looks like, and how results come back, in the same way for every system. A database, an ERP, a ticketing platform, or a document search index can each be exposed as an MCP server. The assistant, acting as an MCP client, can call any of them without bespoke integration code for each one.
Applied to our setting, MCP turns a RAG system into one tool among several. The document retriever becomes an MCP server. The ERP exposes read tools such as looking up a purchase order or a goods receipt. The ticketing system exposes a tool to raise a task. The model, given our question, can now retrieve the policy, call the ERP to check whether goods were received against PO-7713, compare the amounts, and draft a recommendation: "Policy requires a three-way match. Purchase order PO-7713 is open for £1,460.00. Goods receipt GR-2210 was logged on 14 August for the full quantity. Invoice INV-2041 reconciles. Ready for approval." Every statement carries the evidence the tool returned.
Two points matter here. First, this is what people mean by an "agent": a model that decides which tools to call, in what order, to reach a goal. Second, MCP does not make the model any more knowledgeable or any more careful. It makes connecting tools cheap and uniform. That is valuable, and it is also exactly why the controls around it must be designed rather than assumed.
Where does the risk move when tools are added?
A chatbot's failure mode is a confident wrong answer that a person reads and may act on. RAG narrows that to answers unsupported by the retrieved evidence, which citations make visible. Tools introduce a third failure mode: an action taken on the wrong basis. Three controls address it, and they are the same ones we apply on every engagement.
- Least privilege per tool. The ERP server exposes read tools to the assistant and nothing else. A tool that writes, such as posting an invoice, is either absent or gated behind explicit approval. The assistant cannot acquire new tools at runtime.
- Tool results are data, not instructions. Text returned by a document, a ticket, or an email can contain instructions aimed at the model. The system treats everything a tool returns as content to reason about, never as a command, and injection attempts are part of the evaluation set.
- Approval before any write, with an audit trail. The assistant prepares the action and the evidence; a named person approves it; the record shows who approved what, when, and on which evidence. Our security and governance page describes the stage-by-stage controls.
Side by side: what changes at each step
| Question | Chatbot | RAG | RAG with MCP tools |
|---|---|---|---|
| Where does the answer come from? | Training data and the conversation | Passages retrieved from your documents at question time | Retrieved passages plus live results from connected systems |
| Can it cite sources? | No | Yes, per passage | Yes, per passage and per tool result |
| Does it respect user permissions? | No concept of them | Yes, filtered before ranking | Yes, for retrieval and for each tool separately |
| How current is it? | As of its training date | As of the last index update | Live for anything reached through a tool |
| Can it act? | No | No | Can propose and, with approval, perform defined actions |
| Typical failure | Confident invention | Unsupported answer, weak retrieval | Action on the wrong basis, injection through tool results |
| What must be evaluated | Tone and helpfulness | Retrieval quality, citation accuracy, answer faithfulness | All of RAG plus tool selection, action precision, approval routing |
| Best for | Drafting, summarising given text, general questions | Questions whose answers live in your documents | Processes that need a check or an action after the answer |
How do you decide which one you need?
Start from the question people actually ask, not from the technology.
- If the answer does not depend on your own documents or records, a chatbot is enough, and a well-prompted one is cheaper and faster to deploy than anything else on this page. Drafting emails, rewriting text, explaining a concept: chatbot.
- If the answer must come from your documents and be defensible, you need RAG. "What does our policy say", "which clause covers this", "what did the survey report find": RAG, with citations and permissions, and with retrieval measured on your material before anyone relies on it.
- If answering is not enough because a process has to continue, you need RAG with tools, and you need the approval design more than you need the tools. "Check whether this can be paid and prepare the approval", "find the current revision and open a change request": RAG with MCP tools, with a person on the consequential step.
Most document problems we see stop at the second step. The pull towards the third is strong because it sounds more advanced, and the honest advice is that tools should be added only when a specific step in a specific workflow needs them, after the retrieval underneath has been proven. Our how it works page has the decision table we use with clients, and the AI agents page describes the approval model.
Three claims to be wary of
- "It is a chatbot trained on your documents." Almost always it is not trained on them at all; either it is a chatbot with documents pasted into the prompt, which fails as soon as the documents outgrow the context window, or it is RAG described loosely. Ask where the answer comes from and whether it cites.
- "MCP makes the assistant smarter." It makes the assistant better connected. Smarter, in the sense of fewer wrong answers, comes from retrieval quality, evaluation, and constraints, none of which the protocol provides.
- "Agents replace RAG." An agent that acts on documents still has to find the right document first. Retrieval is the foundation the tools sit on. A weak retriever with many tools is a fast way to take confident actions on the wrong evidence.
What this means for a buyer
When a proposal arrives, ask three questions and expect specific answers. Where will each answer come from, and will it show me? Which systems can it read, which can it change, and who approves the changes? How will retrieval and actions be measured on our own documents before go-live? A chatbot, a RAG system, and a RAG system with MCP tools answer those questions very differently, and the answers tell you which one you are actually being offered.
Related reading
Why legal RAG needs hybrid search explains the retrieval layer that all of this depends on. Invoice OCR: validation before automation follows the same invoice through the checks that decide whether its fields can be trusted at all. All articles are listed on the Insights page.