RAG chatbot cost in 2026: what it costs to build, and what it costs every month

How much does a RAG chatbot cost?

A RAG chatbot is a chatbot that answers from your own documents: it searches your content for the passages that match a question, sends them to a language model such as Claude or GPT, and returns an answer with the sources. It has two price tags. The build is a one-off project: with us it starts from $10,000 and typically lands between $10,000 and $20,000. The run cost is monthly: model fees, hosting, the vector index and maintenance.

The model fees are where most people expect the big number, and they are usually the small one. On current small models an answer costs well under a tenth of a cent; on mid-tier models about one and a half cents. The money goes into getting the answers right: preparing the documents, choosing how they are split, testing against real questions and keeping the content fresh.

If you are deciding whether you need retrieval at all, our RAG vs fine-tuning guide covers that first. For chatbots that do not answer from documents, see the broader AI chatbot development cost guide.

What does it cost to build a RAG chatbot?

The build cost depends on how messy your sources are and how accurate the answers must be. The table shows our typical ranges for a one-off build (development, evaluation set and a pilot) next to what a typical US or UK agency quotes for the same scope.

If you want to see the first version working before committing to the full scope, our AI Sprint delivers one working RAG assistant in 4 weeks for a fixed $10,000.

What pushes a quote up:

• Source quality. Clean help articles are cheap. Scanned PDFs, tables, slides, spreadsheets and email threads need parsing and cleaning before anything is embedded, and that work is often the largest single item.
• Permissions. If different users may see different documents, the access rules have to be copied onto every chunk and enforced in the search itself, not in the prompt.
• Accuracy target. A set of 100 to 300 real questions with expected answers and expected sources, scored on every change, is what separates a demo from a bot you can put in front of customers. Our RAG implementation guide explains how we build it.
• Freshness. Content that changes daily needs a sync job that re-indexes only what changed and removes what was deleted.
• Languages and channels. Each language adds evaluation cases; each channel (website, Slack, Teams, Telegram, WhatsApp) adds its own formats and limits.

What does one answer cost on Claude and OpenAI models?

One answer costs between about $0.0007 and $0.07 depending on the model. We assume a typical RAG request: a 600-token system prompt, 8 retrieved passages of about 500 tokens, a short history and the question, so about 5,000 input tokens, plus a 400-token answer. Prices are the standard per-million-token rates on the Anthropic (platform.claude.com/docs/en/about-claude/pricing) and OpenAI (developers.openai.com/api/docs/pricing) pricing pages, checked on November 2, 2026.

The query itself also has to be embedded, which costs a tiny fraction of a cent. Three things move the per-answer cost more than the choice of provider:

1. How many passages you send. Eight passages instead of twenty cuts the input by more than half. A reranking step that picks the best few pays for itself.
2. Reasoning tokens. Models that think before answering bill that thinking as output. A thousand thinking tokens can double or triple the cost of an answer.
3. Prompt caching. The system prompt repeats on every request. Cached input costs a tenth or less of the normal price on both providers, which matters most when the fixed part of the prompt is long.

Our LLM API cost calculator has a "Document search (RAG)" preset if you want to plug in your own traffic.

What does a RAG chatbot cost per month? Three scenarios

The scenarios below use the per-answer costs above. They are illustrations with assumed volumes, not quotes.

Which column applies is decided by your evaluation set, not by the price list. Many documentation bots answer well on a small model once retrieval is good; bots that must reason across several documents or follow complex policies often need the mid tier. A common pattern is to route simple questions to the small model and the rest to a larger one.

Add the other monthly lines: hosting for the chat API and background workers (market range $50 to $500), the vector database if it is not in your existing PostgreSQL, logging and monitoring, and paid channel fees such as WhatsApp Business where they apply.

How much do indexing and the vector database cost?

Indexing is the cheapest part of a RAG system. With 10,000 documents of about 3,000 tokens each, split into 500-token chunks with some overlap and the heading path added to each chunk, you embed roughly 37 million tokens and get about 70,000 chunks.

Storage is small too. 70,000 vectors of 1,536 dimensions take a few hundred megabytes before the index, well under 1 GB in PostgreSQL with pgvector. If you already run PostgreSQL, pgvector adds no licence cost and keeps the vectors next to your tenants and permissions; we show the schema and queries in RAG in PHP with Symfony and pgvector.

A managed vector database costs more but removes the operations work. Pinecone, for example, offers a free Starter plan, then a Standard plan with a $50 monthly minimum, $0.33 per GB per month of storage and usage-based read and write units (pricing page checked November 2, 2026). That makes sense at millions of chunks or when nobody on your team should run a database. For most company knowledge bases, it is not needed on day one.

What does a RAG chatbot cost in the first year?

An illustrative year one for the customer documentation bot above, with assumed numbers:

On a small model the model line falls to about $300 for the year and the total to about $15,900. In both cases the build and maintenance are most of the money, which is why the evaluation set matters: it is what lets you use the cheaper model with confidence.

Is it cheaper to buy a chatbot platform?

At low volume and with a public help center, often yes. Customer service platforms increasingly price their AI agent per outcome: Intercom lists Fin at $0.99 per outcome in November 2026, on top of helpdesk seats. A few hundred outcomes a month cost a few hundred dollars, with no build.

The arithmetic changes with volume. The documentation bot above handles 9,000 conversations a month. If half of them end as billable outcomes, a per-outcome platform costs about $4,450 a month, more than $53,000 a year, against roughly $600 a month to run the custom bot on a mid-tier model. And platforms cannot help when answers come from internal systems, when you need your own access rules, or when documents must not leave your infrastructure. Our guide on AI agents for customer service compares the main platforms with custom builds.

How do you keep RAG chatbot costs under control?

• Measure before you choose a model. Run the same evaluation set on a small and a mid-tier model; pay for the larger one only where it answers better.
• Send fewer, better passages. Hybrid search plus reranking gives the model 5 to 8 good passages instead of 20 mediocre ones.
• Cache the fixed part of the prompt and keep it at the start of the request.
• Summarize long conversations instead of resending the whole history with every message.
• Re-index only what changed, by comparing a content hash before embedding.
• Cap usage per user and per day, and log cost per conversation so expensive topics are visible.
• Batch background work such as re-indexing or generating chunk summaries; both Anthropic and OpenAI halve prices for batch requests.

What have we built with retrieval?

Our most complete retrieval work is the AI Grief Companion for a US startup: chat exports in ten formats are normalized into one message table, encrypted per message at ingest, and retrieval over a fact graph and the chat history gives the model its long-term memory. Our own AI Resume Master runs on PHP and Symfony with the OpenAI API and reached 50,000 monthly active users, which is where our habit of logging cost per request comes from.

Next step

Send us a sample of your documents and twenty real questions your team or customers ask. On a short call we tell you which of the three build types above fits, the model tier we would test first, and a fixed price. If you want to see it working first, the AI Sprint builds one RAG assistant in 4 weeks for $10,000, and our RAG development service describes the full process.