Service S·03
RAG Development
Knowledge-base chatbots and internal assistants that answer from your documents — hybrid retrieval, enforced citations, and an evaluation harness that gates every release. Built by Rohan Jalil.
01 · Definition
What is RAG development?
RAG development is the engineering discipline of building retrieval-augmented generation systems: AI assistants that answer questions from an organisation's own documents instead of a model's memory. A RAG pipeline ingests content — help centres, PDFs, wikis, tickets — splits it into retrievable chunks, indexes it in a vector database such as pgvector, Pinecone, or Qdrant, and retrieves the most relevant passages at question time so the language model composes an answer grounded in real sources. Done properly, RAG development covers far more than an embedding index: hybrid dense-plus-keyword retrieval, reranking, citation enforcement so every claim links back to a source document, sync jobs that keep the index fresh, and an evaluation harness that measures retrieval hit-rate and groundedness before any release. Rohan Jalil builds production RAG systems end to end, from ingestion pipeline to evaluation-gated deployment.
The distinction matters because most teams have already seen a RAG demo. Demos are easy. A system that support staff, customers, or executives can trust with real questions — and that says "I don't know" rather than inventing an answer — is a different engineering problem.
02 · The gap
Why do naive RAG builds disappoint?
A weekend RAG prototype and a production RAG system share maybe 20% of their code. The other 80% is what separates a demo that impresses from an assistant that survives contact with real users.
| Dimension | Naive RAG build | Production RAG build |
|---|---|---|
| Chunking | Fixed-size character splits that sever tables, lists, and mid-sentence context | Structure-aware chunking along headings and semantic boundaries, with overlap tuned per corpus |
| Retrieval | Dense-only cosine similarity — misses exact terms, SKUs, error codes, and names | Hybrid dense + keyword (BM25) retrieval with a reranking pass over candidates |
| Grounding & citations | Free-composed answers with no source trail; plausible text passes for correct text | Every answer cites the passages it used; weak-evidence questions get an explicit refusal |
| Evaluation | "Looks right" spot checks by whoever built it | Golden question set scored for retrieval hit-rate and groundedness on every change |
| Freshness & sync | One-off ingestion that drifts out of date the week after launch | Scheduled sync with change detection, re-chunking, and re-embedding of updated documents |
| Cost | Oversized context windows and one large model for everything; spend is discovered on the invoice | Token budgets per query, caching, and right-sized models per pipeline step |
This is the pattern behind the source-grounded RAG support assistant documented in the case studies: the retrieval and evaluation layers, not the model choice, made the difference between a demo and an assistant that deflected the majority of tier-1 tickets.
03 · Scope
What does a production RAG system include?
Every engagement delivers the same core components. Nothing here is optional; skipping any one of them is how RAG projects fail quietly.
| Ingestion & sync | Connectors for your sources — help centre, Google Drive, Confluence, databases, PDFs — with scheduled sync, change detection, and re-embedding so the index never drifts from the source of truth |
|---|---|
| Chunking strategy | Structure-aware splitting tuned to the corpus: heading-scoped chunks, table preservation, metadata (product, version, audience) attached for filtered retrieval |
| Hybrid retrieval | Dense vector search combined with keyword (BM25) search, merged and deduplicated — on pgvector, Pinecone, or Qdrant depending on your infrastructure |
| Reranking | A second-stage reranker orders candidate passages by true relevance to the question, so the model sees the best evidence first rather than the nearest embedding |
| Citation enforcement | Answers are composed only from retrieved passages and must reference them; when evidence is thin, the assistant declines and routes to a human instead of guessing |
| Eval harness | A golden question set with expected sources, scored automatically for retrieval hit-rate and answer groundedness; scores gate every prompt, model, or index change |
| Feedback loop | Thumbs-down answers and human corrections flow back into the golden set and chunking fixes, so quality compounds after launch instead of decaying |
04 · Evaluation
How do I evaluate RAG quality?
The uncomfortable truth about RAG is that you cannot see quality by chatting with it. Evaluation has to be measured, versioned, and enforced.
Before any pipeline is built, I sit with your subject-matter experts and write a golden question set: 50–150 real questions your users actually ask, each paired with the source passages a correct answer must draw on. This set is the contract the system is built against — not a vibe check after the fact.
Every change to the system is then scored against it on four axes:
- Retrieval hit-rate — does the passage that contains the answer appear in the top results? If retrieval misses, nothing downstream can save the answer.
- Groundedness — is every claim in the generated answer attributable to a retrieved passage? Checked with an LLM judge and verified by human spot audits.
- Refusal correctness — does the assistant decline when the corpus genuinely lacks an answer, rather than improvising one?
- Citation accuracy — do the cited sources actually support the sentences that cite them?
Release gating ties it together: no prompt tweak, model swap, chunking change, or index rebuild ships unless the scores meet or beat the current baseline. That single rule is what keeps a RAG system trustworthy six months after launch.
05 · Process
How does a RAG engagement run?
Corpus audit & golden questions
Inventory your sources, assess their quality and update cadence, and write the golden question set with your subject-matter experts. Scope and success metrics are fixed here.
Ingestion & indexing
Build connectors, chunking, and the vector index — pgvector, Pinecone, or Qdrant — with metadata for filtered retrieval and a sync schedule that keeps it fresh.
Retrieval tuning & citation enforcement
Layer hybrid retrieval and reranking, then wire the generation side: answers composed only from evidence, citations required, refusal paths for thin coverage.
Evaluation & hardening
Run the golden set, close the gaps it exposes, and set the release gate. Add token budgets, caching, and observability so cost and quality are both visible.
Launch & feedback loop
Staged rollout to real users, with feedback and corrections flowing back into the golden set. Handover includes runbooks and the eval harness itself, so your team owns quality.
06 · FAQ
RAG development questions, answered
Which vector database do you recommend?
pgvector, for most teams — it runs inside the Postgres you already operate, so there is no extra service to pay for, secure, or keep in sync with your application data. The assistant in the RAG knowledge-base case study runs on exactly this pattern. Pinecone or Qdrant earn their place when you need very large indexes, dedicated scaling, or multi-region serving; the pipeline is built so the store can be swapped without rewriting retrieval logic.
Can it cite its sources?
Yes — citation is enforced, not optional. Answers are composed only from retrieved passages, each answer links to the documents it used, and citation accuracy is one of the scored axes in the evaluation harness. Users can click through and verify any claim, which is what makes the assistant trustworthy enough for support and internal knowledge work.
How do you stop hallucinations?
By removing the model's licence to improvise. The assistant may only answer from retrieved evidence; when retrieval comes back thin, it says so and routes the question to a human instead of guessing. Groundedness and refusal correctness are measured on the golden question set, and no change ships if those scores drop. Hallucination is treated as a release-blocking defect, not an accepted quirk of LLMs.
Does my data train anyone's models?
No. Builds use the OpenAI and Anthropic APIs under their standard commercial terms, which do not use API inputs or outputs to train models. Your documents and the vector index live in your own infrastructure — with pgvector they never leave your Postgres — and the only data sent to a model provider is the retrieved passages needed to answer each individual question.
07 · Related
Related work & services
A source-grounded RAG assistant that deflected the majority of tier-1 support tickets
Hybrid retrieval, citation-enforced answers, and an evaluation harness that gated every release.
Read the brief → S·01AI Agent Development
When answers aren't enough and the system needs to act: LangGraph multi-agent architectures with human-in-the-loop safeguards — often built on top of a RAG layer.
Explore → ProfileRohan Jalil — professional profile
Top Rated on Upwork, 100% Job Success, 81 jobs, 27,000+ hours. Full track record, stack, and verification links.
View profile →