RAG that cites its sources: FastAPI + vector search
A trustworthy RAG system answers with receipts. The recipe: FastAPI for the API, a vector store for retrieval, and a prompt contract that forces the model to cite the passages it used — or say it doesn’t know.
Retrieval-augmented generation is how you get a model to answer from your documents instead of its memory. But a RAG answer with no sources is just a confident guess. The differentiator that makes RAG usable in serious settings is citations you can click and verify.
Why citations are the whole point
Hallucination is not a model bug you can prompt away; it is what language models do when they lack grounding. Citations convert an answer from “trust me” into “check page 4.” They also give you a cheap, automatic quality signal: if the model can’t cite a source, it shouldn’t answer.
Ingestion and chunking
Quality starts before retrieval. Split documents into semantically coherent chunks — paragraphs or sections, not arbitrary character windows — and keep metadata (source, title, page) with every chunk. Embed each chunk and store it in a vector database.
- Chunk on natural boundaries; overlap slightly to preserve context.
- Store source metadata alongside each vector so citations are exact.
- Re-embed when documents change; stale vectors cite stale facts.
Retrieval with FastAPI + vector search
FastAPI gives you a typed, async endpoint that takes a question, embeds it, runs a vector similarity search, and returns the top passages with their metadata. Keep this layer thin and fast — retrieval latency is felt directly by the user.
Return more candidates than you need and let a lightweight re-rank pick the best few before they reach the model.
Forcing citations in the answer
This is the part most pipelines skip. Pass the retrieved passages with stable IDs, and make the prompt contract explicit: answer only from these passages, attach the ID of every passage you use, and if the answer isn’t supported, say so. Then validate — if the response cites an ID you didn’t supply, reject it.
If the retrieved passages don’t support an answer, the correct output is “I don’t know” — not a fluent paragraph. Abstention is a feature.
Key takeaways
- A RAG answer without sources is an unverifiable guess.
- Chunk on natural boundaries and keep source metadata on every vector.
- Use FastAPI for a thin, async retrieval layer over vector search.
- Force the model to cite supplied passage IDs or abstain, then validate the citations.
Frequently asked questions
What is RAG?+
RAG (retrieval-augmented generation) retrieves relevant documents from your own data and feeds them to a language model so its answers are grounded in those sources rather than only its training data.
How do you stop a RAG system from hallucinating citations?+
Pass retrieved passages with stable IDs, instruct the model to cite only those IDs or abstain, and validate the response — rejecting any citation that references an ID you did not supply.
Why use FastAPI for a RAG service?+
FastAPI offers typed, async endpoints with low overhead, which suits the latency-sensitive embed-and-search step at the core of a retrieval pipeline.