I built a chat app that answers questions about me in my own voice, grounded in my CV and some personal notes. The naive version of this is a weekend project: embed the documents, retrieve the nearest chunks for every question, paste them into a prompt, return the reply.
I did not build the naive version, because it fails in a specific way I was not willing to ship. Retrieval is a similarity guess. A language model handed a bad guess does not tell you the guess was bad — it answers anyway, fluently and with complete confidence. When the corpus describes a real person, that is not a rough edge in a demo. It is software wearing my name, inventing facts about my career, for whoever happens to be asking. Worse than having no chat app at all.
So the retrieval step had to become a decision the system is allowed to reconsider. That is what "agentic" buys you, and it is worth being concrete about what the nodes actually do rather than waving at the word.
The graph
message
│
▼
moderate ──(abusive)──────────────────────────────────► END
│
▼
route ──(code request)────────────────────────────────► END
│ └──(greeting / small talk)──────────► generate ──► END
▼
retrieve ──► grade ──(sufficient OR attempts ≥ 2)────► generate ──► END
│ ▲
└──(insufficient)──► rewrite ──────────┘Six nodes. Three of them are the ones a naive build does not have, and those three are the entire point.
grade: the node that does the real work
This is the idea worth stealing. After retrieval, before generation, a separate model call judges whether the chunks that came back can actually answer the question:
const grader = llm.withStructuredOutput(
z.object({ sufficient: z.boolean() }),
{ name: "grade" },
);
const { sufficient } = await grader.invoke([
{ role: "system", content:
"You judge whether the provided context is sufficient to answer the " +
"question. Answer sufficient=true only if the context actually " +
"contains the information needed." },
{ role: "user", content: `Question: ${state.question}\n\nContext:\n${context}` },
]);Two things make this work. The judgement is a separate call from the answer, so the model is not simultaneously deciding whether it can answer and trying to. And the output is structured — a single boolean through Zod — so there is nothing to parse and no room for a hedged paragraph where a decision belongs.
If the context is insufficient, rewrite reformulates the question into concrete terms likely to appear in the source documents — job titles, technology names, specific hobbies — and the graph loops back to retrieve. A query like "tell me more" becomes something a vector search can actually match.
The retry cap is not optional
const afterGrade = (state) =>
state.sufficient || state.attempts >= config.maxAttempts
? "generate"
: "rewrite";Without attempts >= maxAttempts, a question with no answer in the corpus loops forever: grade says insufficient, rewrite rephrases, retrieve returns the same unhelpful chunks, grade says insufficient. Every lap is two paid model calls. The cap is set to 2 — so at most one rewrite, then the agent proceeds to generate with whatever it has and is expected to admit it cannot answer. A retry loop over a paid API without a hard stop is a billing incident waiting for the right question.
route: not every message deserves a vector search
"Hi" does not need a similarity search over my CV. Neither does "thanks" or "cool". Routing those straight to generation removes a Pinecone round trip and an embedding call from the most common messages in any chat app.
The same node handles a problem I did not anticipate until the app was live. A chat box that talks like a helpful assistant gets treated like one — people paste stack traces into it. A general-purpose model will happily debug them, on my OpenAI bill, in a product that is supposed to be about me. So the router classifies that intent explicitly and short-circuits:
if (codeRequest) {
return { codeRequest: true, answer: pick(CODE_REFUSAL_LINES) };
}Note what this is: a refusal that costs one model call and zero retrieval. The boundary is enforced by the graph's shape, not by asking the generation prompt nicely to stay on topic. Prompt instructions are suggestions; an edge to END is a guarantee.
The distinction the router has to hold is subtle, and it is worth writing into the prompt rather than hoping: asking me to fix code is a code request, asking me which technologies I work with is a question about my experience and should retrieve normally.
moderate: judge the message, not the mood
The first node runs on every message and flags rudeness. The design decision inside it is the one I would defend hardest:
// Judge ONLY this single message on its own — no history — so an apology or
// calm reply after an earlier rude message isn't dragged down by that tone.
const { abusive } = await mod.invoke([
{ role: "system", content: "Decide if THIS one message, on its own, is rude…" },
{ role: "user", content: state.question },
]);Every other node in the graph receives conversation history, because context is what makes "tell me more" resolvable. This node deliberately does not. If you pass history to a moderation call, a genuine apology arriving after a hostile message gets classified by the ambient tone of the conversation rather than its own content, and the user who just calmed down gets punished for it. Sentiment carries across turns in a way that a per-message judgement should not.
Repeat offences are handled through session state rather than by re-reading the transcript: a first offence sets warned and returns a cool-down line, and a second offence while warned is true ends the session.
generate: exactly one behaviour, never two
The final node picks one of three modes — answer from context, reply conversationally, or decline — and the prompt is explicit that these do not combine:
Never combine an answer with a tease. If you can answer from the info provided, just answer.
That line exists because the failure it prevents is the most common one in persona chatbots: a model that answers the question and then adds a coy deflection, which reads as evasive about something it just told you. One behaviour per turn.
The declines themselves get a small trick. A fixed refusal string is fine the first time and robotic the second, so the prompt is handed three randomly sampled examples of the intended vibe each turn, with an instruction to write a fresh one rather than copy them:
function sampleTeaseHints(n = 3) {
return [...TEASE_HINTS].sort(() => Math.random() - 0.5).slice(0, n).join(" / ");
}Randomising the examples rather than picking from a fixed list means the output varies even across identical questions, without the model drifting off-persona. It is a cheap way to get variety out of a temperature-0.2 model.
What it looks like
[INSERT images/askshahzaib-conversation.png — caption: Answering from the corpus, then declining cleanly on something it has no business answering.]
Asked what I have worked on, it answers from the corpus and attaches its sources. Asked my salary, it declines and offers to talk about my experience instead. That second reply is the whole reason the architecture exists — a naive build would have produced a confident number.
Two details that are not glamorous but matter
Contact details are never generated. My email and LinkedIn are extracted from the knowledge base once, cached, and injected into the prompt as exact values with an instruction never to invent alternatives. A model asked for an email address will otherwise produce something plausible-looking, which is the worst possible failure mode for a contact detail. My phone number is deliberately never extracted at all — the safest way to not leak a value is to not have it in scope.
Ingestion is idempotent. Chunk IDs are md5(source)-chunkIndex, so re-running ingestion overwrites the existing vectors instead of adding a second copy of every chunk. Without deterministic IDs, the third re-ingest leaves you with three near-identical chunks competing for the same top-5 slots, which quietly degrades retrieval in a way that is genuinely unpleasant to debug.
What it costs
Agentic RAG is not free, and the honest version of this post has to say so. A single question can spend two to five model calls — moderation, routing, grading, possibly a rewrite, then generation — where naive RAG spends one. Measured against the live deployment:
Routed-away request (code refusal, no retrieval) — ~2.5s
Retrieval answer with grading — ~5s
Cold serverless start — ~8s
For a chat about one person, answering a handful of questions per session, that is a fair trade for not lying. For a high-volume product it would need work: caching the router's verdict on common phrasings, or a smaller model for the moderation and routing calls, which are classification tasks that do not need the same model as generation.
The part I would fix next
The weakest thing about the deployment is not the agent, it is the operational edges. Rate limiting is in-memory, so on serverless it resets with every cold start and functions as a speed bump rather than a limit. And the CORS allowlist admits requests that arrive with no Origin header at all, which is correct for server-to-server callers and also means the endpoint is trivially callable outside the browser. Neither matters at the traffic this gets. Both would matter the moment it found an audience, and "it currently has no audience" is a poor security control.
The generalisable lesson is that "agentic" is not a quality you add to a RAG pipeline — it is the decision to let the system evaluate its own intermediate results before acting on them. The grading node is three lines of Zod and one model call, and it converts a system that guesses into a system that can tell you it does not know. Everything else in the graph is scope control.
Code is at devSol-shahzaib/self-RAG, and the app is live at self-rag-as2s.vercel.app if you want to try to make it say something it shouldn't.