AI / ML
FeaturedAsk Shahzaib- An Agentic RAG Chat
A chat app that answers questions about me in my own voice, grounded in a private corpus of my CV and notes. A LangGraph agent decides whether a question needs the knowledge base, judges whether what it retrieved is good enough, and declines playfully rather than inventing an answer.
Tech Stack
My Role
Solo: agent design, backend, frontend, prompt and persona work, deployment.
Constraint
The corpus describes a real person, so a hallucination is not a quality bug. it is misinformation about me, published under my own name. That ruled out the usual "retrieve and hope" approach. Secondary constraints: every request makes paid LLM calls on a public URL, and serverless deployment means no shared state between invocations.
Timeframe
1 Week · team of 1
Problem
I wanted a chat app that could answer questions about me, my experience, my projects, what I am like to work with, in my own voice, so someone could interrogate my background conversationally instead of reading a CV top to bottom. The obvious build is the one everybody ships: embed some documents, retrieve the nearest chunks on every question, paste them into a prompt, return whatever comes back. That approach has a failure mode I was not willing to accept. Retrieval is a similarity guess and a language model handed a bad guess does not say so it answers anyway, fluently and with total confidence. When the subject of the corpus is a real person, that is not a glitch in a demo. It is a machine wearing my name and inventing facts about my career to whoever happens to be asking, which is a strictly worse outcome than having no chat app at all. The second problem is scope. A chat box that talks like a helpful assistant invites people to treat it as one. Visitors ask it to debug their code and a general-purpose model will happily oblige on my OpenAI bill, in a product that is supposed to be about me.
What I Did
The fix was to stop treating retrieval as a step in a pipeline and make it a decision the system is allowed to revisit. The app runs a six-node LangGraph agent, and the interesting nodes are the ones a naive RAG build does not have. moderate runs first, on every message. It classifies rudeness judging only the current message with no conversation history attached deliberately, so that an apology or a calm follow-up after a heated message is not dragged down by the earlier tone. A first offence gets a mocking cool-down line; a repeat, tracked through session state, wraps the chat. route decides whether a message needs the knowledge base at all. Greetings and small talk skip the vector search entirely, which removes both the latency and the cost of a lookup that was never going to help. The same node short-circuits requests to write, fix or explain code with a playful refusal, so the app stays what it is instead of drifting into being a free coding assistant. grade is the node that does the real work. After retrieval, a second model call judges whether the chunks that came back can actually answer the question. If they cannot, rewrite reformulates the query using concrete terms likely to appear in a CV or personal notes, and the graph loops back to retrieve. A hard cap at two attempts means an unanswerable question cannot loop forever, which matters when every lap costs money. generate then picks exactly one of three behaviours: answer from context, reply conversationally, or decline. The declines are the anti-hallucination mechanism made concrete: when the corpus has nothing relevant, the agent is instructed to refuse rather than infer, in a fresh playful one-liner each time rather than a canned string. Two details I am glad I got right. Contact handling extracts my real email and LinkedIn from the knowledge base once and caches them, so those values can never be fabricated, while the phone number is deliberately never extracted or shared. And ingestion uses deterministic chunk IDs derived from the source filename, so re-running it overwrites existing vectors instead of quietly accumulating duplicates.
Result
The app is live and behaves as designed across all three of its paths: it answers from the corpus with sources attached, it routes small talk and code requests away from the knowledge base, and it declines cleanly on questions the documents do not cover rather than guessing. Asked about my salary, it says that is private and offers to talk about my experience instead which is the behaviour I actually wanted, and the one a naive build would have gotten wrong by confabulating a number. The cost of the design is worth stating plainly. Agentic RAG is slower and more expensive than stuffing documents into a prompt: a single question can spend two to five model calls across moderation, routing, grading and generation. Measured against the live deployment, a routed-away request returns in about 2.5 seconds, a retrieval answer in about 5, and a cold serverless start closer to 8. For a chat about one person that is a fair trade, and naming the trade is more useful than pretending the architecture is free. The clearest remaining weakness is operational rather than architectural. Rate limiting is in-memory, so it resets on every serverless cold start, and the CORS allowlist does not constrain requests that arrive without an Origin header. Neither matters much at current traffic, but both stand between this and something I would leave running unattended.