Most retrieval-augmented systems treat the retriever as a hint and the language model as a writer. The retriever returns its best guess, the model dresses it up, and what comes out the other side is fluent, confident, and — disturbingly often — wrong.
I wanted the opposite. I wanted a system that, when its retriever cannot find anything sufficiently close to the question, simply says so. HonestRAG is the result.
The threshold
The mechanism is unglamorous. Each query is embedded, the top-k chunks are returned with their cosine similarities, and a single number — the maximum similarity score — is compared against a threshold. Below the threshold, generation never runs.
The model never sees the prompt. It cannot hallucinate what it is not asked.
This sounds trivial. In practice, choosing the threshold is the entire problem. Too low and the system answers from noise; too high and it refuses queries it could have handled. I tuned mine against a small set of golden test cases — questions I knew the corpus could answer and questions I knew it couldn't — and watched precision and recall trade against each other.
What refusal sounds like
The refusal message matters. "I don't know" is honest but unhelpful. "I don't have enough information about X in my sources" is better — it tells the user what failed.
if max_similarity < THRESHOLD:
return refuse(query, top_chunks)
else:
return generate(query, top_chunks)Five lines. The whole system pivots on them.
What I learned
People prefer being told "I don't know" to being told something false. This is obvious in the abstract and somehow controversial in the LLM industry. The discomfort, I think, is that refusal feels like a product failure. It isn't. A refusal is a calibrated estimate of confidence rendered as text. That is a feature.
The interesting failure mode is the almost-relevant chunk — the document that talks about the right entity but the wrong attribute. Threshold gating doesn't catch these; you need an additional check, often another model call, to verify alignment between question and evidence. I'm still working on that part.
For now: if you ask HonestRAG something it doesn't know, it will tell you. I find this restful.