Make AI Answer the Right Questions, and Show Its Work
Decide the questions a good answer must cover, then make the AI write one sourced sentence for each.
Main Takeaway
Before an AI writes an answer, it helps to first agree on the list of questions a good answer needs to cover. If the AI then writes one short, sourced sentence for each question on that list, you get an answer that covers what matters and lets you trace every claim back to where it came from. This paper builds an AI system that works exactly that way and it beats the usual approaches.
Who is this for
Anyone who relies on AI to pull together facts from a trusted pile of documents (a work knowledge base, a set of reports, a research library) and needs to actually trust the result. If it matters for you to be able to point to the source behind each sentence, and not just hope that the confident-sounding paragraph is right, then this is for you.
Background
“Incorporating Q&A Nuggets into Retrieval-Augmented Generation” is a 2026 research paper by Laura Dietz (University of New Hampshire) and colleagues at the University of Pennsylvania, Yale, the University of Amsterdam, and Johns Hopkins. It introduces an AI system called CRUCIBLE.
To follow along, you only need one idea. When an AI answers a question using a set of documents, researcher
s like to check its work by writing down the handful of small facts a good answer ought to contain — each one as a tiny question-and-answer pair, like “Who makes Roundup weedkiller? — Bayer.” These little fact-checks are called nuggets (small, self-contained facts). Normally nuggets are used afterward, as a grading checklist: count how many facts AI’s answer got right. The clever twist in this paper is to use that checklist at the start, as the plan for writing the answer — not just the test at the end.
If you saw our earlier post on GINGER, this is the next step in that same story.
The issue
The standard way AI answers a fact question is to first go fetch relevant documents, then write based on them. In such approaches, academia has identified many potential issues. This paper focuses on two:
First, repetition. The same fact often appears in many documents, so the AI sees the same point again and again and misses other points worth focusing on.
Second, and more important, you lose the paper trail. To write a smooth summary, most systems blend all the source material together into one mush before writing. The result reads nicely, but you can no longer tell which document backs any particular sentence. For anything that matters, a claim you can’t trace is a claim you can’t really trust.
