Every AI project produces a golden demo — the rehearsed run where retrieval finds the right document and the answer lands. Far fewer produce a golden dataset, which is the thing that tells you whether the demo was representative or lucky. Our order of operations is fixed: the dataset comes first, the demo earns its applause against it. Evals before features, every engagement.
What a golden dataset actually is
A golden dataset is a curated set of real inputs paired with adjudicated correct outputs — real questions your staff actually ask, real documents from your actual corpus, and for each one a verdict on what a correct answer contains, which sources it must cite, and what it must not say. It deliberately over-represents the awkward cases: the ambiguous query, the outdated policy that was superseded, the question whose honest answer is “we don’t know”, the request the system should refuse.
It is small and dense rather than big and vague. A few hundred adjudicated cases that the business stands behind beat ten thousand scraped pairs nobody has read. And it is never finished: every production failure, every human override, every dispute becomes a new case. The dataset is the institutional memory of what “correct” means in your domain — which is why it belongs in your repos, versioned like source code, owned by you.
Domain owners adjudicate. Engineers don’t get a vote.
Here is the part most teams get wrong: engineers must not be the ones deciding what a correct answer looks like. An engineer reading a contract clause can judge whether an answer is fluent and cited. Only the person who owns that domain — the controller, the compliance lead, the senior support handler — can judge whether it is right. When engineers adjudicate their own evals, the suite quietly converges on “answers that look plausible to engineers”, and the system passes its tests while failing its users.
So adjudication is a working session with the domain owners, and their time is the scarce input we budget for early. It is also where trust is actually built. The moment a sceptical department head has personally marked a set of answers right and wrong, the eval score stops being a vendor’s number and becomes their number. That changes every conversation that follows.
Regression evals in CI, gating every release
A golden dataset you run occasionally is a report. A golden dataset wired into CI is a gate. Every change to the system — a prompt edit, a chunking tweak, a retriever swap, a dependency bump — runs against the full suite before it merges, exactly like a unit-test run, and a regression below threshold blocks the release. Answer correctness, citation coverage, abstention behaviour, hallucination rate: measured per change, not per quarter.
This is what makes AI systems changeable. Without regression evals, every improvement is a gamble that something else didn’t silently break, so teams stop touching the system and it fossilises. With them, a prompt change is as safe to ship as any other reviewed diff. The same suite, run continuously against production, is your drift alarm — when a provider updates a model underneath you, you find out from a failing eval, not from a customer.
Why the harness is often our first deliverable
In a new AI engagement, the eval harness is frequently the first thing we build — before the pipeline it will eventually judge. Partly because everything built afterwards survives on it, as we argued in RAG from demo to production. Partly because it forces the definition-of-correct conversation to happen at the start, when it is cheap, instead of at launch, when it is a crisis. A harness plus an adjudicated dataset is a deliverable a client keeps even if the project stops there: it turns every future vendor claim, including ours, into something checkable.
Model-agnostic, data-jealous
Models are the most replaceable part of the stack, and the eval suite is what makes replacing them boring. When a better or cheaper model appears, you run the suite, read the diff, and decide on evidence — a swap becomes an afternoon of evals rather than a leap of faith. That is the practical meaning of our model-agnostic, data-jealous stance: hold the model loosely, because the suite lets you; hold the golden dataset tightly, because it is the one asset in the whole system that compounds and cannot be bought.
Building the harness first is standard practice in our Applied AI & Automation engagements. If your AI system currently ships on confidence rather than evidence, bring it to a tech lead — a technical call, not a sales call.