Can You Use the Citation Verifier, and Build Your Own?

The most useful finding from the cohort's evaluations was also the most reusable. An LLM asked to judge answers passed almost everything, and a short deterministic check caught the failures that mattered. That check is a small Python script, it is ready to pick up, and this guide is how to run it, what it catches, what it cannot, and how to build your own.

Common Legal Help AI. Working draft for comment. Margaret Hagan, Stanford Legal Design Lab.

In the cohort's first full evaluation, the LLM judge passed every answer. All eighty-eight cells scored a perfect one. The deterministic verifier was the entire source of differentiation. It caught hallucinated links, phone numbers attributed to the wrong organization, and citations to resources that do not serve the right state. None of those failures lowered the judge's confidence, and every one of them is the kind of error that sends a person to a dead end or a scam. A confident wrong citation is the most dangerous output a legal help tool produces, and the model grading the model could not see it.

What the verifier is

The citation verifier is a deterministic Python check that runs on every AI answer. It looks for three kinds of factual claims and checks whether each one is real and correctly attributed: web links, organization names, and phone numbers. It runs alongside the LLM judge in promptfoo rather than in place of it. The judge scores the prose, the reasoning, and the tone. The verifier scores whether the answer's facts are true. Together they tell a fuller story than either tells alone.

It works against an allowlist: a list of the real URLs, domains, and organization names drawn from your own content index and service directory. For each link in an answer, the verifier sorts it into one of a few buckets: a resource in your knowledge base, a real resource not in it, a hallucinated address, or a broken link. A hallucinated or broken link fails the check. Organization names are checked against the directory, and phone numbers are checked for misattribution, the case where a real organization is paired with a number that is not its own.

What it cannot do

The verifier checks whether a cited thing exists and is attributed correctly. It does not check whether the legal advice is sound. That is the judge's job, and the human reviewer's. The verifier is only as good as the allowlist behind it: if your directory does not carry the right phone number, the verifier cannot catch a wrong one. It reduces dangerous hallucinations. It does not certify that an answer is correct.

Can you use it

Yes. It is a short script, ready to pick up, and it runs in promptfoo, which is open source and forks without depending on anyone else to keep it running. You wire the verifier in as a second assertion next to the judge, so each answer gets two scores. Each evaluation round the cohort ran this way cost under twenty-five dollars.

How to build your own

The verifier is small enough that a team can build its own, and the steps are the same whether you pick up the cohort's version or write it fresh. First, export your service directory and content index to a plain list of real URLs, domains, and organization names. Second, run that list through an allowlist builder, which produces a single file the verifier reads. Third, drop the verifier in as a Python assertion in your promptfoo configuration. Fourth, run your evaluation with both the judge and the verifier, so you see the prose score and the groundedness score side by side. The pattern to watch for is the dangerous one: a high judge score with a low verifier score, which is an answer that sounded convincing and cited something that does not exist.

The cohort's version, including the allowlist builder, lives in a private repository and is available by request, shared under an open license once you have access. A team with engineering capacity can read it in an afternoon and adapt it to its own directory.

Why this is worth doing