Common Tools We Built That Every Team Can Use
This guide is the give-back. It is the catalog of what the cohort built so that a state team does not have to build it again: the shared vocabularies, the datasets to build and test with, and the working tools and methods. Each item says what it is, what real problem it solves, how a team would use it, and how ready it is, because a catalog that hides the readiness of its contents is worse than no catalog. The honest summary up front is that some of this is ready to use today, some is built and still being tested, and some exists as a documented pattern rather than a running service. Making all of it openly reachable to a stranger who never joined the cohort is itself part of the work that remains.
Common Legal Help AI. Working draft for comment. Margaret Hagan, Stanford Legal Design Lab.
A hard truth runs under this work: the AI layer only works on top of content that has been kept current and a directory that has been structured, and the structured directory and the current content are the real work. This guide is the answer to the obvious next question, which is whether a team has to do all of that work alone. It does not. The cohort built a set of shared assets so that a team can skip months of groundwork, and this guide is the inventory of them.
These assets are also the raw material for the strategies the strategy guide described. The runtime content layer needs the standards and the content. The open feed is these assets published openly. The public benchmark is the rubrics and the suites and the harness. A team picking these up is assembling the foundation that the other guides keep pointing back to.
One convention runs through the catalog. Each asset carries an honest readiness label, and there are three. An asset that is ready to use now is one a team can pick up and apply today. An asset that is built and still being tested or finalized is usable, with the understanding that it is still changing. An asset that is documented as a pattern is a blueprint or a schema rather than a running service. Where the public home for an asset is still being stood up, the reliable way to get it for now is through the cohort, and standing up the open, public versions is the field-level work the national-leadership guide takes up.
The shared vocabularies
A vocabulary is the least glamorous asset in the catalog and the one that makes everything else possible, because two systems can only work together if they describe the world the same way.
The LIST taxonomy of legal issues. This is the shared vocabulary for what a legal problem is. It holds over 1,300 terms across 20 top-level categories, with hierarchical codes such as HO-02-00-00-00 for eviction, and it is about a decade old, expanded over the course of the cohort with the partners' own taxonomies and content. Its value is interoperability: when Illinois and California both tag their eviction content with the same code, an AI tool can pull from either source and know what the content is about, which is the precondition for the cross-state work these guides assume. It is the vocabulary behind the classification task. It is maintained at taxonomy.legal and is being rebuilt on a new platform, with the ground truth kept in a working Airtable, so the taxonomy is ready to use now and its home is in transition. A local or custom taxonomy can be crosswalked to LIST rather than replaced, as Oregon's LRS taxonomy was, so a team that already classifies its content its own way maps into the shared vocabulary without starting over, and teams with their own taxonomy are welcome to have it crosswalked the same way.
The Legal Help Task Taxonomy. This is the shared vocabulary for the work itself, the fifty tasks across seven stages of what people and providers do, each with a definition, a short code, a starter rubric, and sample prompts. It is the answer to the question the grant set out to scope, which was what tasks AI should help with, and it is the frame the rest of these guides rests on, because every claim about whether AI is good enough is a claim about a specific task on this list. A team uses it to decide what to build, to scope what to evaluate, and to compare its work to another state's on the same terms. It is published on JusticeBench and ready to use now.
The standard jurisdiction list with FIPS codes. This is the shared vocabulary for place. It is a clean, consolidated list of every US state and county with its federal FIPS code, and it solves a problem that quietly breaks routing and federated retrieval, which is that "Cook County," "Cook County, IL," and "17031" all need to resolve to the same jurisdiction before any filtering across state lines can work. It is the asset behind the geography half of the routing task. It is part of the cohort's deliverables and ready to use now.
The standard audience categories. This is the shared vocabulary for who a person is or who a service is for. It is a controlled list of 17 categories, including general population, low income, veterans and military, survivors of domestic violence, immigrants and refugees, seniors, youth, people with disabilities, farmworkers, people experiencing homelessness, people who are incarcerated, and others. A team uses it to tag service directory entries and content so that referrals route to the right population and a tool can match a user to the help meant for them. It serves both routing and classification, and it is ready to use now.
Recording eligibility, composed from these vocabularies. Eligibility is the hardest field in a service directory and the one the field has not standardized, and the cohort has not built a finished eligibility standard. What a team can do today is compose the vocabularies above: the audience categories for who a service is for, the jurisdiction list for where it reaches, and the LIST codes for what it covers, plus a few plain fields it fills in directly, such as income relative to the federal poverty level, case type, and immigration status. Recorded that way, in structured fields rather than free text, eligibility is already enough for a routing tool to act on. A shared eligibility schema that goes beyond this is open work, not a finished deliverable.
Another standard sits alongside these. The Legal Help Knowledge Standard is the umbrella set of field names, categories, and values that make legal help data interoperable across states and tools, and it is at an early version.
The datasets to build and test with
These are the files a team uses to seed its own work and to test its tools, and the point of them is that a team can work with realistic material without exposing a single real client.
Synthetic legal help queries. This is a library of synthetic and real user questions, each labeled with its LIST code and jurisdiction, and calibrated against the actual frequencies of legal need from the LSC Justice Gap report, the HiiL and IAALS studies, and state bar data. It spans the most common civil issues, including housing, family, debt, benefits, employment, immigration, and reentry. A team uses it to test a chatbot, a triage system, or a classifier against a realistic distribution of questions rather than against a handful of cherry-picked demos, and it is published on JusticeBench and ready to use now.
Synthetic legal documents. This is a library of synthetic documents, including eviction notices, debt collection letters, benefits denials, and court orders, each with ground-truth labels, and the synthetic notice generator from the Eviction Notice Defect Spotter work is one of its sources. A team uses it to test a document identification or explainer tool without ever touching a real client's document, which is the responsible way to develop on the document task. It is available through the cohort's linked repositories and is still growing.
The 150 Priority Issues scenarios. This is the set of 150 highest-stakes and most-frequent scenarios across the LIST taxonomy, each with a sample question, a one-line description, a draft risk tier, draft escalation triggers, and a primary user. A team uses it two ways, as a starting point for the safety review of its own content and as a scenario library for evaluation, and it is the backbone of the content safety wrapper described below. It lives in a working Airtable, its risk tiers are still in draft, and it is ready to use as a starting point.
For the datasets aimed specifically at measurement, the eleven datasets and six task suites are the ones the measurement guide walked through, including the High-Risk Challenge set, the refusal and escalation cases, and the query-to-webpage gold pairs, so this guide points back to that one rather than listing them again.
The tools and methods
These are the working software and the documented practices, and they are the part of the catalog that turns the standards and the datasets into something a team can run.
The LIST classifier. This is software that takes a piece of content, whether a title, a description, or a short excerpt, and assigns it the most likely LIST codes. It uses the FETCH method that Quinten Steenhuis developed, which runs ensemble voting across several models from Anthropic, OpenAI, Google, and Mistral, weighting first-choice and second-choice votes, presenting an abbreviated form of the taxonomy to the models, and generating follow-up questions when a classification is ambiguous. It is packaged as a Python library and is also reachable through an MCP server endpoint, so it can be wired straight into a content workflow. A team with a legacy content library that was never tagged with LIST codes uses it to auto-tag at scale and then puts an expert on the low-confidence cases, which is the content-pipeline pattern. It is currently being tested against a 197-case ground-truth set, so it is built and still being validated.
The quality rubric library. This is the set of evaluation rubrics for the common legal help tasks, with one rubric shaped to each task rather than a single rubric stretched across all of them. The most developed is the brief legal help question-answering rubric, which scores eight dimensions: legal accuracy, completeness, jurisdiction precision, safety as a veto, tone, actionability, personalization, and the dimensions specific to the issue area, all on anchored zero-to-three scales. Other tasks carry their own structure: the PII masking rubric scores five checks behind a set of hard fails, and the classification and referral suites score against a known correct answer. What the rubrics share is the discipline of anchored scales, safety as a veto, and standards drawn from the field, not an identical list of dimensions. A team uses the library to score its own tool's outputs in a way that is comparable to other tools and other states, and it is ready to use now in promptfoo or any similar harness. The research behind the question-answering rubric is written up in the Hagan 2024 JURIX paper, which a team can point to when it needs to justify the rubric to a board or a funder.
The evaluation methodology and harness. This is the documented, end-to-end method the measurement guide is about, which pairs the promptfoo harness with a hybrid dataset of representative and adversarial cases, an LLM judge scoring against the domain-grounded rubric, the deterministic verifiers that check citations and jurisdictions and organizations, and the expert blind-review pass that calibrates the judge. A team uses it by pointing the configuration at its own tool and getting results that mean the same thing as everyone else's. The configuration is reproducible now, and the full how-to is being published alongside the cohort report.
The content safety wrapper. This is the scaffolding that sits around a brief-advice tool to catch the high-risk failures the grounding guide found, and it works at the level of the scenario rather than the individual content item. For each high-risk scenario it carries the LIST term, the risk tier, the clarifying questions the situation requires, and the triggers for escalating to a human, drawing on Siobhan Durcan's framing for what a tool must refuse or escalate. A team uses it to wrap a brief-advice tool so that the dangerous questions get a clarifying question and an escalation instead of a confident wrong answer, which is the architectural response to the Red-tier failures the grounding guide reported. The schema and the 150 starter scenarios exist, and the framework is still being built out.
The KB Playbook and the federated publisher pattern. These are the two documents a state team uses to build a knowledge base and then to share it. The KB Playbook is the how-to for standing up and running a knowledge base, covering content structure, tagging, chunking, and refresh cadence. The federated publisher pattern is the reference architecture for how a state team publishes its content to a shared knowledge base while keeping local control of it. It covers ingestion through publisher connectors, respectful crawling, and managed uploads; normalization through a Bronze, Silver, and Gold pipeline; and access through a vendor-neutral retrieval interface with nightly snapshots and update alerts. It is the blueprint behind the open feed that the strategy guide described as the highest-reach strategy. It is documented in the Building a Federated Justice Knowledge Base memo, and it is a pattern rather than a running national service.
The OCR and data-extraction playbook. This is the cross-state working group's playbook for pulling text and structured fields out of documents, and it is the groundwork under the document identification and explanation task and under the OCR suite from the measurement guide. It covers what works on clean digital documents, where handwriting and state-specific local court forms break extraction, and the rules-first-with-AI-fallback pattern that proved most reliable. A team building any tool that reads an uploaded document starts here rather than from scratch. It exists as a working-group playbook, and its public home is still being stood up.
The PII masking memo and test set. This is the guidance and the test material for keeping private information out of places it should not be, which matters both for any tool that handles user inputs and for the internal templates and documents a content pipeline touches. The memo covers the approach, and the PII-bearing test dataset and its suite measure whether a tool catches and masks personal information, scored on recall, precision, and whether it identifies the right type of data. A team uses it to check a tool's exposure before deploying it and to set a masking standard for internal materials. The memo exists, and the dataset and suite are part of the evaluation set the measurement guide described.
How these fit together
A team that picks up these assets is not collecting curiosities. It is assembling the exact foundation the AI layer depends on. The standards make the content and the directory machine-readable. The datasets let the team build and test without real client data. The classifier and the publisher pattern handle the pipelines. The rubrics, the harness, and the safety wrapper handle quality and safety. Put together, they are most of the unglamorous groundwork that a single state would otherwise spend a year and a budget reinventing.
It is worth naming what this adds up to against what the grant first promised. The original grant proposed a single fine-tuned model, a pipeline to feed it, an evaluation of that one model, and a share-out. What the cohort delivered instead, and what the revised deliverables reflect, is this stack: four shared vocabularies, a library of build-and-test datasets, a classifier, an evaluation harness with a rubric library, a content safety wrapper, two playbooks, and a federation pattern. That is more than a single model would have been, and it is more durable, because a model goes stale and a standard does not.
What is not ready yet
Honesty about the catalog means naming what is not finished. The open, public homes for these assets are still being stood up, so for a stranger who never joined the cohort, several of them are not yet a click away. The datasets need a versioned public release with clear provenance and licensing before anyone outside the cohort should lean on them. The classifier is still being validated, the safety wrapper is still being built out, and the federated publisher is a pattern that no one is yet running as a shared national service. None of this makes the assets less real. It means that the distance between built and openly reachable is itself a piece of work, and that work does not belong to any single state.
One scope note belongs here as well. This catalog is the shared infrastructure the cohort built together, and it is not an inventory of the individual states' own projects. Several partners are building notable tools of their own, including retrieval systems and intake tools that go beyond what this catalog covers, and the cohort has not documented those and will not describe them on a partner's behalf. Any state team that wants its own project written up and shared with the field is invited to do that writing itself, and the field will host it. The point of the give-back is to lower the cost of the shared groundwork, and to leave each team the credit and the telling of its own work.
What comes next
This guide listed what the cohort built. The getting-started guide is about where a particular team should start with it, because a team of one person with a stale content library and a team of five with a structured directory should not pick up these assets in the same order. The national-leadership guide is about who stands up the open, public, maintained versions of everything in this catalog, since that is the part no single state can carry.