Stop paying to check the same evidence twice.
Published September 2, 2026Read first
What it saysWhen comparing systems that find documents, many results overlap. This method saves earlier judgments and reviews only new documents, then compares every system using the updated shared collection.
Why it mattersEach round of testing can make the next round cheaper.
Try thisCompare the documents returned by two search versions.
How to use it
Suggested steps
- Compare the documents returned by two search versions.
- Keep earlier judgments when the document and scoring rule are unchanged.
- Judge the new documents; compare both versions on the shared collection.
| Check | What to look for |
|---|---|
| Same document, same rule | Reuse the judgment |
| New document or changed rule | Review again |
Keep in mind Recheck judgments when the content or quality rule changes. Report uncertainty around close results.
A quick check for yourself
What is the practical lesson?
Show answer
Each round of testing can make the next round cheaper.
Source & why it’s here
Incremental Pooled LLM Evaluation (opens in a new tab)
Original issue reading order, preserved from the archive.