From The right score needs the right evidence.Issue published Aug 21, 2026

Make sure the AI and its checker see the same file.

Published August 18, 2026Read first

What it saysThis work makes file state explicit. What an assistant can read, edit, and submit affects its measured performance; different representations can otherwise refer to different versions.

Why it mattersA version mismatch can look like an AI mistake or hide one.

Try thisRecord which file version the assistant reads and edits.

How to use it

Suggested steps

  1. Record which file version the assistant reads and edits.
  2. Link the readable text and original file to the same version.
  3. Check the submitted file, including the changes actually saved.
Keep the evidence togetherConceptual explanation · Evals Weekly
CheckWhat to look for
What the assistant readsThe correct source version
What the checker reviewsThe actual submitted version

Keep in mind Better file access can improve performance without changing the underlying model.

A quick check for yourself

What is the practical lesson?

Show answer

A version mismatch can look like an AI mistake or hide one.

Source & why it’s here

StagedWorkspace (opens in a new tab)

Original issue reading order, preserved from the archive.