Test every layer that makes the decision.

What it saysSANE reads a judge’s specification and creates controlled changes that should preserve or move its score. Across five criteria it produced 77 validated test families versus 21 for a fixed-library baseline. Its faithfulness tests ranked 11 judges similarly to a human-labeled benchmark (Spearman correlation 0.852 versus 0.100).

Why it mattersAverage agreement can hide brittle behavior. A targeted test tells you whether a judge reacts correctly when the exact thing it claims to measure changes—or stays the same.

Try thisChoose one judge. Write one change that should preserve its verdict and one that should reverse it, then run both before the next release.

What it saysOn 236 human-labeled responses, three rule-guided judges reached 89.4%–95.3% agreement, compared with 80.5%–87.3% for generic prompts. Each prompt’s scoring guide separated prohibited behavior, allowed exceptions and usefulness instead of asking for one broad safe-or-unsafe verdict.

Why it mattersA named policy category is not enough guidance for borderline cases. Concrete rules and exceptions make disagreements easier to diagnose and reduce the chance that caution simply replaces usefulness.

Try thisFor one high-risk intent, write the prohibited behavior, valid exceptions and minimum useful response as separate checks.

What it saysDuring a three-week high-access assessment, external evaluators found several routes around a historical agent-monitoring system, including one that could hide hundreds of recent actions and another caused by differences between the context seen by the agent and monitor.

Why it mattersA strong classifier is only one link. Routing, serialization, logging, tool coverage and isolation can silently decide whether the monitor sees the event at all.

Try thisFor one day of agent traffic, reconcile every eligible event against monitor logs and investigate every missing record.