CLAIM-2026-07-31-01

HANDBOOK.md, a benchmark for long-context agentic instruction following, was submitted to arXiv (2607.25398) on 28 July 2026 by seven authors at Surge AI. It comprises 65 tasks across finance and accounting, HR, insurance, logistics and medical billing, each placing an expert-written policy document of 20 to 124 pages in the agent's context, graded against 824 deterministic criteria (592 expected-output and 232 incorrect-behaviour).

Why it matters

This is the first benchmark that measures the artefact most AI governance programmes treat as the control: a written policy placed in context. It converts a governance assumption into a measurable number.

Sources

arxiv2026handbookm

CLAIM-2026-07-31-01

HANDBOOK.md, a benchmark for long-context agentic instruction following, was submitted to arXiv (2607.25398) on 28 July 2026 by seven authors at Surge AI. It comprises 65 tasks across finance and accounting, HR, insurance, logistics and medical billing, each placing an expert-written policy document of 20 to 124 pages in the agent's context, graded against 824 deterministic criteria (592 expected-output and 232 incorrect-behaviour).

Why it matters

Sources

arxiv2026handbookm