CLAIM-2026-10-14-03

The CRBugBench headline figure is a top-five measure over the agent's comments rather than a single verdict, and the paper reports agents weaker on complex changes, on bugs requiring reasoning beyond the local code, and noticeably worse on proprietary repositories where project context matters more.

Why it matters

Top-five is a recall measure, so the figure presumes a reader choosing among five comments. Quote it as a gate and you have assumed away the person the metric depends on.

Sources

• arxiv2026when