A ticket that says "bumped the dependency, tests green, closed" is not evidence the vulnerability is gone.

A benchmark published last week measured exactly that gap, and the split in how agents fail is worse than the headline score.

VulnBump (arXiv:2610.17374, 12 October) takes 307 dependency-upgrade tasks built from 2,066 fixable vulnerabilities across 43 repositories and six package managers. What makes it useful is the scoring. An upgrade passes only if, in a fresh container, all four hold:

1. Dependencies resolve. 2. A rescan confirms the targeted vulnerabilities are actually fixed. 3. The project still builds. 4. Every previously passing test still passes.

A non-agentic baseline that just applies the recommended version passes 46.9%. Claude Code unaided passes 59.6%. Hand the same agent a scanner and it passes 77.5% — 17.9 points from tooling alone. A purpose-built remediation agent tops it at 90.2%.

The failure modes split, and that is the part worth keeping. The general-purpose agent mostly leaves the vulnerability in place. The specialised one mostly breaks the build. One fails loudly. One fails green.

Now the regulatory reflex, because it points the wrong way. The Cyber Resilience Act is usually invoked as the reason to automate remediation. But the CRA's substantive vulnerability-handling obligations do not bind until 11 December 2027. What has been live since 11 September 2026 is Article 14 — the duty to report actively exploited vulnerabilities.

So the only CRA clock running today rewards knowing, not fixing. An agent that marks a vulnerability resolved when it isn't does not merely fail to fix it. It corrupts the input to the one duty you already owe.

Three things I would change Monday:

1. Put the scanner inside the agent's loop, not after it. 17.9 points is not a prompt tweak. 2. Gate the ticket on a clean rescan, never on a green suite. 3. Treat a refused rescan as a reporting signal, not a retry.

One caution on the headline number: the benchmark draws its task distribution from one vendor's production data, and the agent that wins is that vendor's own. The method is strong. The league table is theirs.

Which of the four checks does your pipeline actually run today?

#AgenticEngineering #CyberResilienceAct #AppSec #SoftwareSupplyChain #EUTech