Nobody else ran it — Gemini 4 Argon claims first place on a benchmark with two metrics, and the duty that would ask you to supervise is fourteen months out

On 30 September 2026 Google announced Gemini 4 Argon, released first to trusted cyber defenders through its Fairwind Program, and reported first place on AutomationBench with a score of 51.3% alongside 77.9% on DeepSWE v1.1 and a tie for first on CWE-bench v1 at 68%. The figure does not stand alone: AutomationBench is two benchmarks sharing a name. Zapier's, whose paper (Shepard and Salimans, Zapier, April 2026) defines grading as programmatic and end-state only with binary pass/fail over 600 public tasks and roughly 500 endpoints across 47 apps, is topped on its own public leaderboard by Claude Opus 4.8 at 30.33%, and reported the best frontier models below 10% at introduction. Artificial Analysis's AutomationBench-AA scores the average share of each task's objectives completed without guardrail violations over 657 tasks, and its leaders sit at 71.3%, 69.5% and 68.9%. A score of 51.3% is therefore consistent with first place only on the strict metric, and Gemini 4 Argon appears on neither public board — so the ranking is today the vendor's own, unreproduced. The practitioner consequence follows from the grading rule rather than the ranking: where a task is scored on end state alone, a failed run has already written into a CRM, an inbox and a chat tool with the job unfinished, which makes replayability, per-system write scoping and end-state reconciliation the controls that matter rather than step logs. The regulatory shape is a gap: the Digital Omnibus, Regulation (EU) 2026/1744, moved Annex III high-risk to 2 December 2027, taking Article 14 human oversight with it, while Article 50 transparency has applied since 2 August 2026 with machine-readable marking from 2 December 2026 — a duty to declare the machine, not to check it. The limb that reaches a wrong record written by an agent is GDPR Article 5(1)(d) accuracy, which carries no deferral.

Post

A contact created, an email sent, and the deal stage exactly where it was on Friday. That is what a failure looks like when the grading is end-state.

On Wednesday Google shipped Gemini 4 Argon — to trusted cyber defenders first, through its Fairwind Program — and reported first place on AutomationBench at 51.3%.

Flat:

  1. The headline numbers are Google's own: 77.9% on DeepSWE v1.1, a tie for first on CWE-bench v1 at 68%, and #1 on AutomationBench at 51.3%.

  2. Here is the number that belongs beside it. Zapier's public AutomationBench leaderboard — 600 tasks, strict pass rate, 1.0 only if every assertion passes — tops out at Claude Opus 4.8 on 30.33%. When the benchmark was introduced in April, its paper put the best frontier models below 10%.

  3. Before you conclude frontier agents now finish half your business workflows: there are two AutomationBenches. Zapier's grades binary, end-state, no partial credit. Artificial Analysis's AutomationBench-AA grades the average share of each task's objectives completed without triggering guardrail violations, and its top three sit at 71.3, 69.5 and 68.9. A 51.3% can only be first place on the strict metric — and Argon appears on neither public board. The claim may be perfectly good. Today it is Google's word, on Google's run.

  4. And if your instinct is that a rule catches the gap: the duty that would ask you to supervise an agent is not live. Annex III high-risk — where the HR workflows in this benchmark would sit — moved to 2 December 2027 under the Digital Omnibus, Regulation (EU) 2026/1744, and Article 14 human oversight went with it. What binds today is Article 50 transparency, in force since 2 August 2026, with the machine-readable marking limb 62 days out on 2 December. That one makes you declare the machine, not check it. The limb that bites on a wrong CRM record is older and has no deferral: GDPR Article 5(1)(d), accuracy.

My Monday: make every agent write replayable, so a rerun is safe rather than doubled. One write scope per agent, per system. Reconcile the end state, not the step log — the step log is what the agent believed happened.

A benchmark score is a claim about a metric, and about who ran it.

Which of your numbers has a scorekeeper who is not the vendor?

#AgenticAI #EUAIAct #GDPR #AIGovernance #DevSecOps

First comment

Sources, the vendor's own post first.

Google's announcement, 30 September 2026 — the Fairwind Program, the benchmark table I quote, and the introductory pricing: https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-4-argon/

Axios, same day — the staged release to cyber defenders, and the Bloomberg line about internal testing that Google disputes: https://www.axios.com/2026/09/30/google-gemini-4

Zapier's AutomationBench repository. Read the leaderboard and the scoring note: pass rate is 1.0 only if every assertion passes: https://github.com/zapier/AutomationBench

The AutomationBench paper, April 2026 — 600 public tasks, ~500 endpoints across 47 apps, programmatic end-state grading, and the line that the best frontier models were then below 10%: https://arxiv.org/abs/2604.18934

Artificial Analysis's AutomationBench-AA — the other scoring metric, the average share of objectives completed without guardrail violations, and the board Argon is not on: https://artificialanalysis.ai/evaluations/automationbench-aa

AI Act Article 52 — notification to the Commission without delay and in any event within two weeks, and the list the Commission must publish under 52(6): https://artificialintelligenceact.eu/article/52/

GDPR Article 5 — paragraph 1(d), accuracy, and the duty to erase or rectify without delay: https://gdpr-info.eu/art-5-gdpr/

Cut for length: Argon also leads Gray Swan's indirect prompt-injection benchmark on Google's own numbers. That is the one result here I would most like to see someone else reproduce, because it is the only one whose failure mode is an attacker rather than a missed field.

Claims

Sources

Corrections

No corrections. Two namespace hazards caught during the run and handled rather than published: the cite key axios2026google already existed in the vault for a different Axios article of 18 September, so this edition's Axios source is keyed axios2026googleb; and the AutomationBench identifier was disambiguated against both the Zapier paper and the Artificial Analysis variant before any figure was used, because a keyword match on the benchmark name returns two different scoring metrics.

AI disclosure

The video for this edition contains AI-generated media: the narration is a synthetic voice, the cover art and any background imagery are generated, and the brand ident animation is generated. The script, the claims and the source verification are human-authored and human-checked. Assets carry IPTC DigitalSourceType and C2PA Content Credentials.