The sector is not the test — agents clear a quarter of real professional work, and Annex III classifies by neither healthcare nor finance

DAYJOB is a benchmark of 130 tasks built by working professionals, split 50 in healthcare and 80 in finance.

Why it mattersIt is the first long-horizon benchmark whose tasks were authored by the people whose job it is, rather than derived from public datasets. That is what makes the scores load-bearing.

arxiv2026dayjob · surgeai2026dayjob

The strongest configuration cleared 24.7% on DAYJOB healthcare and 23.9% on DAYJOB finance.

Why it mattersA ceiling under 25% on real professional work is the number to quote when someone proposes putting an agent in front of a client deliverable unsupervised.

arxiv2026dayjob · surgeai2026healthcar · surgeai2026finance

Across 30 model configurations from 13 developers, the median configuration cleared 0.6% in healthcare and 2.5% in finance.

Why it mattersThe gap between the leader and the median is the whole story: this is not a capability that arrived across the field, it is one configuration pulling away. Model choice is not interchangeable here.

arxiv2026dayjob

On the live DAYJOB healthcare board the leader at 24.7% is more than double the runner-up at 11.6%; on finance the top four sit within 4.1 points.

Why it mattersTwo different shapes of competition in one benchmark. In healthcare a swap costs you more than half your score; in finance it costs almost nothing.

surgeai2026healthcar · surgeai2026finance

DAYJOB prompts average 81 words in finance and 49 in healthcare.

Why it mattersThe short brief is the test. If your evals hand the agent a fully specified task, you are not measuring the part that fails in production.

surgeai2026dayjob

DAYJOB grades each task against a rubric carrying a median of 57.5 binary criteria in finance and 47.5 in healthcare.

Why it mattersBinary criteria at that density is a completeness test, not a correctness test. It is directly copyable into an acceptance gate.

arxiv2026dayjob · surgeai2026dayjob

DAYJOB grading is performed by an agentic LLM judge running inside the task container; the harness is Apache-2.0.

Why it mattersThe grader is itself an agent, which is both the reason the benchmark can scale and the reason to read the scores as indicative rather than exact.

github2026surgeaida

AI Act Annex III point 5 is written as use cases about natural persons: 5(a) public authorities evaluating eligibility for essential public assistance benefits including healthcare, 5(b) creditworthiness of natural persons excluding fraud detection, 5(c) risk assessment and pricing for life and health insurance.

Why it mattersThis is the sentence that decides whether an agent in a bank or a clinic is high-risk. Neither 'healthcare' nor 'finance' appears as a sector anywhere in Annex III.

euaiactexpl2026annex

Article 6(3) takes an Annex III system out of the high-risk category where it performs a narrow procedural task, improves the result of a previously completed human activity, detects deviations without replacing human review, or performs a preparatory task — and never where it performs profiling of natural persons.

Why it mattersThe preparatory-task limb is the one most agent products actually sit on, and profiling is the condition most of them quietly fail.

euaiactexpl2026classific

Article 6(4) requires a provider relying on Article 6(3) to document that assessment before the system is placed on the market or put into service, to provide it to national competent authorities on request, and to register under Article 49(2).

Why it mattersThe duty is keyed to placing on the market, not to a calendar date. A team that plans to write the assessment 'before December 2027' has misread the trigger.

euaiactexpl2026classific · euaiactexpl2026registrat

Article 49(2) requires providers who have concluded their Annex III system is not high-risk to register themselves and that system in the EU database referred to in Article 71.

Why it mattersConcluding you are out of scope is itself a registered act. There is no silent exit from Annex III.

euaiactexpl2026registrat

Regulation (EU) 2026/1744, the Digital Omnibus on AI, is dated 8 July 2026 and entered into force on 27 July 2026.

Why it mattersThe citation to use when someone quotes a 2025 article on AI Act deadlines.

eurlex2026regulatio

Under the Digital Omnibus, Article 50 transparency applies from 2 August 2026, the marking obligation from 2 December 2026, Annex III high-risk obligations from 2 December 2027 and Annex I from 2 August 2028.

Why it mattersFour dates, two of them already live. Most published commentary still runs the pre-Omnibus timeline.

huntonandre2026eu · eurlex2026regulatio

The Bundesnetzagentur announced on 29 July 2026 that it takes the central role in implementing the AI Regulation in Germany, under the KI-MIG in force since the same date.

Why it mattersFor a Berlin team it names the addressee: the Article 6(3) assessment is a document the Bundesnetzagentur can ask to see.

bundesnetza2026bundesnet · bundesamtfu2026kimig

The DAYJOB paper was submitted to arXiv as 2610.01306v1 on 1 October 2026.

Why it mattersIdentifier and date verified together, against the story rather than the topic.

arxiv2026dayjob

DAYJOB finance tasks span corporate finance, banking, credit, investing and real assets; healthcare tasks span clinical, pharmacy, operations, payer and compliance.

Why it mattersThe domain lists are what show that only a minority of the tasks map onto a named Annex III use case — payer and consumer credit do, corporate credit and clinical analysis do not.

surgeai2026finance · surgeai2026healthcar