CLAIM-2026-08-16-06

LangChain's Switchyard benchmark ran 145 multi-step agentic tasks averaging 6.3 model calls each, routing between NVIDIA Nemotron 3.5 Lightning and Claude Opus 4.8, with Gemini 3.1 Flash Lite as judge.

Why it matters

Sets the scope of every number quoted from it: 145 tasks with an LLM judge, not a production workload.

Sources

• langchain2026how