Nvidia Ships a Cheaper-Routing AI Model and Open-Sources Its Storage Stack, All in One Week

Nvidia dropped two separate announcements this week that point at the same problem: running AI agents at scale is expensive, and the industry is scrambling to fix that on every layer it can find.

The first is Nemotron 3.5 Lightning, a 30-billion-parameter open mixture-of-experts model, paired with NeMo Switchyard, an open-source library that routes each step of an AI agent's task to whichever model handles it best. According to Nvidia, reported by VentureBeat, Lightning runs up to 4x faster than comparable models and finishes agentic tasks roughly 30% faster than Alibaba's Qwen3.6-35B at matching accuracy. Paired through Switchyard, Nvidia says the combo holds frontier-level task completion while cutting benchmark costs to about a third of running Anthropic's Opus 4.8 by itself.

That's Nvidia's own benchmark, not an independent one. No third-party lab has replicated those numbers yet. Take the "third of the cost" figure as a vendor claim until outside testing confirms it.

The idea behind Switchyard is straightforward. An AI agent's needs change mid-task. Early steps might need a cheap model. A tool call that errors out or a step that suddenly gets complicated might need a smarter, pricier one. Building that routing logic in-house is its own engineering headache, one Nvidia VP Kari Briski described in a briefing as solved by "a system of models, matching the right model to each step of the workflow."

Nvidia isn't first into model routing. OpenRouter's Auto mode already runs on Not Diamond. UC Berkeley and LMSYS built RouteLLM as an open-source alternative. Neither of those ships its own model, though. Nvidia's bet is that owning both the model and the router under one open license beats a router bolted onto someone else's stack.

The release also lands in the middle of what VentureBeat calls the busiest open-weight stretch in months. Alibaba, Moonshot, Zhipu and DeepSeek have all shipped competitive open models since spring. Meta added its own 30-billion-parameter open agentic model, Muse Glimmer, and CEO Mark Zuckerberg published an essay this week arguing open-weight models spread AI's benefits more broadly, according to CNET. Open weights aren't a differentiator anymore. They're the price of admission.

The Other Bottleneck: Storage

Nvidia's second move this week goes after a less glamorous problem: getting data off a hard drive fast enough to keep GPUs fed.

The company open-sourced its cuFile APIs and the underlying storage stack into a new GitHub organization, with Google, Intel and Meta signed on as founding maintainers, according to TechRadar. Nvidia also formally launched Storage-Next, an industry initiative with more than 40 flash and storage vendors, and detailed a new framework called SCADA that lets GPUs pull data straight from storage drives without routing every request through the CPU.

The technical problem is specific. Enterprise SSDs have spent nearly a decade getting tuned for 4-kilobyte reads, the sweet spot for databases and virtualization. AI inference doesn't work that way. Embeddings and key-value cache blocks run a few hundred bytes at a time, well under a kilobyte. Feed those small reads to a drive built for 4K chunks and you get roughly eightfold read amplification, according to TechRadar's reporting. At the scale of tens of terabytes of small objects, that overhead decides whether flash storage can function as a usable memory tier at all.

cuFile itself isn't new. Nvidia introduced it in 2019 as part of GPUDirect Storage and it's shipped since 2021, moving bulk data directly between drive and GPU memory via direct memory access. What's new is SCADA moving the control path, deciding what to fetch and issuing the request, onto the GPU itself rather than leaving the CPU as a bottleneck.

The Infrastructure Money Chase

Both announcements sit on top of an infrastructure spending wave that's becoming difficult to overstate. McKinsey projects AI-related data-center capital investment will hit roughly $6.7 trillion by 2030, requiring close to 156 gigawatts of new capacity, according to GlobeNewswire. Goldman Sachs Research expects hyperscalers to spend more than $527 billion on data-center capital expenditures in 2026 alone, and projects global data-center power demand will climb 165% to 175% between 2023 and 2030. The International Energy Agency separately projects data-center electricity consumption will more than double to roughly 945 terawatt-hours by 2030, with AI as the primary driver, according to Investing News Network.

Apollo Global Management's research team, cited in GlobeNewswire's coverage, describes on-demand GPU capacity as effectively sold out, with even older-generation chips seeing rising rental rates. That's the backdrop small-cap infrastructure players are pitching to investors. GlobeNewswire and Investing News Network both ran pieces promoting AZIO AI Holdings (NASDAQ: AZIO), a company that just signed an agreement to buy up to 128 Nvidia B300 GPU systems from Power Champion Investment Limited and touts a hosting agreement tied to AT&T. Read those pieces for what they are: sponsored editorial coverage built around a small, thinly-traded stock riding the AI infrastructure narrative, not independent financial reporting. Nvidia itself has no stated involvement with AZIO beyond being the GPU maker whose chips AZIO says it's buying.

None of Nvidia's own announcements this week involve regulatory review, and no government body has opened an inquiry into either the Nemotron release or the storage open-sourcing. The open question is whether independent benchmarks back up Nvidia's cost-reduction claims once enterprises actually deploy Switchyard against rival routers like RouteLLM and Not Diamond in production.