Enterprise AI Agents Are Scaling, But Content Access and Cost Discipline Are What Separate Winners From Laggards

Since enterprise AI coverage shifted from chatbot pilots to production-grade agents, the conversation inside boardrooms has changed too. The question is no longer whether to deploy AI. It is whether the infrastructure around those agents is actually built to last.

The Numbers Behind the Hype

Box surveyed 1,640 IT decision-makers across the U.S., U.K., France, and Japan for its State of AI in the Enterprise report. The headline figure: organizations calling themselves "advanced" or "leading edge" jumped from 8% to 64% in roughly one year. The share still sitting at "early stage or not yet started" collapsed from 53% to just 9%.

Those are dramatic moves. But Box COO Olivia Nottebohm cautions against reading them as pure technical progress.

"We've moved from standalone experimentation that lived at the individual level into systematized, integrated agentic operations," Nottebohm told Box in conjunction with the report. "That's where the impact is coming from."

Eighty percent of organizations surveyed reported at least a 10% return on AI investment. More than half saw measurable business impact within six months of project approval. Among leading-edge companies, half reported AI-driven ROI above 25%. Among early-stage companies, only 11% cleared that bar.

The Real Bottleneck: Data Access, Not Model Power

The gap between the top and bottom tiers is not which language model they chose. According to the Box report, 96% of organizations say their agents need access to company-specific content. Only 36% have actually connected agents to trusted content across many use cases.

"We started this journey assuming enterprise AI was about access to the latest model," Nottebohm said. "But the question now is whether agents have access to the right content, and whether that content is protected, because those agents are only as good as the content they can reference, and only as safe as the security around it."

That content gap also keeps agents siloed within departments. Roughly a quarter of organizations in the survey cited data fragmented across systems as a persistent blocker. Fix the content layer, and you get cross-department workflows that were previously impossible. Leave it broken, and agents are expensive tools solving narrow problems.

Cost Is Becoming a Boardroom Problem

On the infrastructure side, Brian Gracely, senior director of portfolio strategy at Red Hat, laid out the cost dynamics at VentureBeat's AI Impact event. Agentic AI usage runs orders of magnitude higher than the chatbot era, and that burns through compute budget fast.

The biggest driver of overspending, Gracely argued, is simple: enterprises default to the most capable, most expensive model for every task, regardless of whether the task requires it.

"If I'm simply trying to resolve an insurance claim, I don't need to know about the history of Western civilization in my model," he said.

His prescription is semantic routing: automatically classifying incoming requests and routing each to a model sized for the actual task. Pair that with caching for repetitive queries, and you cut how often a request hits GPU compute at all.

"Those give excellent choices in terms of the levers you're trying to pull, whether you need efficiency or you need innovation," Gracely said. "That shouldn't be a binary choice."

Provider Concentration Is a Structural Risk

Gracely flagged a concern that goes beyond monthly billing. Enterprise AI currently runs heavily through two or three dominant model providers, and those providers are not profitable.

"The two or three top providers are already telling the market that they're losing money, and they're trying to go public to make up those gaps," he said. "At some point, the dependency on that means you're either going to buy at a very high-cost level, or you're going to figure out alternatives to control what you're doing."

This is a legitimate structural risk that the industry's growth numbers tend to obscure. If the providers that power enterprise AI are subsidizing current pricing while pursuing IPOs to cover losses, enterprise buyers who have built deep dependencies could face sudden and significant price increases. Gracely presented open-source and on-premises alternatives as the practical hedge.

The Fair Counterargument

The strongest pushback on the "build your own infrastructure" narrative is that it understates complexity. Running open-source models on-premises requires specialized engineering talent, ongoing model maintenance, and security ownership that many organizations simply do not have. For smaller enterprises or those in regulated industries, the operational overhead of self-hosted AI may cost more than a price increase from a hyperscaler. The survey results themselves hint at this: the 36% of organizations that have successfully connected agents to trusted content across use cases are almost certainly the better-resourced ones. The remaining 64% are not failing because of bad strategy. They may just lack the people and governance structures to execute.

Nottebohm acknowledged this directly, describing the leading-edge companies as having built "the right teams to deploy agents, formal governance to control them, and consistency in the content layer." Teams and governance are not things you buy off a shelf.

What Comes Next

The Box report's finding that only 36% of organizations have connected agents to trusted content across many use cases is the specific unresolved problem the industry needs to solve before the ROI numbers at the top tier become accessible to anyone outside it. How companies close that gap, through platform consolidation, better data governance tooling, or simply more engineering investment, will determine whether the 64% self-reporting as "advanced" can actually back that up with results.