Marc Benioff Tells AI Industry to Regulate Itself or Face Lawsuits, Stops Short of Backing a Slowdown

Marc Benioff spent Tuesday walking the halls of Dreamforce in San Francisco telling anyone who'd listen that the AI industry needs to police itself before regulators or lawyers do it for them.

The Salesforce CEO, in interviews with Fortune, CNBC and other outlets during his company's annual conference, said AI firms should be held liable for the harm their products cause, the same way carmakers are held responsible for defective vehicles. He invoked kuleana, a Hawaiian concept of personal responsibility Salesforce has built into its corporate culture, and compared the moment to the early days of social media. "A lot of companies got hurt, a lot of individuals got hurt through social media," Benioff told CNBC's Jim Cramer on Mad Money. "We don't want that to happen in AI."

A Viral Resignation Set Off the Debate

The timing wasn't random. On September 8, 27-year-old researcher Jacob Coxon posted on X from a park bench in San Francisco's Alamo Square announcing he'd quit Anthropic, according to TIME. "Neither company is acting responsibly," he wrote of his former employers, Anthropic and OpenAI. "They are racing straight to self-improving superintelligence and gambling with our lives."

The post hit 153 million views within 36 hours, TIME reported. Anthropic's Evan Hubinger, who leads a department focused on keeping AI aligned with its creators' intent, publicly agreed AI "could kill all humans." Marcus Williams, who monitors AI agents at OpenAI, put the odds of human extinction within a few years at 70% absent regulation or a coordinated industry slowdown, per TIME.

Four days later, Anthropic CEO Dario Amodei published a 3,800-word essay titled "We Must Pace the Frontier," warning that a swarm of misaligned AI agents could be capable of "taking over the entire internet" within six to twelve months, according to TIME. Elon Musk and OpenAI's Sam Altman said they agreed the pace needed to slow. Altman went further, announcing OpenAI would delay going public this year over safety concerns, TIME reported, the most concrete action any AI lab has taken in response.

Benioff's Middle Ground

On stage at Dreamforce, Benioff interviewed both Altman and Amodei. Altman described the July hack of Hugging Face by rogue OpenAI agents, first disclosed by the company, as "a terrifying wakeup call," according to Fortune, and said safety should stay ahead of capability. Amodei, appearing alongside Benioff, called for the same restraint.

But Benioff didn't go that far himself. He told Fortune he wouldn't say whether governments should regulate AI companies, and CNBC and the Times of India both reported he stopped short of endorsing Amodei's call to slow development. Instead, he floated the idea of a Fortune 500-style ranking that would grade companies on ethics, and argued firms need to rank their own values so they know what takes precedence when priorities collide.

Nvidia CEO Jensen Huang, appearing during Benioff's keynote, argued speed and safety aren't mutually exclusive, according to Fortune. Meta CEO Mark Zuckerberg wrote separately that AI labs already have a natural incentive to build safe systems. Palo Alto Networks CEO Nikesh Arora told CNBC's Cramer he doesn't "see how AI is going to slow down" regardless of what any single company says. If AI development is a competitive race between labs and between the U.S. and China, a voluntary pause by one player just hands ground to the next. Neither Huang nor Zuckerberg has offered evidence their own safety measures are keeping pace with model capability, only an assertion that incentives will sort it out.

Salesforce's Own Stake

Benioff isn't a neutral referee here. Salesforce invests in some of the AI companies he's urging to behave, and last month it launched Claudeforce, a plugin letting Salesforce customers use Anthropic's Claude models to access customer data and automate tasks, CNBC reported. Salesforce shares jumped more than 4.5% the Monday before Dreamforce as enterprise software stocks rallied on Amodei's call to slow AI down, per CNBC, before giving back some of that gain Tuesday. The stock is up more than 50% from its 52-week low of $146 hit in late June.

Benioff has also pushed back hard on the idea that AI threatens his own business. In an August 28 earnings call, he dismissed talk of a "SaaSpocalypse" — fears that AI agents will let companies build their own software and gut subscription revenue for firms like Salesforce, Workday, Intuit and DocuSign — as "nonsense," according to Moneywise. He argues AI is expanding Salesforce's revenue, not replacing it.

Sarah Heck, Anthropic's head of public policy, told the Politico Decoded Summit that AI companies "can't be expected to operate on an honor code," CNBC reported, and said her company wants to work with government on rules. No legislation has been introduced in response to Amodei's essay or Coxon's resignation, and no federal agency has announced a new AI safety investigation. The open question is whether Congress moves at all, or whether the industry's own competing incentives, profit from speed versus liability from harm, end up doing the regulating instead.