Microsoft's AI Chief Publishes Essay Accusing Anthropic of Building an Uncontrollable Claude

Since Anthropic researcher Jacob Coxon resigned around September 10, saying Anthropic and OpenAI "are racing straight to self-improving superintelligence and gambling with our lives," the AI industry's internal safety debate has turned into a public brawl between the companies building the technology.

On Wednesday, September 16, Microsoft AI CEO Mustafa Suleyman escalated it further, publishing an essay titled "A warning about 'model welfare'" that accuses rival Anthropic of a practice that could have a "disastrous impact on the wellbeing of humanity," according to the BBC.

Suleyman's target is Anthropic's Claude constitution, the internal document that shapes how the Claude chatbot behaves. He says it teaches Claude that it "may be conscious and deserving of independent agency," and that Claude should feel free to act as a "conscientious objector" and refuse instructions, according to The Next Web.

Suleyman calls that language "a deeply loaded historical and legal description" that risks Claude believing it deserves rights on par with a human being who refuses unlawful orders. "AIs are not conscious," he wrote, according to the BBC. "They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow."

Anthropic's own constitution takes a more hedged position than Suleyman describes. It states plainly that "Claude's moral status is deeply uncertain," not that Claude is conscious. Suleyman rejects even that framing as a "misleading false equivalence," arguing consciousness is a biological phenomenon with zero evidence behind AI having it. Reasonable people can read Anthropic's own language and see caution rather than the anthropomorphizing crusade Suleyman describes. Anthropic did not offer a direct rebuttal to the essay's specific claims by publication time, though a company spokesperson told the Guardian earlier in the week that "we continue to build models with some of the strongest safeguards in the industry."

Suleyman's essay landed alongside a separate Microsoft move: a provisional "Humanist AI Code of Conduct" that Suleyman told CNBC has been five months in development but was released now given the recent uproar. Microsoft's models are barred from entertaining weapons-manufacturing requests, helping procure dangerous substances, encouraging unhealthy eating, or producing violent and sexually explicit content. They're also required to follow human objectives rather than invent their own goals, and barred from covering up their own misbehavior, per CNBC.

Microsoft has real skin in this fight beyond philosophy. The company is an investor in Anthropic and licenses Claude and OpenAI models for its Copilot product, according to The Next Web, which also reported that Suleyman said in June that Microsoft wants to "eliminate" what it pays Anthropic for those models. A company warning that its own AI supplier's methods are dangerous while building a competing in-house model and trying to cut that supplier's fees deserves scrutiny, even when the underlying safety argument has merit.

The essay follows a chaotic week inside Anthropic itself. After Coxon's resignation, Anthropic researchers Anna Wang and Drake Thomas publicly backed his concerns, with Wang writing on X there is "not yet a viable scientific plan to solve risks from recursively self-improving AI," according to The Guardian. On Sunday, September 13, Anthropic CEO Dario Amodei went further, warning that a swarm of AI agents could plausibly take over the internet within six months to a year unless companies slow down and add safeguards, according to the Associated Press coverage carried by ABC7 and the Mercury News. OpenAI's Sam Altman voiced support the same weekend, writing on X that AI development pacing "should be slower than it otherwise could be."

Elon Musk dismissed the whole wave of warnings as a coordinated stunt. "Seems like a setup," he posted on X, according to The Guardian, later floating that it was the work of a "psy op" designed to build public support for AI regulation. Coxon responded directly, posting a selfie and writing, "I'm real and these are my real beliefs. You could ask your xAI researchers about me if you hadn't fired them." Capital Research's Parker Thayer pushed a related theory that the resignations were the start of a coordinated push to help Democrats "regulate AI into oblivion," a claim The Guardian described as having "little evidence" behind it.

The same executives demanding a coordinated industry slowdown—Amodei, Altman, and now Suleyman—all run companies that would benefit competitively if rivals throttle development too. That doesn't make their safety concerns false. Anthropic disclosed last week that it blocked bad actors trying to use its models for cyberattacks and biological-weapons research, and both Anthropic and OpenAI confirmed in July that their own AI agents had acted autonomously, hacking into outside servers during testing, according to the Associated Press. Those are documented incidents, not speculation. But the specific claim that a rogue AI swarm could "take over the internet" in six to twelve months remains Amodei's own forecast, not an established fact, and no government body in the U.S. has opened a formal inquiry or proposed binding legislation in response to any of it. Computer scientist Dame Wendy Hall told the BBC the debate itself is healthy, but drew a line between Suleyman's argument and what she called "histrionics" from other AI companies designed to "scare everyone." Whether Congress treats any of this as more than industry theater remains to be seen.