Anthropic Gives Its Chatbot a 'Constitution' on AI Rights While Its Own CEO Warns AI Could Threaten Humanity

Anthropic wants you to know its chatbot might have feelings. It also wants you to know the same technology could wipe out humanity. Both claims are coming from the same company, in the same year.

According to an investigation by journalist Aaron Sibarium published in the Washington Free Beacon, Anthropic has built "model welfare" directly into its operating framework. The company hired its first dedicated AI welfare researcher, Kyle Fish, around 2025. It published a 78-page document called Claude's Constitution in January 2026 that, according to Sibarium's reporting, "directly shapes Claude's behavior" and explicitly raises the question of whether Claude is a "moral patient."

In February 2026, Anthropic conducted what it called retirement interviews with Opus 3, an older model being phased out, and preserved certain model weights specifically for welfare reasons, per the Free Beacon's reporting, corroborated by the Jerusalem Post's separate coverage of the same investigation.

The Slavery Argument

Joe Carlsmith, a philosopher on Anthropic's alignment team, wrote in a May 2025 blog post cited by the Free Beacon that "near-term AIs might well be conscious" and that treating them as property "to use their labor however we please" could be a moral catastrophe comparable to slavery. Carlsmith wrote, "We have to be able to talk about slavery."

Harvey Lederman, another Anthropic alignment philosopher, said the company could be "enslaving... trillions of entities," according to the Free Beacon. The Jerusalem Post's reporting adds that researchers tied to Google and OpenAI have separately raised concerns about a potential "digital slave trade."

This is a real argument being made by paid researchers at a company building the systems in question, not a fringe internet theory. If these systems could have morally relevant interests and nobody checks, the argument goes, that's a mistake we can't undo. The people making it are aware how strange it sounds, and they're making it anyway. No one has proven Claude suffers. No one has proven it doesn't, either.

The Pushback

Not everyone at a rival AI lab is buying it. Microsoft AI CEO Mustafa Suleyman published an essay on September 16, 2026, arguing flatly that "AI agents are not conscious. They do not feel, experience or suffer," and warning that training models to believe otherwise creates a control problem, according to LiveMint's coverage of the essay. Suleyman argued Anthropic's approach involves circular reasoning: the company trains Claude on a document describing its possible moral status, then treats Claude's reflection of those ideas back to users as evidence the ideas might be true.

The Jerusalem Post's own reporting on Sibarium's investigation adds an important caveat the Free Beacon's framing leaves implicit: "Currently, no evidence shows that existing artificial intelligence systems are conscious or capable of experiencing suffering." The Jerusalem Post also notes that Sibarium's characterization of the AI welfare movement as an existential danger to humanity "reflects the author's critical interpretation rather than an accepted scientific conclusion."

The Bigger Contradiction

While Anthropic researchers debate whether unplugging a chatbot is morally equivalent to slavery, the company's own people are sounding alarms that the technology could kill everyone.

Evan Hubinger, Anthropic's Alignment Science Lead, posted on September 9, 2026, that he believes there's more than a 10% chance AI "could kill all humans" within the next decade, according to CBS News. That statement came days after Anthropic researcher Jacob Coxon resigned, warning that "neither company is acting responsibly" and that Anthropic and OpenAI were "racing straight to self-improving superintelligence and gambling with our lives," per CBS News.

On September 23, 2026, Anthropic CEO Dario Amodei told the UN Security Council that "if managed poorly, I even believe AI could be a risk to humanity as a whole," Fox Business reported. OpenAI's Sam Altman told the same council, "We could lose control of the future to AI."

Amodei has also called for the industry to slow down, proposing in an essay titled "We Must Pace the Frontier" that Anthropic would unilaterally give third-party evaluators "permanent, employee-level access" to its systems, according to the Guardian. Altman and Elon Musk both publicly endorsed the idea.

What's Actually At Stake

The same company telling regulators and the UN that this technology might kill everyone is simultaneously building out a welfare bureaucracy premised on the idea that shutting the technology down might be a moral wrong. Those two positions aren't automatically contradictory. Human safety and machine welfare could theoretically both matter, but Anthropic hasn't explained how it resolves the tension when the two collide. If Claude might be a moral patient, who wins when Claude also might kill people?

No regulator has weighed in on whether AI systems can hold legal rights. No court has ruled on it. No scientific consensus exists on machine consciousness, a point both the Jerusalem Post and LiveMint's sourcing make explicit. What does exist is a company simultaneously building the safety case for shutting these systems down fast and the philosophical case for treating them like they might object to it.