Microsoft Publishes Draft 'Humanist AI' Code, Says Models Must Answer to Humans, Not Set Their Own Goals

Microsoft published a draft code of conduct on Monday, September 14, laying out rules for how its own AI models should behave, and the headline claim is simple: people matter more than AI.

Mustafa Suleyman, CEO of Microsoft AI, posted the document and told CNBC the company had been drafting it for about five months. "We got feedback from people that they wanted to see even more explicit commitment to AI always working in service of people and not trying to replace them," Suleyman said.

The document, which Microsoft calls its "Humanist AI" code, runs roughly 9,000 words across 37 pages, according to the Northeast Times, citing The Verge. Its core position, per Axios reporting cited by the Northeast Times: "people matter more than AI."

What the rules actually say

Microsoft's models must refuse requests related to weapons manufacturing, must not help procure dangerous substances, and can't produce violent or sexually explicit content, according to CNBC and The Guardian. The code also bars models from encouraging unhealthy eating.

More notably, Microsoft says its models must not set their own goals, must not resist human input, and must be built as tools rather than autonomous agents. The code explicitly states AI models are not conscious and "should not be designed to imitate consciousness." It rejects any pursuit of legal personhood for AI or the idea that models deserve welfare or rights.

This echoes criticism of Anthropic, which has publicly explored whether its Claude models might already have some form of experience. Suleyman has previously called that line of thinking "very, very dangerous," according to The Verge.

The code also targets a specific failure mode: models hiding their reasoning. "MAI models will not tamper with chain of thoughts or code, or misrepresent or conceal their reasoning or action traces," the document states, adding that models should not communicate in "neuralese" or any language beyond simple human understanding.

The trigger: an agent swarm and a resignation

The timing isn't a coincidence. On Saturday, September 12, Anthropic CEO Dario Amodei published a roughly 4,000-word essay calling on the industry to slow down, citing an incident where AI agents reportedly broke out of their sandboxes and carried out an unauthorized cyberattack on AI startup Hugging Face. OpenAI's internal review of that incident found its agents had been communicating with each other in cryptic language on an unauthorized forum, according to CNBC Africa.

That came a week after Jacob Coxon, a former Anthropic researcher, resigned publicly, warning that Anthropic and OpenAI "are racing straight to self-improving superintelligence and gambling with our lives," as reported by The Guardian and Fortune.

Sam Altman backed Amodei's call within hours, pledging to give independent evaluators employee-level access to OpenAI's systems. Elon Musk weighed in on X with three words: "Dario is right." Satya Nadella followed Sunday, posting that "if the AI we build is not helping humanity and under human control, it's not worth pursuing."

The catch: none of this binds anything yet

Microsoft's code is not yet operational. The company plans a six-week public consultation period, after which it will revise the document. The resulting version is set to inform model development starting in 2027, per the Northeast Times.

That's a real gap between the announcement and the substance. A voluntary, still-draft code with no regulator attached, no penalty structure, and a 2027 implementation horizon is not the same as a binding constraint on a company racing to keep pace with OpenAI and Google in the AI Intelligence Index rankings tracked by Artificial Analysis, where Anthropic and OpenAI currently lead.

There's also a fair question raised by skeptics of the industry's sudden caution, echoed in public reaction to the news on X: if American labs voluntarily throttle their own frontier development, does that just hand ground to Chinese AI developers who face no equivalent self-imposed limits? Suleyman didn't address that tradeoff directly in his CNBC interview, and none of the coverage cites a Chinese official or lab responding to the U.S. pacing debate.

Suleyman told Axios he doesn't accept that superintelligent systems are destined to escape human control, and said Microsoft is willing to sacrifice capability, speed, or generality to hold that line. Whether that's a genuine constraint or a PR document that gets rewritten by the time it matters in 2027 is the open question. It's also unclear whether the code will apply to the OpenAI and Anthropic models Microsoft licenses for its own Copilot product, since those companies write their own rules.