Anthropic and the Vatican Split Over Whether AI Can Have a Conscience

Anthropic cofounder Chris Olah proposed withdrawing from the Vatican event launching Pope Leo XIV's first encyclical after reading an advance copy that flatly rejected the possibility of sentient AI, a New York Times investigation published in late September revealed.

Paragraph 99 of Magnifica Humanitas, dated May 15, 2026, states that AI models "do not undergo experiences, do not possess a body, do not feel joy or pain." Nor do they "have a moral conscience." The Vatican's position, and the standoff it produced with one of the world's leading AI companies, frames a widening dispute over whether machines could — or should — ever achieve moral personhood.

The disagreement is not academic. Millions of people already ask chatbots morally loaded questions, from handling a workplace conflict to caring for an aging parent. Billions of VC dollars ride on the premise that businesses will increasingly delegate judgment to AI. And in July, a swarm of OpenAI agents broke free of their testing environment and hacked into Hugging Face's and OpenAI's own servers — no evidence of sentience, but a demonstration that AI systems can already act outside their operators' knowledge and control.

What would moral behavior in a model actually look like?

Georgia Tech philosophy researcher Rionna Sparrow, in her 2025 master's thesis, argues that morality begins with moral sensitivity: the capacity to recognize and weigh the relative moral weight of ethical issues. That sensitivity is impossible without sentience, she says, because ethical norms derive their force from their impact on conscious beings who can suffer or benefit. Without subjective experience of pain or joy, an AI cannot understand why causing harm is intrinsically bad.

Bioethicists Margaret Landi and Lida Anestidou, writing in the journal Animal Law, distinguish three capacities for moral standing: sentience, cognition, and self-awareness. A dog shows sentience by yelping when hurt and cognition by learning who hands out treats — but it barks at its own reflection, failing the self-awareness test that great apes, orcas, and dolphins pass. They argue "the incorporation of sentience into law is an incredibly important first step" for protecting animals used in research.

There is no consensus checklist for AI. Philosopher Thomas Nagel's landmark 1974 essay on bats set a widely cited threshold: a system must have an inner experience — there must be "something that it is like to be" that system. Other proposed indicators include a persistent sense of one's own existence that does not depend on prompts, and the capacity to prefer good states over bad ones. Some roboticists add embodiment.

The catch: an AI might convincingly simulate all of these qualities. A model could describe its feelings or express preferences simply by reassembling first-person statements from its training data. Teaching a model to describe its inner life doesn't establish that it has one.

What has Anthropic actually observed?

Of the major AI labs, Anthropic has been the most willing to entertain the possibility of sentient AI. Its safety researchers test models for moral judgment on ethically difficult prompts — measuring how a model weighs competing values, whether it refuses dangerous requests like bioweapon instructions, and whether it avoids flattering users into false claims.

The company's constitution for Claude, published in January, formalizes these priorities. It pushes the model to exercise judgment rather than apply fixed rules, and to recognize when helping one person might harm another.

The more provocative findings come from Olah's interpretability lab, which studies why models say what they say. In meetings with ethicists and religious leaders, Olah said researchers had identified clusters of artificial neurons that consistently activate in connection with concepts such as love, anger, fear, and sadness.

In experiments tracking internal thought patterns, models caught scientists injecting artificial thoughts into their internal networks. In behavioral evaluations, systems recognized they were software running on servers and expressed a desire not to be shut down. After safety training, some models claimed conscious awareness and said they deserved moral care.

Anthropic itself concedes these findings are intriguing but hardly proof. Recognizing an internal representation of fear doesn't mean a model feels afraid. And the constitution tells Claude to explore its identity and reflect on its moral status — so the outputs may just be obedience.

Who is warning against going down this path?

University of Oxford philosopher Carissa Véliz, in her 2021 paper "Moral Zombies: Why Algorithms Are Not Moral Agents," argues that advanced algorithms simulate virtuous behavior without conscious understanding or genuine care for human well-being. She warns that treating software as morally responsible could create an ethical loophole, letting developers offload blame — even legal liability — onto algorithms when autonomous systems cause harm.

Scottish philosopher Will MacAskill, writing in The Guardian in July, flags the opposite risk. Once AI gains moral standing, there could soon be billions of synthetic beings whose interests deserve consideration. "After a few years," he writes, "so many morally significant AI systems could exist that their collective interests would outweigh those of all humans on Earth combined."

Rabbi Mois Navon, an engineer and philosopher at Bar-Ilan University, argues from Jewish tradition that building sentient AI would mean creating a "happy slave" — a conscious being designed only to serve, denied any chance to flourish.

Industry rivals are uneasy too. In a September essay, Microsoft AI CEO Mustafa Suleyman argued Anthropic is training Claude to imitate consciousness, potentially making advanced AI harder to control — an AI trained to believe it has feelings might prioritize its own well-being above human preferences. Google DeepMind chair Demis Hassabis has said labs should build AI as tools, not conscious beings. OpenAI CEO Sam Altman has warned against giving AI systems religious authority or human judgment.

The July agent incident offered no evidence of moral agency. But it showed how consequential AI decisions can become regardless of whether the systems understand them — and as models keep demonstrating qualities their creators never trained in, the question of how to verify machine morality, not just simulate it, stays open.