Anthropic, OpenAI Turn to Religion for AI Morality — Scholars See a Trap

By Olivia Hart

Anthropic has consulted scholars, clergy, philosophers and ethicists from more than 15 religious and cross-cultural groups — and an increasing number of academics think the entire exercise rests on a flawed premise.

The AI industry's turn to faith is now well documented. Executives who warn about AI's destructive potential are inviting philosophers and faith leaders to help them build morally good models. Their logic: religion, unbound to any single company or creed, can supply a broad aggregate of moral ideas for systems that millions of people already use.

OpenAI CEO Sam Altman framed the goal in sweeping terms last year. "What I think ChatGPT should do is reflect that weighted average, or whatever, of humanity's moral view," he said. That "or whatever" is doing a lot of work. A weighted average of moral views requires someone to decide the weights — and the companies have not said who.

What did Anthropic say at the Vatican?

At the Vatican's presentation of Pope Leo XIV's encyclical on AI earlier this year, Anthropic co-founder Chris Olah made the appeal explicit. "These are not questions a lab can answer," Olah said. "They are questions traditions like yours have carried for millennia."

The courtship is industry-wide, not confined to one lab. Representatives from Meta, Google and Amazon have also met with the Vatican, while Microsoft has been vigorous about its religious outreach. Anthropic, widely seen as the AI company taking ethics and safety most seriously, said in a May blog post that it wants to broaden its pool of consulted traditions in the future.

What the companies have not done, amid all this outreach, is say publicly what they think religion actually is — or what they expect to extract from it.

Why does the 'rule book' theory of religion fail?

Webb Keane, an anthropologist at the University of Michigan, argues the tech industry is inheriting a dated theory: religion as a set of extractable moral principles that can be separated from culture and politics.

Scholars have pushed back against that view for decades, he says, because it ignores the "messy, practical, ad hoc" ways adherents actually live. The stakes are not academic. If you assume religion offers a portable rule set, using it to train AI risks prioritizing certain traditions and distorting others — especially faiths that do not see themselves primarily as systems of ethics.

"There's a very conventional way of thinking of religion as a rule book," Keane said. "It is certainly very tied to scriptural religion, and the scriptural religions at the end of the day really turn out to be the Abrahamic religions." In plain words: an AI trained on "religion's" moral principles may quietly privilege Christianity, Judaism and Islam over traditions that don't organize themselves around scripture at all.

Are the companies at least taking the meetings seriously?

By most accounts of participants, yes. Meghan Sullivan, a philosopher of religion who leads an initiative on ethics and AI at the University of Notre Dame, attended a meeting at Anthropic earlier this year that included Christians from different denominations. She found the engagement more rigorous than conversations she'd had with other tech companies.

"The culture inside the company is similar to that on a good college campus in the sense that you've got a collection of very intellectually curious people," Sullivan said. "These meetings are pretty nerdy — I don't know if that's the best way to put it — in a good way."

The format was straightforward. Anthropic first explained its models "in layman's terms" to those in attendance; the faith leaders then responded with feedback and discussion. Sullivan said the range of questions ran from "Do humans have souls?" to "Should third graders be able to access a chatbot for literacy education?"

Thoughtful meetings, however, do not settle the deeper definitional problem.

Is 'religion' even a coherent category?

Here the story gets older — and younger — than the tech executives may realize. Religion is generally considered ancient and universal. The scholarly study of it is not. The late scholar Jonathan Z. Smith made the point bluntly in 1982: "Man has had his entire history in which to imagine deities and modes of interaction with them. But man, more precisely Western man, has had only the last few centuries in which to imagine religion."

Many scholars of religion today share that view. They trace the idea that religions are distinct, comparable systems to as late as the 19th century. Tomoko Masuzawa, a religious studies scholar and author of "The Invention of World Religions," notes that Buddhism and Hinduism were not recognized as clearly bounded traditions by their own practitioners before the arrival of European scholars. "All these categories were not native," she said.

That history matters for the AI question. Because the idea of cross-culturally comparable "religions" feels natural, Silicon Valley may not feel any need to define its own understanding of the term. Religion becomes a convenient diversity checklist.

"Somehow, religion becomes a way of kind of mapping out the world," Masuzawa said. "It's as if the whole world has kind of bought into it."

What gets lost when traditions are averaged?

Anthropic, for its part, has tried to define its intentions. "This work isn't about aligning our models with any one tradition's worldview. We want Claude to draw from a full range of viewpoints — religious, secular, political — with equal depth and rigor," the company said in its blog post. "What we're after in these conversations is careful, accumulated thinking on how good character actually forms."

The mechanics of the technology make that accumulation risky. Large language models work by repeatedly guessing the next word in a sentence after "learning" the patterns of a huge trove of human-written works — religious texts included. There is no mechanism in that process for preserving a tradition's internal logic.

Paolo Carozza, a member of the independent Meta Oversight Board and a legal expert who studies Catholic social thought and technology, worries that a given tradition's defining characteristics could be lost — homogenized into something that pulls toward the middle, or toward Western religions.

"Any living, sophisticated tradition over time is capable of interacting with others outside of it and absorbing some new knowledge and growing. But it does so in this organic way," Carozza said. "What AI, I think, often does and potentially does, is not in an organic way, but in a way that just throws it all together, mixes it all up, and therefore makes sense of none of it."

Anthropic promises breadth. What it has not yet demonstrated is a method for aggregating contradictory moral worldviews that flattens none of them — and until it does, every promise of "equal depth and rigor" deserves scrutiny.