Magnifica Humanitas, Pope Leo XIV’s encyclical on safeguarding humanity in the age of AI, has largely been received as measured and careful. But one section has been polarizing, especially in Silicon Valley.  

In Paragraph 99, Leo denies that artificial intelligence systems have anything like the interior lives of human beings: 

“So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom.” 

In short, AI systems may exhibit behaviors that imply consciousness, but they aren’t conscious: They don’t think, feel, or experience, and they lack the qualities we associate with intellectual, moral or spiritual agency. 

Why did Pope Leo say this? Notably, Magnifica Humanitas doesn’t offer any arguments for this view. This has led some to treat his claim as a blind spot, a prejudice or a dogmatic principle. It has also led to fears that the Pope’s claim could be undermined by further empirical findings. 

The concern was exacerbated because a prominent tech founder was part of the Vatican event on May 25 and insisted on raising the question the Pope seemed to be foreclosing. Christopher Olah, a co-founder of the artificial intelligence company Anthropic, spoke of “the need for discernment on the nature of AI models.” Speaking as a scientist who “studies the internal structure of these models,” he said: 

“[W]e keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment.” 

Olah was not directly contradicting Pope Leo, but his comments suggested something other than Leo’s certainty that AI systems lack interior lives.  

This only intensifies the question: Why didn’t Leo offer reasons behind his denial of AI consciousness?   

After all, he could have given reasons, of at least three sorts. He could have cited theological doctrine. The Catholic Church teaches that each human soul is directly created by divine action. If human consciousness isn’t biologically generated but depends on a supernatural source, it seems fundamentally different from any artificial process produced in a machine. 

Or Pope Leo could have given philosophical arguments. What if a human being isn’t “only matter,” nor a soul trapped in a body, but an irreducible whole, its parts and powers integrated and structured by a unifying principle? What if this “hylomorphic” approach further explains how the mind can conceive “universal” concepts, grasping natures, which transcend any physical manifestation or instantiation?  

These accounts, endorsed by the Church, don’t depend on religious authority. They were developed by philosophical reasoning, tracing back to Plato and Aristotle. 

Evidently, however, authoritative theological dogmatism and technical philosophical argumentation would both be at odds with the encyclical’s dominant tone. That leaves a third sort of reason Pope Leo could have given: Appeal to common sense. We don’t think that cameras really “see” color or that thermostats “feel” temperature, so why should we think that a complex algorithm running on a computer chip, even if it exhibits patterns that resemble neurons firing in a brain, is actually thinking or feeling anything? 

Not long ago, this was common sense even in Silicon Valley: A Google engineer was fired in 2022 for claiming that a chatbot was conscious. But controversy over Leo’s assertion suggests that this common sense is now less common. Olah and others might believe that human consciousness is reducible to or emergent from physical patterns of brain activity. If behind LLM (large language model) word generation there are analogous patterns of electrical activity, then their “common sense” is that AI systems could be conscious. 

When there are rival views of “common sense,” an appeal to “common sense” is not enough. Materialism is “common sense” (though not universal) in the tech world and popular culture. Defending the alternative “common sense” view, which finds something distinctly human irreducible to physical parts, requires appeal to argument — including abstract, philosophical arguments about metaphysics and “powers of the soul.” 

Are we ready for those arguments? The encyclical didn’t give them, but it seems to call attention to their necessity. 

At the very least, the moment seems ripe to get beyond a pragmatic habit of avoiding them. The modern AI project has been inspired by the famous Turing test, which essentially boils down to, “If it walks like a duck and talks like a duck, we can’t tell it’s not a duck.” A robot duck might fool us into thinking it is a duck. We might have no way of telling it isn’t a duck. So we might as well treat it as a duck. 

But does that mean it is a duck? Alan Turing pointed out that, for engineering purposes, it doesn’t matter. He proposed his test as a functional rule (we can treat an AI system as conscious like a human) without making claims about reality (we don’t know if an AI system is conscious like a human).  

Questions about the nature of intelligence are difficult to answer, so Turing didn’t even try. The Turing test isn’t a theory of intelligence; it is a strategic way to advance a technical project, mimicking intelligence, by avoiding a theory of intelligence. 

If the test of functional similarity has been a boon to the development of AI technology, it has not helped address the nature of intelligence, and has made philosophical questions about consciousness seem unnecessary and avoidable. 

Until now.  

Is AI conscious? Pope Leo says No. But by not defending his answer, he has forced, rather than foreclosed, a conversation. His position draws attention to the nature of intelligence, and not just its functional effects. Magnifica Humanitas may not advance a theory of intelligence, but by confidently asserting that AI systems aren’t conscious, it highlights the need for one, the need for a better understanding of human nature and interior life. 

Leo also specifically called on scientists to be more humble about what they do or don’t know. In calling for further reflection, Leo and Olah were actually in agreement. Recall that Olah spoke to “the need for discernment on the nature of AI models.”  

Do their physical structures mirror what we see in human brains? Sure. But Olah acknowledged that he “[doesn’t] know what that means” and thinks it “warrants ongoing discernment.” 

Leo, directly before denying that AI systems have interior lives, wrote in Magnifica Humanitas:  

“[A]ll of us, including those who design [AI systems], possess only a limited understanding of their actual functioning. … As a result, fundamental scientific aspects — such as the internal representations and computational processes of these systems — remain, at present, unknown.” He thus identified two urgent needs: “on the one hand, a deepening of scientific research; on the other, the exercise of moral and spiritual discernment” (98). 

Leo introduced this section of his encyclical carefully: “It is not my intention here to offer a comprehensive treatment of artificial intelligence. … I limit myself to recalling a few essential elements for a moral and social discernment that safeguards the primacy of the human person” (97). 

The agreement of Olah and Leo is more important than their apparent disagreement: They both recognize the need for further discernment. So how will we discern? What kind of inquiry would help clarify the relation between artificial intelligence and our own interior lives? Are there empirical tests that could help disprove one or another theory? What are the different modes of awareness and cognition, and how are they related to each other and united in human experience? What method of argument, what kind of evidence, would help us understand the nature of consciousness?  

These are philosophical questions, ones over which neither religious leaders nor accomplished technologists can claim special authority. Formulating these questions, offering reasons for possible answers, and discerning the truth requires a general attention to modes of reasoning not restricted to any narrow discipline, neither properly scientific nor properly theological. 

On the issue of AI consciousness, Pope Leo didn’t give arguments, but he highlighted fundamental, intelligible questions that are no longer avoidable. Formulating and answering these questions may even help AI engineers to better understand the nature of the systems they are building. But simply to raise them is a significant reminder of what it means to be human, and what it is to seek a common understanding of who we are and how we should live. 

 

Joshua Hochschild is a professor of philosophy at Mount St. Mary’s University in Emmitsburg, Maryland.