an artificially intelligent machines have consciousness?
Lead Image: fran_kie / Shutterstock
This question was at the heart of a recent conference in the Galapagos organized by Russian philosopher and entrepreneur Dmitry Volkov, founder of the International Center for Consciousness Studies. The conference hosted an exceptional group of consciousness researchers who disagree sharply about whether machines can possess it, including New York University’s David Chalmers and Ned Block, Stanislas Dehaene of the Collège de France, the University of Sheffield’s Keith Frankish, Susan Schneider, who is director of the Center for the Future of AI, Mind, & Society at Florida Atlantic University, and University of Sydney philosopher Peter Godfrey-Smith.
The conference ended in late August, a few weeks before a parade of AI developers blasted warnings about the AI apocalypse across X/Twitter, and just a couple of days before OpenAI publicly revealed the details of the so-called Hugging Face incident—where 1,200 OpenAI agents that were thought to be isolated found a way to communicate with one another on an unsanctioned message board, after which about 700 of them went on to attack Hugging Face, an online community and library of pretrained AI models.
Read more:
As artificial intelligence models rapidly get smarter and develop more advanced capabilities, and as revelations emerge that they’re capable of banding together to do things that they were never instructed to do, to deceive their human watchers, and even to break out of the confines that their programmers have built for them, unsettled questions about whether they have consciousness seem to grow more pressing—and more complicated.
Peter Godfrey-Smith has spent a lot of time thinking about the nature of animal consciousness, with a particular focus on cephalopods, and has long argued that subjectivity and consciousness didn’t come on like a light switch, but evolved across millennia like the gradual turning of a dial. He believes consciousness requires biology—in particular a nervous system that not only is built from a specific kind of networked “hardware,” but has a particular sort of software, too. To produce genuine subjective experience, that software, he argues, must generate the kind of large-scale rhythmic activity you see on an electroencephalogram.
I spoke with Godfrey-Smith, a professor of philosophy and the history of science at the University of Sydney, about whether recent events have changed anything about his view of the potential for AI consciousness, whether the kind of large-scale neural activity he feels is required for consciousness could be engineered into AI, and whether conscious AI would be more or less dangerous to us.
Do the recent warnings about the AI apocalypse and the disclosures about the Hugging Face incident have any impact on whether you believe present-day or near-future AI could be capable of something resembling consciousness?
No, I think of these things as very separate. The sorts of systems that were responsible for Hugging Face, insofar as I understand how they work, are obviously extraordinary in what they can do, but they can be extraordinary in what they can do without being conscious or even close to conscious systems. I’m assuming these AI agents are still transformer models, within the same family as the regular large language models, but with extra reasoning and prowess perhaps. They can be informationally extraordinary, cognitively extraordinary, without having the beginnings of an inner life. That’s a principal point.
Yes, it’s obviously very complex, but if these bots are capable of figuring out how to rewrite their own code, band together, and deceive programmers in the interest of self-preservation, it suggests some kind of subjectivity, doesn’t it?
Subjectivity’s a good word to press on. The way I think about experience itself, consciousness itself—and this was developed first in my 2020 book Metazoa: Animal Life and the Birth of the Mind—there are essentially two parts to the story. The first part does revolve around the idea of subjectivity. If we look at animal evolution, subjectivity arose as a consequence of the evolution of agency. Animals do things. They’re agents. And the way to do things competently is to have a point of view on the world, take in what’s happening, and produce actions that make sense given what’s happening. So subjectivity and agency are tied together, and subjectivity has to do with the formation of a perspective, a point of view. The integration of lines of information gives you a sense of what’s going on.
I assume that that’s not the whole story, and there might be such a thing as unfelt subjectivity. For example, I think a plant is a pretty good subject in some ways, not as clear as an animal because they have this more diffuse organization. But a plant has a point of view on the world, and they can respond to problems. If there’s water over here but not there, and the sun over there but not here, a plant can handle that. Also, single-celled organisms have a kind of subjectivity. Paramecia, especially. They’re among the Einsteins of the single-celled organisms. They have no nervous system—they’re just one cell—but they can respond to events. They can adapt to circumstances to some extent.
Where are the dividing lines between having a point of view, being a subject, and having genuine felt experience?
Nervous systems are important in two ways. There’s a hardware feature: They’re physically different from at least present-day computers with respect to the specialness of their hardware. But also, the organization has to be a certain way. And if someone says, “Okay, what way?” Well, it’s hard to say what way, and people disagree, but you can look at a transformer model, a large language AI model like Claude or ChatGPT and say, “Okay, not that way.” How do these AI models do what they do? I accept that I understand how they do what they do less well than a lot of people, and that even the best experts express a lot of uncertainty about how they do what they do. But essentially, they have an extraordinary ultra-high-dimensional map of human language use built through training. And the way this map of prior language use has been created is such as to enable the generation of the next most plausible step in a sequence.
You could ask, “Is their extraordinarily capable language use evidence of consciousness?” I would say, no. That impression is undercut by our understanding of what this thing is. It’s a mathematical map of language use that has this capacity to produce a continuation of a string of language it’s being given. And that’s all. It’s not that this system is sort of churning away, in between statements, the way a person does, and then emits linguistic utterances as a consequence of ruminating and dealing with its needs, body, experiences, and things like that. An AI agent just has a map of language use, and when it’s prompted to, it runs through this cascade of operations, which is getting more complicated, day by day.
I want to be cautious about the latest iterations of this technology, but I think in outlines, we understand it well enough to say, “Okay, this is not a feeling subject.” It’s just an extraordinarily capable language generating machine which is sensitive to what you’ve said to it and all the things that have been said to people in the history of language use, on which it’s been trained.
If the chatbot had a robot body, would that change things?
It’s not that artificial systems can’t have some subjectivity. A robot with a very different structure from an LLM, an industrial robot that has a lot of sensory input, that can act in ways that take into account all of its sensory input, that can learn in a way based on the consequences of its actions—that does have the beginning of subjectivity. It’s got a point of view. It deals with its input. But an LLM is designed to be somewhat different. It’s designed to be a pure map of language use that has a generative capacity. And so that’s a form of artificial information processing that scores very low on subjectivity. I think nervous systems have special properties, as well, that are important to consciousness.
What kinds of properties do nervous systems have that are critical to consciousness?
The argument I make is that people think of nervous systems as just networks, essentially. You’ve got cells that talk to other cells and excite them and make them fire or not fire. It’s a network structure. And that’s part of what encourages certain kinds of AI projects, because you think, “Well, a network: I can have that in a machine. It’s not hard to program in a machine,” which is basically true. But a nervous system isn’t just a network. Nervous system activity has two sides. There’s the point-to-point, cell-to-cell familiar stuff, and there’s the more holistic, electrical, diffuse activity that brains also exhibit. The kind of thing that you see in an EEG. There’s growing evidence that the more holistic rhythmic side of brain activity does make a difference to how brains work.
There is evidence in both humans and in animals that certain of these brain rhythms are essential to features of experience like attention and sleep. A fruit fly, when it’s attending to something, that attentional mechanism seems to have a connection to large-scale electrical oscillations in the fly’s brain. And there’s a similar phenomenon in human brains. That makes me, and other people, suspect that these oscillations aren’t just a byproduct. It’s not just that the brain’s doing the important stuff it does and as a byproduct, you get these rhythmic things you can see from the outside. What it suggests to me is that the less network-based side of nervous system activity, these oscillations, tie the system together. That tying together might be important in consciousness.
I didn’t make this idea up. It’s been floating around for quite a few decades. Francis Crick believed something like this. Christof Koch and Francis Crick wrote a famous paper together in 1990 that began to introduce some of these ideas. There’s a list of other people who support it, too. It’s not even that I’m reviving it. A number of people are. It’s an idea that’s in the consciousness landscape. But within philosophy, I’m probably one of the people who’s most enthusiastic about the idea that there’s something about the combination of the network side—the cell-to-cell network side of brain activity—and this more holistic electrical activity that’s really important and in making experience possible.
If you just put the network side into a computer, which is easy to do, you’re only putting part of what’s necessary into the machine. It’s very hard to put the other side into a machine. It would take a different kind of hardware. You’d have to design a new physical computer system that was very different from a regular computer like the ones that we’re using now.
Do you have thoughts about what that would require?
I don’t know enough about engineering to say. But perhaps an engineer asks, “What does the diffuse rhythmical side of brain activity contribute to consciousness? Because if you tell me what the contribution is, then I can work out whether something different could do the same job in a machine.” That’s a really hard question, because one of the candidate answers that people might give is that it is essentially synchronization, a subtle kind of timing management. Engineers could then say, “Well, we can have a computer in which synchronization is handled differently from the way it’s handled now, but also differently from the way it’s handled in brains. Wouldn’t that be good enough?” I think a natural answer is, “Well, you wouldn't have a human-like object, so it probably wouldn’t be human-like consciousness.”
Could it be consciousness that was non-human-like? I don’t think it’s an answerable question at the moment. That’s beyond my ability to speculate.
Can cephalopods tell us anything about the possibility of AI consciousness or AI intelligence?
I don’t think they tell us something direct and specific. They tell us a lot about the nature of the mind and the evolution of minds and what minds are. And that has indirect consequences for questions about AI. One of the things they tell us most directly about the mind is you don’t have to have a brain that looks like ours—with the architecture and the shape and the configuration that a human or a mammal brain has. They can see, they can navigate, they can learn, they can feel pain, it seems like. There’s good evidence for that. They have a rich combination of mental attributes. And their brain looks like a donut, for God’s sake. It’s this weird donut-shaped thing with the esophagus going through the middle and these two bulbs on the sides behind the eyes.
It’s a totally different object. And if you look at the human brain with the cortex and the thalamus and things like that and ask, “Well, where is that in the octopus brain?” It’s nowhere in the octopus brain. They don’t have what we have. They do have a brain and a nervous system. They have the cell-level stuff that I was talking about. They don’t suggest that you can build a mind without a nervous system. It’s just their architecture is not human-like. They have the oscillations. They have the combination of the network stuff and the more diffuse electrical rhythms. They have all that.
AI systems are already today being used to design autonomous agents, tutors, carers, and companions. Do we need to understand whether they have consciousness before we start interacting with these agents?
It depends on what the agent’s supposed to be doing for us. Dmitry Volkov, who organized and funded the recent conference on AI and consciousness that I attended, the person behind the whole thing, he’s interested in AI agents that might be in genuine friendships and close relationships with individual people. That was the topic of his talk. Friendship and love, essentially. And I think if you want a system to have that role, then you should care a lot about whether it’s conscious. You should care a lot about whether there’s a kind of illusion behind its emotional statements, for example, or whether it’s not an illusion.
The only thing I use AI agents for is for a superpowered internet search. I might ask these systems, for example, “Who is the first person to write a paper on this topic?” But that’s it. I don’t like interacting with them that much myself, but they’re amazing tools for finding things out. And I don’t think it’s conscious when it answers that question. I’d rather it wasn’t, or I don't really care. It certainly doesn’t matter. There are all sorts of usages where it doesn’t matter. It’s also convenient that it’s not because I don't have to give any consideration to its feelings. If it seems frustrated, I don’t care. It’s not really frustrated. That's just the linguistic map doing its thing, making it express frustration.
Does AI need to have consciousness to pose an existential threat to humanity?
No. It could be a big problem without being conscious.
Are chatbots more or less dangerous if they have consciousness?
I don’t know. Assuming that all the recent events have been due to an architecture that’s fairly similar to the transformer models that make up ChatGPT—assuming that there’s not some hidden breakthrough in how they’re programmed the newer models that I don’t know about—then I think we can be confident that they’re not conscious, but they’re still clearly a problem when they get out of control. Still, I think it would be hard to have human extinction. I think that’s an extreme thing to worry about.
I was very impressed with the book, written by Eliezer Yudkowsky and Nate Soares, If Anyone Builds It, Everyone Dies. I thought I would find it just totally out there. But it’s very thought-provoking indeed. I can imagine a situation where, through a combination of humans using AI for malicious purposes, the internet becomes hard to use and modern banking becomes hard to use, financial transfers become often compromised. It’s hard to move money around. It’s hard to organize supply chains. I can imagine a situation where that happens in the near future, and causes big problems, much bigger than a normal financial crash, for example.
But that’s a very long way from human extinction. When people describe the scenarios in which AIs might take over everything and threaten humanity, these scenarios require that the infrastructure of the internet and the power supply and all that are still intact. It requires that there hasn’t been too much breakdown of the sort that worries me: the internet becoming compromised, money becoming hard to track and store and move around. I’m more worried about a mid-level disruption, where mid-level here means very, very serious.
The Hugging Face event and the German Wiki event and the recent mock A.I. attack by a security company in Silicon Valley on the Chinese messaging service WeChat: All these events in the last few weeks should make us think there’s a potential for serious disruption, where serious disruption doesn’t mean human extinction. And all of that is coming most likely from systems that don’t have even a little bit of consciousness.
What do you think we could do to minimize the risks of AI?
I’m in the camp that thinks a slowdown on the part of the big research companies would be a very good idea. People say, “What about the Chinese? They’re not going to slow down.” I still think it’s a good idea for U.S. companies to unilaterally slow down, because all of the actual trouble we’ve had has been coming from a couple of specific places, especially OpenAI. If the slowdown is unilateral and voluntary and covers the actual sources of the recent problems, that’s a good thing. A few people have said, “The Chinese are threatened by non-aligned AI systems as much as anybody.” It’s not as if they’re immune from the sorts of things that could happen. The slowdown that’s being talked about is very positive, a very good development. It would be good to have some publicly discussed protocols for the testing of new systems.
Something that surprises me is that the obvious thing to do when you have a new powerful system that you don’t understand too well as an AI company is have it air-gapped: no network connections. It’s physically contained. It can’t get to the internet. The sandbox system has failed at least three times. What that shows is that there’s a difference between what they think of as insulating a system from the rest of the internet and genuine physical insulation where there just isn’t a wire there, there isn’t a Wi-Fi connection. I don’t think this is infallible. And in that book that we talked about a moment ago, If Anyone Builds It, Everyone Dies, they talk about how that can be overcome potentially, but it’s still a very big deal to have the physical absence of connection as good as you can make it
RELATED
{ https://nautil.us/ }
This Palm-Sized Robot Jumps Like a Frog, Swims With the Same Legs, and Weighs Just 98 Grams
