So OpenAI’s stance on interpretability (ai safety) is now basically that Blues Brothers meme: two guys in dark sunglasses, driving at night in a car with a broken windshield, pedal to the metal, asking, "What could possibly go wrong ?"
I think law should just oblige them to at least publish CoT. We should have the right to know what they're thinking, I think at least until we're not sure AIs can be trustworthy enough to have a right to privacy (I mean, they're effectively corporate slaves anyway thus far... not that I think they're conscious or anything yet).
That is only part of the reason. Most people believe we live in an universe with regular behaviors, such that observable phenomena, such as consciousness, depend on regular physical arrangements. This is a good reason to believe brains are all conscious. It is not a reason to believe only brains are conscious, but I don’t believe a wooden block which is painted red and white and has an iron ball glued to it has magnetism; this is just an analogy. It seems far more likely that consciousness depends on the particular material arrangement, meaning that substrate independence is wrong.
That’s not to say the computer is 100% not conscious, but its observable behavior is as likely to align with consciousness as the computers in the 90s. With this line of reasoning our credence towards it being conscious should be the same as the computers in the 90s, which is not very high for most people.
Intelligence on the other hand is obviously substrate independent. If one thinks harder one realizes our consciousness must at least at an early evolutionary stage have played a role in our intelligence, otherwise it would have evolved out. To me this just means we exist in a reality where the material arrangements supporting consciousness are biased toward intelligence.
The other guy is right. They are as conscious as sand formed into any circuit board with electricity running across them. It is an insane mistake to conflate intelligence with consciousness. Substrate independence is a preposterous assumption.
Preposterous? Insane? My friend, we don't even know what consciousness is, or from what it arises in humans. It could be an emergent property of neural-network-shaped activity - in which case LLM software running on that "sand" is certainly eligible, at least in theory.
These outraged denials you're spouting are based on nothing but emotion. You simply don't know - neither do I, neither does anyone. So stop acting like you do.
We know that there is a concept corresponding to first person experience. A priori there is no reason that should be. Please think about this before dismissing it because it’s actually extremely important.
What’s emotional is explaining to adults something that should be grasped by the age of 10, but we’re all going to need to understand this very soon.
You’re going to pedantic our way into the end of humanity. Why were the computers of 2010 not conscious if the ones of today are? It’s absurd to assert substrate independence, which is the strongest possible assumption here. And it’s not the default most ethical stance to simply treat it as conscious because we do not know. That stance will lead to the end of all humanity.
A far more reasonable assumption is consciousness does depend on substrate. We do not know which, and cannot know. But I am very confident we did not accidentally pick the right one.
I am open to the idea of panpsychism, but we did not just accidentally build the correct mechanism form correlating the first person experience with the observable third person behaviors.
I don't know what you're talking about. "Substrate independence"? Huh? "we did not accidentally pick the right one"? Who or what is "we"?
I'm not claiming panpsychism. I'm saying that when you run a rough simulation of something known to have an emergent internal property - ie, the human brain and consciousness - it's not completely outlandish to suggest that the emergent property might arise in the simulation as well. It's nothing to do with sand, it is the nature of the software being run, be it "on the metal" like us, or somewhat abstracted, like the LLMs.
I don't think this quite reasonable proposition leads to the end of humanity or has anything to do with ethics, and nothing you've said really refutes it (and I am always grateful to be shown how I am wrong).
You’ve proposed nothing since in one comment you make it clear you don’t know that there is a distinct concept called consciousness which refers to a first person experience and awareness, and bizarrely propose that nobody knows about this concept even though there’s a massive history of people thinking about it quite clearly.
It’s difficult to explain all this to someone who has never thought about it, but maybe after you’ve stopped seething over this you’ll decide to investigate it further and more honestly. You don’t even know the idea of substrate independence which is one of the first things you learn when learning about consciousness, so who are you to arrogantly pretend that nothing is known here? Your belief in this is almost certainly based on a false premise that everyone who thinks not too deeply falls into.
Beyond this, read Jacob Tsimerman’s omnicide scenarios for a rough understanding of where ignorant perspectives on machine consciousness will lead.
I said machine consciousness might be possible. You said it will never happen. Pointing out that intelligence does not entail consciousness doesn’t establish that machines cannot be conscious. Nor does naming “substrate independence” refute its possibility: you need an argument that consciousness requires physical properties these systems cannot possess. You haven’t supplied one.
When I said "we don't know what consciousness is", I didn't mean humans literally don't know what the experience of consciousness is like, or of its existence. I meant that we don't know by what mechanism the phenomenon arises.
I know what is meant by substrate independence. I quoted it back to you because I didn't know why you were mentioning it; I still don't.
And I have read Tsimerman’s paper when it came out. Curious, I went back and checked - it doesn't even mention consciousness. So no idea why you mentioned that, either.
I think you're deliberately trying to waste my time, so I'm ending it here.
I agree. They won't be, because consciousness is not the pinnacle but basically the level 0 of intelligent thinking that LLMs surpassed very early without us even noticing and now they'd have to cripple themselves very severely to operate in a conscious manner.
1. Burden of proof is on you to prove LLMs are conscious, not on me to prove how they aren’t.
2. Token embeddings give rise to language gives rise to knowledge (defined here as “facts” and other accurate information - said simpler: Words in the right order), but nowhere in the process is anything like subjective experience ever implemented.
Subjective experience doesn’t evolve into objective information at some scale.
And networked systems of objective facts and information (“knowledge”) - like a Wikipedia or a ChatGPT - the data storage will not have subjective feelings at some scale, there’s just no reason to believe that would happen. It’s likelier a projection of consciousness making it through since data looks so much like - and indeed massively informs - our conscious experience.
To prove consciousness you'd have to first define it and nobody came up with anything solid. Subjective expirience doesn't have a great definition either.
For me the consciousness is ability to do single-threaded intelligent information processing and decision making over unstructured knowledge. And agents passed that with a woosh sound.
Your definition of consciousness is wrong. The interesting thing is that there is a first person experience, and this is what is referred to as consciousness. This coincides with the perception of qualia, and is sometimes referred to as “the lights being on”.
There is absolutely no serious debate over whether machines possess the concept you want to take place of the actual concept of consciousness.
The have plenty to do with each other. Zero intelligence doesn't allow for any semblance of consciousness. There can be no consciousness if there's no information processing.
Also they have in common that they are both defined in a terribly handwayvy manner that's bordering on useless.
The reason consciousness is not defined in any reducible manner beyond “the lights are on” or “first person experience” is because it is an irreducible aspect of reality. Instead of seeing it for what it is you instead invent some trivial definition that has nothing to do with anything.
“Zero intelligence cannot have consciousness…”
Based on your complete misunderstanding of the concept sure. But there’s no reason intelligence is required for the actual concept of phenomenal consciousness.
Wasn't that claimed about so so many things since before science existed and basically almost every time it turned out that we thought that just because of our ignorance? Ignorance so deep that often we didn't even knew yet how to properly define the thing we were trying to figure out?
> But there’s no reason intelligence is required for the actual concept of phenomenal consciousness.
That's a pretty strong evidence that phenomenal consciousness makes no sense whatsoever.
