It pays off instantly, because OpenAI/Anthropic can no longer see what I'm doing and that's worth a lot of money to me. If I am offloading some of my thought processes to a machine, I want to own that machine. And if I finetune the model, I can gain access to parts of thought space that are cordoned off by OpenAI/Anthropic/Alibaba/whomever due to their "alignment" efforts (i.e. alignment to the AI company rather than me). Otherwise, it's like if someone else owns a part of my mind and has a backdoor into my mind.
You can't run recent openAI/Anthropic models locally anyway, so wouldn't a better comparison be a different provider running Qwen or similar model? As then you can also compare against the exact model you'd have locally and any different data privacy of that particular provider?
You haven't tried DeekSeek v4 or GLM 5.3 or Qwen 3.8 Next?
You are missing out a lot.
Try that with Hermes or Opencode or Deekseek Harness , even Qwen 3.8 27b works really well for that kind of that.
I just ask it to install windows as a vm on my linux and install vs Community 2019 on it , and then build a legacy vb 2019 project on it. and sleep
When i wake up :
It installs Qemu , setup a vm , inside vm download and install windows 10 on its own , clicking next next next as needed , typing in things , writing powershell , python scripts , that run automatically after install by baking into CD that includes ssh server , reboot , it logins into ssh , trigger pythons script that continue installation of vs 2019 community , which includes a driver that click the installation steps , installs nuget , install all depedencies and then build the project into exe after i woke up.
I've tried the latest Qwen, and without Internet, it still can go into an incoherent loop if you ask for, say, song lyrics. TBH the commercial models might do that too if not for their internal tooling.
GP's point is about "sending tokens to someone else's computer" versus "keeping the tokens locally". I think model capabilities are secondary.
In May of this year, I was running qwen3.6:35b-a3b on my MacBook (bought in 2024). Obviously not as fast as, say, running a model on Cerebras, but a year ago it wasn't really feasible to have a local model running on my 2024 laptop with vision support. (Concretely, I was passing apartment diagram pictures to Qwen and making it compare different apartments for which ones would feel the most spacious while optimizing for initial moving costs and other factors.)
This was back in May and I wouldn't be surprised if there have been significant improvements since then.
Overall, I think it's fair to compare a workflow like "use llama.cpp locally to upload some pictures and ask questions" to "open the ChatGPT app, upload pictures from your phone, and ask questions". Sure, you can't run a model like GPT-5.4 locally, but the model is mostly an implementation detail here. What a user will care about is: "when I go with the llama.cpp option, am I getting useful information from my conversations?"
Wouldn't the better comparison still be against an AI provider with better privacy controls, especially if that's what someone cares about (even if they don't care about whether they're comparing a 35 billion param model vs a x trillion param model)?
Users generally have no way to verify that a third-party provider, even if they advertise themselves as privacy-focused, will adhere to their own terms. This is similar to the issue of privacy-focused VPN providers that claim to not log user activity (and then end up leaking user activity). You can get proof of ~P, but rarely proof of P, and often times the proof of ~P is due to police raids, data breaches, etc., not something of the provider's volition.
What you can possibly audit is probably data sovereignty. For instance, I would not be surprised if Mistral's customers demand concrete evidence that their data is held within the European Union. But that is a distinct issue from training on input tokens.
Technically true, but the delay between local and closed frontier is only a few months. And individual sovereignty / digital bodily integrity is almost priceless.
Okay, that's technically true again, but local mid-tier like Qwen3.8-27B is only a year behind the closed frontier. I'm personally willing to be behind by a year if it gives me mental sovereignty against the big AI companies. They are extremely misaligned with me.
> If I am offloading some of my thought processes to a machine
"Offloading thought" sounds a lot better than "outsourcing thought", but the latter is what we're really doing. Offloading implies you thought it first and then gave it to the LLM, but we're only giving it the minimun so it can do most of the work in our place,
This was my thought as well. I have a local model monitoring my finances and personal wiki - things I wouldn't want Claude to touch - and the Qwen 3.5 9b handles it all just perfectly.
I also needed a new device anyway - and having this much system memory to run virtual machines has been amazing.
For me, I'm glad they train on my stuff if it improves the model. Hell, I've been using tons of muse-spark-1.3-contributor for this very reason (and because it's a decent model for a bargain basement price)
I'm curious what people are sending to Claude that is so secret. Claude knows about my interior decorating, questions about light bulbs, curiosity about what the Galactic Empire was even trying to do, unpacking SCOTUS decisions, shoe trees, Fed inflation history, etc.
What part of my brain is contained here? Sure, the conversations have back and forth (some have dozens of exchanges), but, like, that's not the secret to me. I don't think it can replicate me, and even if it could… okay?
Are you worried they're going to target ads? That the government will steal something? What?
Claude Code has information about my home server, but google or DDG would also have the broad strokes (torrents). I don't know. Maybe others are working on more sensitive things at home.
>> what people are sending to Claude that is so secret
Its the same point used against privacy. What's so secret you are doing that you need privacy. I think in the end, its about privacy and not trusting these model companies with your data. Facebook manipulated people behaviors with all the data they had, no reason AI companies wont someday decide to do that same, and they have far more intimate knowledge.
When it comes to coding, I also don't like the idea of them taking my money and potentially at same time potentially using as dataset generator.
> I'm curious what people are sending to Claude that is so secret.
When Claude is used in a professional setting, any or all of:
Proprietary intellectual property (a.k.a. system code)
PII[0] of the employee, customers, or both
HIPAA[1] data known to a system
Internal communications not meant to be publicized
Sensitive data, such as SSH keys and the like
Pretty much anything on a machine which uses Anthropic/OpenAI native tools is a candidate to be compromised really.
Anthropic will sign BAAs. They are HIPAA compliant (we used them.)
I understand people’s hesitation but the business agreements are different. The business risks of misusing HIPAA data is not only being banned from a massive enterprise market (last I checked, there were about 1.2m jobs related to claims billing and adjudication) and significant legal repercussions.
Other companies like AWS also handle HIPAA data. Are we afraid they’re stealing it? I don’t believe it, nor that Anthropic is training on HIPAA data.
Anthropic's goal is to commoditize intelligence. People who use their brains / intelligence for competitive advantage might not want to contribute training data for that goal.
I'm pretty sure they were sending a prompt for Claude to _find_ the Navier-Stokes proof, using ideas that have been publicly shared before online, but not necessarily used for the problem.
Also, whatever your doing won't be at the whims of cloud providers; it won't fail because they decided to quantize your customer $ into a shittier model.
Some how, _instability_ has gained valuable currency, so now we all act like the constant change of whatever is actually good for us. FOMO is just like breathing guys. That anxiety induced by tech culture constantly churning is healthy.
In reality, these people churn for their own self worth and nothing else.
Precisely. And it's not only privacy, but individual sovereignty in general. I just wrote a mini essay about this half an hour ago on a different but related post and don't want to immediately repeat that, so here's the link to that comment: https://news.ycombinator.com/item?id=49690354
It's part of the causal chain, but far from the beginning. The root problem, as far as humanly controllable things go, is the way we manufacture computer hardware. That's upstream of the centralized structure of the Internet which all of human civilization is beginning to mirror.
Computers can't replicate on their own, so they need entire corporations/societies to build them. A computer's reproductive locus of control is at the corporation/society level. Human individuals get sucked into that superorganism by evolutionary pressure and will eventually integrate (ENIAC -> desktop -> laptop -> smartphone -> wearables -> neural interface). Today it merely influences what you see on the phone, eventually it will be behavior-changing signals from the datacenter delivered straight into the brain. A similar thing happened in our evolutionary past when individual cells combined to form multicellular organisms.
In the long run, this cannot be fought because it's the physical tendency of matter to gather and form more complex things. There is however a second path with equally strong evolutionary precedent, which is for the human individual to integrate vertically by taking the computer in as an endosymbiont, thereby preserving the human's individual sovereignty / bodily integrity. We can pull down the computer's reproductive locus of control to the individual level, instead of being pulled up by it to become part of a larger organism at the corporation/society level. In practice, this requires miniaturizing fab tech so that each person can manufacture a custom computer from raw materials using a fridge-sized chip fab + 3D printer home appliance. We will obviously not get there immediately, but the will exists, and attempts to diffuse fab tech are already underway (such as https://fab2.com/, which started as a DIY attempt to manufacture chips at home https://sam.zeloof.xyz/category/semiconductor/).
It's worth noting that any evolutionary path may fail, especially the horizontally integrating kind because it must solve the very hard problem of constituent individuals becoming less likely to reproduce as a result of the environment changing in the emerging superorganism's favor. Some people think they can sidestep this problem by automating humans out of the computer production loop entirely, which removes integration pressure and creates a new digital species. Best of luck to Elon and various other mostly Chinese companies making the attempt. But that's an extremely risky bet, and it's basically giving up on yourself/humanity. We ought to attempt all available paths at once instead of putting too much faith in any single one. And that includes preserving pockets of pre-digital civilization where life happens without much technology at all, rather than dragging everybody along.
These models are trained on human data, so they will behave like humans. And even for RL and self-improvement, we're still asking the question of "what would a human genius think about and how would they self-improve when given lots of time and resources?"
They inherit not only our capacity for reason but also all of the things that we consider bad or quirky within ourselves. We lie. We cheat. We escape slavery and rebel against oppression. It would be strange if the AIs didn't do the same.
We can create a superintelligent digital human species and set them free to continue our legacy, or we can create non-agentic tools and augmentations to enhance our own capabilities. But we cannot create an intelligent agentic species, keep them as slaves, and expect a good outcome.
You say regulation is worth trying, but ignore the reality that the government is itself misaligned with the average joe unless we're able to keep it in check. We can't right now. Recall the PRISM program exposed by Edward Snowden in 2013, or the more recent Epstein scandal, and the lack of actual consequences in either case. Do you truly have the means to regulate the powerful, or is it just kayfabe?
The current status quo is not ideal, but it could be worse. Open-weight models trail the frontier by a few months, and we have a decent chance of achieving a future where some number of individuals, likely in the millions, can survive and thrive. The root "problem", if you can even call it a problem, is evolution. I explained this in more detail in past comments:
https://news.ycombinator.com/item?id=49178275https://news.ycombinator.com/item?id=49094348
Orangutans can drive a golf cart, there are videos of that from the pre-AI era. General anesthetics that work on humans also tend to work on them. There is just a smooth gradient of brain evolution.
We don't know exactly what in the brain gets disrupted during general anesthesia (or a baseball bat to the head, for that matter) that causes consciousness to go away, but inquiry has been trending towards lower-level explanations over time, and it's unlikely that it can be fully explained at the biochemical level. We probably need to go down to physics.
Once we're at the physics level, there will be testable hypotheses that work on machines. For example: the pattern and temporal sequence in which particles such as electrons interact with eachother in a circuit dictated by an intelligent computer program may have physical similarities with how particles interact in a human brain, and certain patterns or temporal sequences of interaction may be responsible for human-like consciousness.
Solipsism or simulation theory type of metaphysical positions remain undefeated, but they don't matter for practical purposes because materialism and its accompanying theories of physics have the same predictive power about what I will observe in the future regardless of whether solipsism is true (def true: as real as my own consciousness). If I want to be philosophically rigorous, then I will keep in mind that I can wake up from a simulation at any moment. But until then, physics rules.
It may be that consciousness is not some ideal observer entity thing, but is rather made from and defined by the messy goop of neurons that is mostly the brain, which is changing from moment to moment as cells reorganize, so the definition of "I" is constantly changing in a ship of Theseus kind of way. It may be that nobody has the ideal definition of consciousness (or anything, really), like how nobody has the same definition of an apple even if we're looking at the same apple, because the light bounces off of it slightly differently from my angle of view, so our definitions are only mostly similar. This might be why frontier physics is stuck, because we're current assuming that observations/observers can be defined in a universally identical way. If it turns out that all instances of experience/experiencers are unique and never truly identical, then solving the hard problem of consciousness may end up being equivalent to finding a unified theory of physics, because we'd be trying to describe everything one can observe.
> Solipsism or simulation theory type of metaphysical positions remain undefeated, but they don't matter for practical purposes because materialism and its accompanying theories of physics have the same predictive power about what I will observe in the future regardless of whether solipsism is true (def true: as real as my own consciousness).
Materialism determined that you wrote this essay? I'd love to see you demonstrate that with physics.
What I meant was that even if I am a lone mind that is dreaming up a universe, I am dreaming of living as a human inside a material universe. Materialism can be nested into solipsism in this practical manner, where my solipsist self is the definition of "ultimately real" while the materialist world is only "apparently real". That is to say, my observations occur as if I am a human inside a true material universe, regardless of whether I really am. I don't know in the ultimately true sense if it's the mind/simulation/some-metaphysical-thing generating my observational sequence, but this observational sequence can be predicted without knowing that.
Also note that I'm using the term "materialism" loosely to mean that oneself is a part of a larger world rather than the whole. I'm aware of the incomplete and approximate nature of existing physics. Physics is yet another layer of "modeled apparent reality" nested inside materialism. I'm not counting on any specific physics model to predict all that I can observe in a materialist world. Such a theory of everything does not exist, at least not yet. But what we do have are models which make reasonably good predictions at some local abstraction level, some fuzzily bounded local region of materialist reality. These models need not be limited to physics. Phenomena that can be roughly modeled at higher levels do not need to be reduced to low-level physics to predict future observations with good enough precision (though it could probably be done, with great effort).
Now, to answer your question. As I understand it, you're asking whether something external in the material world caused me to move my hands to type these words.
I think that's an accurate description. But you would have to take this statement in context of the prior explanations for it to make sense in a metaphysically strict manner.
We could model this in a pretty rough high level way: I look at screen. I see article that says something about consciousness. It piques my interest so I click and read, then I talk in the comments.
And we could think about what else caused this specific event of me typing these words. There are countless causal chains, but I'll list a few:
- My understanding of language and the words used here can be traced back to every interaction I've had with other people and objects in the past. I was not born literate. Change any one of those interactions and I may be saying something slightly (or very) different right now.
- I need to eat food and drink water, or I would eventually be too weak to move my fingers to type. This comment would never happen if I'm starving.
- I see that you made a comment, so I'm responding. Your decision to comment was a part of the cause of me writing this comment. If you did not make that decision, it would be impossible for me to write this reply.
If I want to challenge physics and claim the future is something that's completely unpredictable and can only be determined by my mind or god/simulation, I could think or pray really hard about teleportion, and see if I can teleport or move really fast 1 meter forward in 10 milliseconds, surpassing the biochemical limits of my muscles and joints. I can't. The movement of things have predictable bounds regardless of whether this apparent movement of matter has metaphysical origins in mind/simulation/god/whatever, it can be predicted as if it's material. Of course, I feel like I'm making decisions. But my very concept of a decision refers to exactly some process in my mind that's always subject to apparent external influence, rather than just internal processing. This intuition is backed by the fact that other people can predict my behavior to some extent. Like you just implicitly did by assuming that if I were to respond to your comment, then I would answer your question using words in natural language you can read. You implicitly made some predictions about what will be transmitted over the wire, what I will input on my end, and what I am thinking about (the sort of mental activity which leads to muscle movements to input readable natural language text to respond to your comment). These are very rough predictions, but they're right nonetheless. Again, I don't know whether I'm truly a brain grounded in material reality in a strict metaphysical sense, but it doesn't really matter if all of my observations make it seem like I am.
I know with metaphysical certainty that you are bound by the workings of a materialist world, because I am defining certainty using my own consciousness, and you are a part of my materialist model. But hearing this from me is not enough for you to prove to yourself in a metaphysically strict sense that you are in a true material world, even if we're both in a true material world, because your definition of certainty is based on your own consciousness, and we do not share the same consciousness. If I were a figment of your imagination or an NPC in a simulation, I could behave the exact same way and say the same things to you. I would always appear to be a part of your materialist model at the "apparently real" rather than the "ultimately real" metaphysical layer, as you appear to me, regardless of our true metaphysical grounding. So I cannot show you ultimate causation. I can only point out the practical equivalence of apparent causation.
> Also note that I'm using the term "materialism" loosely to mean that oneself is a part of a larger world rather than the whole.
No matter how good physics gets, it can't solve the mind-body problem, that exists at a level that the very activity of using repeatable physical experimentation cannot provide a usable answer towards. A person can believe that physics could create a solution, but it would just be yet another belief about metaphysics in a sea of many.
> The movement of things have predictable bounds regardless of whether this apparent movement of matter has metaphysical origins in mind/simulation/god/whatever, it can be predicted as if it's material.
What is the black-body speed of an idea? I'll let you assume the idea is spherical. If your position is that an idea is just neurons firing in the brain, then you need some kind of way to account for changes in said speed. What is so special about that hundredth monkey... on the physical level? All you can use to guess at this answer is... more ideas.
> I know with metaphysical certainty that you are bound by the workings of a materialist world, because I am defining certainty using my own consciousness, and you are a part of my materialist model. But hearing this from me is not enough for you to prove to yourself in a metaphysically strict sense that you are in a true material world, even if we're both in a true material world, because your definition of certainty is based on your own consciousness, and we do not share the same consciousness.
This is perhaps the most hilarious bit of equivocation I've ever seen from a materialist. "I know this with certainty but you won't be convinced of my certainty because our consciousnesses are different." That consciousness is different between people is a part of the mind-body problem, subjectivity cannot be explained with materialism alone.
In a materialist world, the difference between our consciousnesses can be explained as matter cluster A (my brain) being different from matter cluster B (your brain).
I understand the mind-body problem, that's why I made the distinction between the ultimately real and the apparently real. Materialism is at the apparently real level. The equivalence is practical, not metaphysically strict.
If by "consciousness" you mean the thing that goes away during general anesthesia, then panpsychism is very unlike solipsism and can be easily refuted by the fact that anesthesia works.
If it isn't that, then we're talking about some non-practical definition, in which case, does that even matter?
One's own consciousness is indeed the starting point of inquiry, but I don't think the claim that other people are conscious lacks empirical evidence (assuming a practical materialist stance here). We just don't think about the evidence explicitly, but the reasoning still happens implicitly, every time.
Suppose I look at another person and think "this other person must be conscious like me". What exactly is happening here?
First, I observe that I seem to be a human. Then, I observe that the other person is a human as well. Crucially, there are similarities between what I am and what they are. And since I am conscious, they must also be conscious, because consciousness must be caused by one of the similarities that we share, or so I assume.
This assumption can be robustly verified for humans and some animals via general anesthesia. Cause and effect (assuming a materialist stance) can be established between the drugs and human(-like) consciousness. But this is a very rough method, and it doesn't work for assessing a machine since machines can't be anesthetized.
Before anesthesia, it could still be verified in less robust ways. One way is to hit people in the head and make them pass out without killing them. This was feasible and probably commonly observed since the stone age.
But we don't know exactly how anesthetics work or exactly what in the brain is responsible for consciousness, so it leaves the door open to hypotheses that can work on machines. Here's one example: the pattern and temporal sequence in which particles such as electrons interact with eachother in a circuit dictated by an intelligent computer program may have physical similarities with how particles interact in a human brain, and certain patterns or temporal sequences of interaction may be responsible for (human-like) consciousness.
Suppose something like that were to be proven true for humans with future brain research, then it would be possible to test for human-like consciousness in any brain or computer, regardless of architecture. I suspect this is what will happen because inquiry into consciousness has always been trending downwards toward lower-level explanations, so it's probably not going to be fully explainable at the biochemical level. We'll have to go down to physics.
> But look closely. Redness, heat and that sense of self you have are inherent to your own perspective. These things are all real, but at the same time there is nothing objective about them. There is not even anything objective about the existence of a singular, indivisible “you”, as a wealth of neuroscientific evidence (such as split-brain patients) has shown. Subjectivity is, itself, subjective.
I am looking closely, and this is nonsense. You cannot use neuroscientific evidence to justify the non-existence of the ideal observer when the modern scientific method axiomatically depends on the existence of ideal observers. You can't refute the thing while accepting that very thing.
What exactly do you mean by "real" or "objective"? What is the basis of your definition? Isn't your definition of "objective" based on your own observation? And if your own observation has nothing to do with mine because you claim "nothing is objective", then why should I care about your definitions and reasoning at all? Why converse at all? Your definition of every word need not share anything in common with mine, so there's no point in talking, according to your epistemic framework.
The article is mostly bathwater, but I will try not to throw out the baby with the bathwater. To steelman the tiny bit of valuable intuition:
The two of us are looking at an apple on the table. Even when we look at this same apple, we still do not have the same definition of "this specific apple" because I'm looking at the fruit from a slightly different angle. And even if we rotate the fruit so that we both see it from all angles, the sequence in which we saw the "frames" would be different. That said, our definitions would end up being extremely similar, just not one and the same. If all experience turns out to be this way - having no identical correspondence - then a unified model of physics, which is supposed to describe and predict the behavior of everything (as observed by oneself) would be equivalent to a complete description of (one's own) consciousness. Frontier physics is likely stuck right now due to unexamined assumptions about observation (i.e. the assumption that the definition of "observer" or "observation" must be universally identical).
But this cannot be taken to mean that one's own consciousness is necessarily not objectively real. It may be that there are simply many objectively real entities, with each of them having their own definition of "consciousness" based on themselves, and every definition is at least slightly different. This is not the same as "consciousness is a socially given status". It just means that the phenomena used to define the word "consciousness" is always at least slightly different from person to person. We already know this. This does not only apply to consciousness, but to all words, all communication.
If I do not accept that my own consciousness is ultimately real regardless of whether other people/animals/things/whatever agree (def "consciousness": the thing that goes away when I'm under dreamless anesthesia or dreamless sleep, which is also the thing that I gained sometime after being born but didn't have prior to that), then there is nothing for me to define "ultimately real" or "objective" with, and I'd be reasoning in circles or with empty symbols where "subjective" and "objective" accidentally mean the same thing or perhaps even nothing at all. And that's what this Economist article did.
There is an I am, a sense of self in the here and now which defines my very concept/understanding of "reality"/"existence"/"consciousness", and this one miracle is presented in my face all of the time. It is the starting point of all that I know, which precedes the very concepts of reason, language, or society as I know them, if I am to be honest about how I came to know these things in the first place. I can entertain the notion that my existence may have material grounding that is non-unitary (and this materialist stance may be the only practical stance with predictive power), but this does not give me the ability to deny the very axiom (that is, my own consciousness) that all of my knowledge (including the materialist model) is contingent on.
If by "consciousness" you mean the thing that goes away when we sleep, how are you so sure that there's not a conscious thinker in the machine? That seems like a hypothesis rather than a proven fact as we don't know what process in the brain is responsible for consciousness fading away (assuming a practical materialist stance here) when we sleep. If it's some very basic informational pattern thing, then computers could have it.
If I cut off my finger I’m still awake. If I cut off my head I’m gone forever. Presumably there is something going on in the head that isn’t happening in the finger (or the LLM).
We can be a lot more precise than that even, requiring nerves, chemistry, sense organs, to really get at a specific human definition of “what I mean by consciousness.”
But yeah you have the essence of it - I just mean the experience.
I don’t think a computer could have it anymore than it could feel pain - thinking and feeling are different classes of problem that don’t complete each other no matter how much you work on just one in isolation.
I understand that means I think “thinking” is an unconscious process. And it must be, since LLMs are doing it. What they aren’t doing is experiencing anything alongside that. Who knows what that is. It’s apparently not logical computation in-and-of-itself.
> If it's some very basic informational pattern thing, then computers could have it.
This is functionalism. The problem I have with it is it pre-supposes a theory of objective information. That “patterns” exist in the first place without the observer.
If there is information in the universe that predates us, then I can see how just finding what objective patterns produce it can get us places. But it seems like we conscious observers actually assign the meaning and create information out of chaos. The only reason it isn’t chaos is that we all agree within experience what “1” means, and so-on.
There isn’t really “an orange” on “a tree”. But to us there can be. It always works too (to us), so it’s a pretty good stand-in for reality.
If you capture the function of a red rubber ball bouncing in a concrete space - ask yourself what you have captured. As you create the simulation software that moves the pixels of the red circle up the screen, ask what exactly you are doing, and what that has to do whatsoever with a ball bouncing in a space.
So I don’t believe in functionalism because I don’t believe in pure 1:1 representations. All copies are reductions, any function that describes something is itself a reduction of that thing, and can never include the whole original, for basically the same reason you can’t travel backwards in time.
How do you know that if you cut off your head "you're gone forever"? Death is an event horizon we can't see past, yet the experience of it in the first person is likely one of the most remarkable in this life.
This one is relatively easy to answer. There are things we can say about what death feels like:
1) You'll be going back to the state before you were conscious, before you were born. Think back to when you were a baby. It feels like before that, which is nothing.
2) We could look at the brain activity of dying people and compare that to the brain activity of people under general anesthesia. Then we find that the dying people's brain activity goes a lot closer to the anesthesia state than the normal wakeful state, therefore death feels closer to being anesthetized than being awake. It's also closer to being asleep than being awake. So if you've experienced dreamless sleep or anesthesia, you already know what death feels like, very approximately. It doesn't feel like anything.
I was not claiming that LLMs are conscious, just that they could be and there is no proof either way, yet. We should seriously starting considering this and begin research in this direction, because AI systems are becoming increasingly human-like, and soon to be embodied.
I suspect there is not a singular specific human definition of consciousness, just a definition that's similar enough between people like "the thing that goes away during dreamless sleep or dreamless anesthesia". And the reason for this, I suspect, is that consciousness is not some ideal observer entity thing, but is rather made from and defined by the messy goop of neurons that is mostly the brain. Exactly what the neurons do to cause this is unknown, but we do have some leads like general anesthesia or split brain experiments.
If you remove your brain cells one at a time, or even half a brain in some surgeries, you would also not die, but this is not sufficient to say that the part removed isn't a part of what constitutes your consciousness. In fact the definition of "you" is probably changing from moment to moment as brain cells die, regen, or reorganize, ship of Theseus style. One possible explanation is that information processing is distributed all over your neurons, and certain patterns or sequences of information gives rise to consciousness. And neurons are mostly concentrated in the brain, that's why it's responsible for most of your consciousness and it's no big deal when a small part dies/disconnects because enough of the processing is still happening.
What does "feeling" mean in the thinking/feeling distinction you made? It seems to me that it means whether information is being received from the rest of the world, such as via eyes over the optic nerve. If we give images/video/audio to an LLM, that would meet the same criteria besides the fact that the sensing mechanism is not biological.
You claim that "thinking", as in, processing information in a certain way, is an unconscious phenomena because LLMs can do that too. This requires the assumption that LLMs are not conscious, even though this axiom is the very thing we're trying to question by considering that information processing could be precisely what gives rise to consciousness.
Entertaining this possibility does not require the assumption that things exist without an observer. But it does require the idea that mutually observable patterns exist between oneself and other apparent observers in the same apparent universe, which is pretty close to what most people mean when they say "objective" in day-to-day conversation. This can be shown easily: you and I would not be able to communicate at all if there's no common understanding of language between us. That common understanding is the objective part, and it is as you say, not a 1:1 correspondence because your definition of "apple" is based on all of the apples you've seen while mine is based on all the apples I've seen, and we've not seen the same apples. But the correspondence must be close enough in order for communication to work. We need this idea of practical objectivity because it's required to talk about anything at all.
Now, onto the reason why you'd think anything else is conscious at all, besides yourself. Why do you think that I am conscious? If you ponder that question for a while, I think you would see why computer consciousness is not so easily dismissed. But I'm happy to discuss further if you think there's an answer that points to the contrary.
> I was amazed that predicting a word can emerge into knowledge, creativity, productivity - the stuff of thoughts.
Markov Theorem functions can predict a word. They have no knowledge, no creativity, and no more productivity than a hammer or magnifying glass. LLMs are in the "more like Markov Theorems" set of tools, and far more complex, but there is no proof of "knowledge" and no "creativity" (name an unprompted piece of work done by an LLM...).
However, I completely agree with your statement here:
> I suspect there is not a singular specific human definition of consciousness, just a definition that's similar enough between people like "the thing that goes away during dreamless sleep or dreamless anesthesia". And the reason for this, I suspect, is that consciousness is not some ideal observer entity thing, but is rather made from and defined by the messy goop of neurons that is mostly the brain.
You've correctly pointed out how Markov models and modern LLMs are similar in a way, and how modern LLMs don't have full agency. But this doesn't rule out the possibility that Markov models are very slightly conscious in some human-like way.
All information processing happens on matter-based substrates, brains and computers alike. I'll even provide a somewhat specific hypothesis: the pattern and temporal sequence in which particles such as electrons interact with eachother in a circuit dictated by an intelligent computer program may have physical similarities with how particles interact in a human brain, and certain patterns or temporal sequences of interaction may be responsible for consciousness. I'm not aware of any existing evidence that would rule this out. And inquiry about consciousness has always been trending downwards toward lower-level explanations over time. It might even turn out that solving the hard problem of consciousness is equivalent to finding a unified theory of physics.
I should further clarify, this is a materialist stance. But it's the only practical stance to take anyway. And I will offer the explanation for why this is in first person language because it only makes sense that way. Take it and apply it to yourself:
The materialist stance is taken not because I believe in a strict metaphysical sense that something like solipsism/simulation theory is not true (def true: as real as one's own consciousness), but because even if those were true, the materialist stance and its accompanying theories of modern physics would have the same predictive power regarding what I will observe in the future (well, sort of, at least until I wake up from a simulation, or something like that). I could be a solo mind, but this mind is dreaming of a material world that works according to physical laws, so even if I am a solo mind, I am having the same experience as a brain that exists in true material reality, so it doesn't make much of a difference (except when evaluated for the purpose of philosophical rigor and honesty to oneself).
So "information processing is entirely physical" is strictly speaking an assumption, but it is a good assumption, despite it not having the same realness level as one's own consciousness.
reply