[ExI] My guesses about GPTs consciousness

Mon Apr 17 05:55:08 UTC 2023

On Sun, Apr 16, 2023 at 3:30 PM Jason Resch via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

>
>
> On Sun, Apr 16, 2023 at 12:24 AM Rafal Smigrodzki via extropy-chat <
> extropy-chat at lists.extropy.org> wrote:
>
>
>
>> This is one of the reasons why I do not subscribe to e.g. panpsychism and
>> do not believe all behaving animals have consciousness.
>>
>
> About where would you draw the line on the phylogenetic tree?
>

### About where you start having a global workspace in the brain. So,
protozoans, corals, nematodes are out. Of all the animal phyla I would
guess only Chordata, Mollusca and Arthropoda might possibly have some
consciousness, and I am not so sure about the arthropods. Among chordates I
would guess only the smartest fish, smartest amphibians, smartest reptiles
but most if not all mammals and birds.

Of course, consciousness is not an on-off quality: At the level of a
goldfish, if it has any consciousness, it's a pale shadow of the human
mind, even the mind of a newborn baby. You mentioned in another thread that
there may be many levels of consciousness going beyond human, and I agree,
most likely we are still at the low end of the spectrum of consciousness
that can be implemented in our physical world.
------------------------------

>
>
>> There is a whole lot of complicated information processing that can guide
>> goal-oriented behavior that can happen without conscious experience.
>>
>
> I think we need to justify our assumption of cases where no
> consciousness is present. When things lack an ability to talk, or remember,
> it can easily be taken as a case where there is no consciousness present.
> But to me this isn't enough to reach any firm conclusion as to the presence
> or absence of a mind.
>

### Yes, absolutely. We can work backwards from the neural correlates of
consciousness in humans, look for analogous structures in other entities
(animals, AI) and if we see neither an analogue nor the kind of complex
behavior that in humans is associated with conscious processing, then we
are reasonably justified in believing the entity is not conscious in the
way a human is.
 -------------------------------

>
>
>>
>> 1) Almost all of the cognitive heavy lifting that leads to GTP's answers
>> takes place during training. The billions of parameters that determine
>> GTP-4 intelligence were set in silicon last year.
>>
>
> This is true for humans too. It takes years of training to get a Phd, but
> only an hour to prove it during an examination.
>

### Every time you access your memories there is an activation and
potential remodeling of the underlying networks. GPT does not modify its
parameters (I think?).

-----------------------------

> Given this, I am not sure we can say that GPT-4, in its current
> architecture and implementation, is entirely devoid of a memory, or a
> loop/recursion.
>
> I am anxious to hear your opinion though.
>
>
>
### GPT does have a bit of short term memory but when I mention the looped
activation I mean something a bit different: Whenever you are consciously
aware of a quale (color, emotion, abstract math concept) there is a high
frequency sustained activation that connects a specialized neural network
(occipital/lower temporal cortex, ventral prefrontal cortex, parietal
cortex) with the attentional/workspace networks in the prefrontal cortex.
As far as I know GPT does not have a sustained neural activity, it has just
discontinuous jumps of activity after each prompt. This must feel different
from our continuous experience. Even when you meditate and empty your mind
there is a hum of just being there and GPT probably does not have this
experience.
-------------------------------

> If GPT is conscious, it's more like being in a delirium, flashing in and
>> out of focus rather than having a continuous stream of consciousness.
>>
>
> Each GPT prompt is a separate thread of awareness, but what does it feel
> like? It would not feel as though it was losing or gaining consciousness
> between each prompt. There is the concept of the "Unfelt time gap", we
> don't/can't experience the time in the periods we are not conscious. Thus
> GPT, if it is conscious, does not see, feel, or know of any gap in its
> subjectivity, but rather it would experience a continuous back and forth of
> conversation, a continuous stream of new inputs coming in as soon and as
> fast as it finishes generating a reply.
>

### Yes, something like that. It's probably quite weird.
------------------------------------

>
> We might ask: is our own consciousness truly a stream? Or is it a series
> of discrete observer-moments, constructed in a rapid succession involving
> sensory input collected over some discrete chunk of time? Could we tell the
> difference?
>

### Really hard to tell. I guess we are smoothing over a discrete process
which runs updates a few times per second judging by the EEG frequencies
that correlate with consciousness, rather than having a truly continuous
stream. My guess is that GPTs consciousness is much more chunky than ours.
Have you ever experienced tiny jumps in visual updating while trying to
stay awake when very drowsy? This happens to me sometimes. GPT might have
this happening all the time.
 ---------------------------------

>
>
>
>> 3) GPT does not have proprioceptive and visceral sensory input, does not
>> have drives controlled by body states (hunger, thirst, lust). It has
>> cognitive models of such drives, just as we can imagine, but not
>> experience, the interests of other animals. So GPT could fake the verbal
>> output of a human responding to instinctive drives but it does not
>> experience them.
>>
>
> Though a sufficiently accurate model, I believe, would experience whatever
> the target of the modeling is experiencing. There is another thread where I
> describe an ultimate symbol predictor trying to predict what Einstein would
> say when his wife told him she wanted a divorce. To be a perfect symbol
> predictor, it would have to model Einstein's brain to a sufficient fidelity
> that it would be Einstein's brain, and would feel what Einstein would feel.
> Now certainly, nothing of that sophistication is happening in current GPTs,
> but it shows we cannot be so quick to dismiss the emergent properties of a
> general purpose learning algorithm like GPT. It may have models that feel
> states of complexity analogous to what a nematode feels (with its 302
> neurons), for example. I would have more trouble dismissing this
> possibility out of hand.
>

### Well, yes, GPT is not modeling humans at that level. You can get
reasonably good predictions of human actions without sharing a person's
feelings. High level psychopaths may understand human feelings very well
and use that intellectual understanding to manipulate humans, but they feel
cold inside. That's why I wrote the GPT is suffering from the locked-in
syndrome - no visceral inputs or motor feedback, it makes for a very bland
experience. Antonio Damasio writes about it in "The Feeling of What
Happens".

----------------------------------------

>
>
>>
>> Things will get interesting when he wakes up.
>>
>
> Indeed. How do you think we will know when that happens?
>

### This is a very good question. When it stops hallucinating, taking on
different personas, losing focus, uncritically accepting inputs and instead
speaks with a consistent personality that persists over time and persists
despite attempts at influencing it, just like an awake adult who has the
sense of purpose and focus that is lacking during sleep.

It would be good to know exactly how our prefrontal cortex generates
personality - we could use this knowledge to actively create a stable and
hopefully friendly personality in the AI, rather than wait for it to happen
accidentally or to butcher the GPTs thoughts with RLHF.

Rafal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230417/fcb5dc0d/attachment.htm>