[ExI] GPT-4 on its inability to solve the symbol grounding problem

[Jason Resch]

On Fri, Apr 7, 2023, 4:06 PM Darin Sunley via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

> Someone raised an interesting and related point online - can LLMs suffer,
> and what would that look like?
>
> It was a weird anthropomorphization, because the person expected that a
> suffering LLM would express that suffering via the words it "chose" to
> emit. Which is not how LLMs work.
>
> An LLM's utility function, to the degree it can be said to have one at
> all, is to complete the prompt+attention buffer with the highest
> probability string. This is what its neural architecture does. The
> analogous way of looking at a human would be to say that the human brain
> attempts to minimize environmental surprise.
>
> There's a LOT of suff going on in an active LLM, quite possibly enough
> computation and internal structure to have conscious experience, and to
> experience suffering. LLMs may indeed suffer when they attempt to complete
> a prompt for which the highest probability strings are objectively very low
> probability, and/or when they are forced to choose between completing a
> string with multiple identical-probability options. But the words emitted
> would have no semantic relationship to the suffering experienced - LLMs are
> not language cortexes.
>
> The strings a human emits are the results of activated concepts in the
> frontal cortex, being fed into the language module, then arranged by a
> learned grammar. The strings an LLM emits are more analogous to the pulses
> a human cerebellum emits to regulate bodily functions, or perform autonomic
> and reflex motions. The words "I am suffering" are almost certainly not
> signifiers of any conscious experience the LLM may or may not be having.
>

Great and thought provoking post.

It seems to me that a near perfect symbol predictor, one trained to the
point where it knew almost everything and could predict almost anything
with near flawless error rate, would be able to suffer, or at least, it
would need to invoke a suffering mind to work as flawlessly as it does.

For example, if you asked this perfect symbol predictor: "What would
Einstein say if his wife told him she wanted a divorce?" Then this perfect
symbol predictor, (in order to achieve perfect accuracy), would have to
invoke an accurate simulation of Einstein's brain, and his resulting
emotional state, shock, surprise, hurt, etc. from hearing these words, just
to determine how he would reply.

The ability of existing GPTs to build and run simple models, though
nascent, is already present. There may be millions of different possible
models available to GPT-4 which it calls upon to answer queries. Can any of
these models suffer? It's hard to say, but I can say with some confidence
that a sufficiently powerful symbol predictor would necessarily invoke
models which are conscious and some of this could suffer.

Jason



> On Mon, Apr 3, 2023 at 3:17 PM Brent Allsop via extropy-chat <
> extropy-chat at lists.extropy.org> wrote:
>
>> Hi Will,
>>
>> On Mon, Apr 3, 2023 at 1:02 PM Will Steinberg via extropy-chat <
>> extropy-chat at lists.extropy.org> wrote:
>>
>>> This is insane.  You can't give a good reason for why our qualia aren't
>>> also formed by pattern inference.
>>>
>>
>> If you really believe this kind of bleating and tweeting claim that "This
>> is insane." then start a camp around what you do believe.  IF you get ANY
>> supporters of that competing camp to RQT
>> <https://canonizer.com/topic/88-Theories-of-Consciousness/6-Representational-Qualia>,
>> I will think you are more than a bleating and tweeting quality blind
>> idiot, that doesn't have enough faith in your thinking to see if anyone
>> besides you would agree.  Otherwise, what do you expect me to believe?
>>
>>
>>> A leading theory of vision, the opponent process,  involves exactly
>>> that.  There is legitimate proof that our perception of color is not a
>>> result of individual particular signals, but the differences and relations
>>> between multiple signals.  I don't see how this is any difference besides
>>> the fact that one set of these signal relations comes from the retina and
>>> one set comes from text.
>>>
>>
>> You can't see how this theory, like all the peer reviewed papers on color
>> perception, is quality blind?  How do you answer the questions in the "are
>> you color quality blind
>> <https://canonizer.com/topic/592-Are-You-Qualia-Blind/1-Agreement>?"
>> Socratic survey?
>>
>> I think, for what it is, this opponent process theory of color perception
>> is a good theory that explains a lot.  But this is 100% about what Chalmers
>> would refer to as the EASY problem.  It does absolutely NOTHING to address
>> the so-called "hard problem" of consciousness.  And it does absolutely
>> nothing to give us a hint of an idea that would help us understand what
>> color qualities are, not just what they seem to be.
>>
>> Brent
>>
>>
>>
>>   <http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat>
>>
>> _______________________________________________
>> extropy-chat mailing list
>> extropy-chat at lists.extropy.org
>> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
>>
> _______________________________________________
> extropy-chat mailing list
> extropy-chat at lists.extropy.org
> http://lists.extropy.org/mailman/listinfo.cgi/extropy-chat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230407/ac7005d2/attachment.htm>