[ExI] Emily M. Bender — Language Models and Linguistics (video interview)

[Jason Resch]

On Mon, Mar 27, 2023, 1:12 AM Gordon Swobe <gordon.swobe at gmail.com> wrote:

>
>
> On Sun, Mar 26, 2023 at 10:39 PM Jason Resch via extropy-chat <
> extropy-chat at lists.extropy.org> wrote:
>
>>
>>> It knows how the words "left" and "right" relate to each other and to
>>> other symbols related to spatial dimensions, syntactically but with no
>>> access to the referents to give them meaning.
>>>
>> The fact that  GPT can construct a coherent essay from what is to it
>>> meaningless symbols is to me no less amazing than the fact that it can
>>> create a mathematical model from meaningless symbols. It's all basically
>>> the same amazing process, a giant web of inter-relationships between
>>> meaningless symbols that have no meaning to it, but which do have meaning
>>> to us on account of it was trained on the forms and patterns of our
>>> language.
>>>
>>
>> But your (and I think Bender's) assertion was that there was no possible
>> way to learn any meaning whatsoever without a Rosetta stone, or without
>> programming in some model of reality from the start.
>>
>
> It must be getting late where you are.:) I meant that there is no Rosetta
> Stone *for GPT* to understand the meanings of the symbols. We need no
> Rosetta Stone if we are using the English version of GPT.
>
> If that's true, how then does the LLM come to learn the spatial meaning of
>> a word like 'down' when all the model encounters are "meaningless symbols"
>> which are themselves only defined in terms of other "meaningless symbols"
>> ad infinitum?
>>
>
> It never learns those meanings, but because it understands the grammatical
> (syntactic) relationships  between the symbols,
>

But appropriately constructing a mathematical object suggests it has
semantic meaning, does it not?

Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230327/a28ae841/attachment.htm>