[ExI] ChatGPT-4 gets the joke, almost

Tue Apr 25 10:50:23 UTC 2023

*Why do you suppose Sam Altman and Ilya Suskever, founders of OpenAI,
believe the only good test of a conscious LLM would involve training it on
material completely devoid of references to first person experience?*I head
them saying this and it makes no sense at all. The fact they are the
founders of OpenAI doesn't mean they really understand deeply how LLM work,
they may be aware of the technical details of how they are structured and
how they operate but not necessarily how they achieve their abilities given
really nobody knows. We don't have models that take the weights of the LLMs
and can predict what the output would be based on some theoretical
understanding of the weight relationships. The only way to know what would
happen if one changes slightly the weights is by doing that and observing
the result.

The idea of using material that is devoided to references to first person
could be a decent experiment but not relevant to the matter at hand. I have
a strong intuition that it will not produce an LLM that has the ability to
interact with us like GPT-4 is able to do. Basically, this type of material
will be something like an encyclopedia and the result would be an expert
system and nothing like the master of language that GPT-4 is.
To master the language you have to train an LLM on real language with all
its complexity, its implicit theory of mind, and the presence of a large
body of language that is based on first-person experience.

What is implied by the suggested experiment above is that GPT-4 arrived at
a sense of self by being influenced by the presence of first-person
language in the text. But this is trivially obvious. GPT-4 itself would
tell you that his use of "I" is a convention that it adopted to help with
communication with human users. It also can be instructed to use a "third
person" to describe itself.

Giovanni: How would be possible to have a conversation without using these
conventions?
GPT-4:
It's possible to have a conversation without using some of the conventional
language patterns, like first-person pronouns, by adopting alternative
phrasing and focusing on the content being discussed. For example, instead
of using "I," the AI model can use passive voice or refer to itself in the
third person. Here's an example of how a response might look without using
first-person pronouns:

User: Can you tell me about the weather today?
AI: Today's weather can be described as sunny with a high temperature of
75°F.

User: Do you have a favorite color?
AI: As an AI language model, having personal preferences is not possible.
However, assistance in providing information about colors can be given.

Using alternative language patterns might make the conversation feel less
personal and more formal, but it can still be informative and helpful. The
key is to ensure the information being shared remains clear and
comprehensible to the user.

What is exciting for several of us that are interested in the possibility
of self-awareness in machines is not the fact that AI uses "I". I consider
the process of individuation the hallmark of consciousness in humans but
this can be easily simulated in a chatbot. Even the simplest and most
boring forms of chatbots can speak in the first person. So focusing on the
fact GPT-4 talks in the first person completely misses the point.

The most astounding GPT-4 abilities are around creativity, logic,
contextual understanding, the ability to reason about spatial information,
theory of mind, getting jokes, and extracting subtle literary styles from
the writing of a specific author (it picked nuances of spike writing from a
couple of short emails) and the list continues. The people in this list see
these as early signs of a possible internal model of the world, as early
signs of a possible sense of awareness, and the type of intelligence that
we attribute to humans and not calculators are the last people that could
be fooled by stuff like this if it was not real, testable and consistent.
The most fascinating aspect of GPT-4 is that is a playground for
experiments in cognition, logic, theory of mind, and semiotics. One could
write 100 papers based on experiments with GPT-4 that one can do from home.

You believe yourself superior because you are not falling for the smoke and
mirrors tricks of a sophisticated autocomplete but sorry Gordon, it shows
you don't understand the basics of how these systems work because a system
with billions of parameters that models language in all its incredible
complexity (in several world idioms) is nothing like an autocomplete. What
you are missing is that statistics alone would not have worked in creating
this mastery of language. Why? I'm sure you understand the idea of a
combinatorial explosion. A statistical word predictor would need to
calculate the probability of 2 words coming one after another, then 3
words, 4 words, and so on. By the time you are calculating the probability
of a word in a sentence (to have a chance to have some contextual
relevance), you end up with crazy large numbers that no machine can handle
even if given billions of years of processing.

The only way to accomplish this goal of word prediction is to make models
of how words follow each other in a sentence and extrapolate to more and
more word combinations without actually calculating the probabilities for
each combination. That is the key to understanding LLM. They make models of
the universe of language that for us humans is more and more the entire
universe.

How do they make these models? Using Deep Neural Nets training on language
data because this architecture seems to be what the brain comes up with to
solve problems and make sense of the world, including language. We don't
know how these nets work in detail but they seem to work. It was an
incredible bet and many said it would never work. But it worked.

So it is not the statistics that allow LLMs to predict the next word but
their model of language. Let me make an analogy. It is like training a NN
by watching an object falling and calculating stats of the motion of an
object, 2 objects and so on and then showing rain with thousands of
droplets. It can calculate the stats for a couple of stones falling, how
much time it takes to reach the ground for example, and by the time you
show a video of a fluid or sand falling it cannot do it. It has to build
models of how these phenomena work and this just from some simple stats
related to stone fallings. It seems crazy but you train the AI and ask to
simulate the behavior of a falling stone, 2 stones, and it does a good job
but by the time you go to the behavior of falling water, it messes it up
badly. Instead of giving up you give it more videos to watch, you increase
the number of nodes in the NNs, and spend millions of dollars in training
the system with human feedback. And one day the system can simulate the
behavior of fluid perfectly, it can show you the dance of stars in a galaxy
or the motion of clouds in the sky. It basically derived the laws of
physics by watching videos of natural phenomena. Nothing to do with
autocomplete. Just somebody that has zero clues on how these system work
will call them a stochastic parrot.
The same applies to LLMs. They may have started with simple stats on the
occurrence of words in a sentence but this doesn't give mastery of
language. You have to build a model of how language works. These LLMs did
exactly that. They have a model of the universe of language. They are the
opposite of a stochastic parrot. I will write soon an article on this
topic. If LLMs are stochastic parrots then physics is stochastic science
(with this I don't mean a science that studies stochastics but a science
that predicts things based on some not understood statistical patterns of
nature). We understand these patterns because we have laws that allow us to
make precise predictions. Even in the case of complex systems where precise
predictions are not possible, we can describe the general properties and
scaling of these systems. We do all this because we have good models of
reality. The LLMS have models of language in their "heads" otherwise they
would not work as well as they work. They figure out how language works.

They didn't derive laws and they cannot explain to us how language works
because the model is the weights in the net. The weights are the model.
Same with our brain, we figure out how language works (and for some of us
not as well as state of art LLMs) but it is all in the weight and we do not
know really how language works explicitly (using formulae or algorithms) in
particular linguists like Bender.

The main question then is, do you have some primitive form of awareness if
you figure out the language, if you have a model in your "head" about
language? Do you have to understand what language is about to have a good
model of language? Can you use this model of language that is able to find
patterns and regularity in a complex universe of relations to solve other
problems like spatial relationships, logical inference, and understanding
cause and effect? Can this model be used by the system to self-reflect and
understand its own states? These are the relevant and important questions
that I don't see many people discussing at all.

This list is one of the few places where I see meaningful discussion along
these lines (Jason and Ben in particular) and Gordon, you repeating that
GPT-4 is just an autocomplete doesn't bring any depth to the discussion and
it is insulting to all of us that are trying to understand this important
moment in the history of humankind.

Giovanni

On Tue, Apr 25, 2023 at 2:28 AM Gordon Swobe <gordon.swobe at gmail.com> wrote:

> I’m not hell-bent on anything, Giovanni, except getting the truth out that
> these language models are no different in principle from any other
> application, most similar to an autocomplete function in a word-processing
> app, just vastly more sophisticated and deeply trained on massive amounts
> of text such they can do remarkable things including the mimicry of someone
> writing in the first person. They do this so well that gullible people are
> fooled into believing they are interacting with conscious beings.
>
> Riddle me this, genius Giovanni,
>
> Why do you suppose Sam Altman and Ilya Suskever, founders of OpenAI,
> believe the only good test of a conscious LLM would involve training it on
> material completely devoid of references to first person experience?
>
> To put this another way, why you do believe the founders of OpenAI think
> we cannot say GPT-4 is sentient while you preach otherwise and harrass and
> annoy me for agreeing with them?
>
> -gts
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230425/174108fb/attachment.htm>