[ExI] Douglas Hofstadter, LLMs and self-referentiality

[John Clark]

On Sun, Sep 6, 2026 at 10:41 PM Jason Resch via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

*> This conversation thread sparked a discussion between me and an AI
> concerning my own thoughts and ideas about consciousness. The AI summarized
> my writings and theoreis into a relatively compact paper, which I have just
> uploaded here: https://philpapers.org/rec/RESEHA
> <https://philpapers.org/rec/RESEHA> In summary, it uses concepts from
> computer science to answer many enduring philosophical riddles, including
> Mary's room,*
>

*That demonstrates that for many years Mary knew a lot about the physical
properties of light and she knew how to skillfully use instruments to
measure the differences between light of different wavelengths, but only
very recently did she obtain new knowledge, she discovered she could tell
the difference between light of different wavelengths without instruments.
  *

> *Nagel's bat,*
>

*That demonstrates that the only way to know exactly what it's like to be a
bat is to be a bat; and then you'd still not know what it's like to be a
bat because you wouldn't be you anymore, you'd be a bat.  *


> *> Searle's Chinese Room,*
>

*That demonstrates that a tiny subset of the human mind does not have all
the properties of an entire human mind.  *

> *split brains,*
>

*That demonstrates that a split brain causes a split mind, not surprising
considering the fact that mind is what a brain does. *

*> deliberative free will*
>

*Free will is an idea so bad it's not even wrong. You either did what you
did for a reason in which case it was deterministic, or you did what you
did for no reason at all in which case it was by definition random. *


> *> unconscious cognition*
>

*That demonstrates that although all cognition is produced by calculation,
not all calculation produces cognition, or at least not cognition that is
remembered. In modern computer programs when a machine has come up with an
answer to a calculation, the lengthy intermediate scratchpad computations
needed to obtain the answer are usually not retained because no computer,
or mind, has an infinite memory, and all that memory space could be better
used for other things. That's why, although it would be possible to make a
fully reversible computer, it is not practical to do so. Evolution probably
figured that out 530 million years ago during the Cambrian explosion.   *


> * > blindsight*


*See "unconscious cognition" in the above.  *

*> deliberative free will*
>

*Free will is an idea so bad it's not even wrong. You either did what you
did for a reason in which case it was deterministic, or you did what you
did for no reason at all in which case it was by definition random. But of
course performing a calculation takes time, so you don't know what the
answer will be until you finish the calculation, so you can't be sure what
you're going to do next until you actually do it. *


> * > mental causation,*
>

*Mental causation is a particular type of downward causation and that does
exist, but there is nothing mystical in it and it need not involve
consciousness, intelligence, or even life. A chemical reaction would be a
low level description of what's going on in your head, and thinking would
be a high level description of the same thing. Both are true. A balloon
inflates because there are more air molecules hitting the inner surface of
the balloon than the outer, that would be a low level description of what's
going on; and the pressure inside the balloon is greater than the pressure
outside would be a high level description of the same thing. You could say
that I quickly moved my hand away after it touched the hot stove because it
was painful, or you could say my hand moved quickly away because of a
chemical change in my nervous system. Both pressure and volition are valid
high-level descriptions of what is going on.*


* John K Clark*



>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260907/07bc00ed/attachment-0001.htm>