[ExI] Douglas Hofstadter, LLMs and self-referentiality

[John Clark]

On Wed, Sep 2, 2026 at 8:34 AM Jason Resch via extropy-chat <
extropy-chat at lists.extropy.org> wrote:

*> I never viewed Hofstadter's strange loops as being necessary for nor
> central to intelligence. Rather in my readings of G.E.B. I see
> self-reference and strange loops as the origin of mysteries of the "I" and
> central to the understanding of how things like free will*
>

*I said it before I'll say it again, free will is an idea so bad it's not
even wrong.  *

* > emerge when the direction of causality gets twisted in strange ways*
>

*Nobody can know for sure what the result of a calculation will be until
they have finished the calculation, and they don't know for sure what
they're going to do next until they've actually done it.  And after all
that some people like to say "I decided to do that of my own free will". I
say you either did it for a reason or you did not, and if you did not then
what you did was an event without a cause, in other words it was by
definition random. *

*> "Our thoughts seem to run about in their own space, creating new
> thoughts and modifying old ones, and we never notice any neurons helping us
> out! But that is to be expected. We can’t. *[…] We should remember that
> physical law is whatmakes it all happen–way, way down in neural nooks and
> crannies which are too remote for us to reach with our high-level
> introspective probes."
> *— Douglas Hofstadter in “Gödel, Escher, Bach” (1979)*


*I still agree with  Hofstetter about that.  *

*> we are as ignorant of what our own neurons are doing as LLMs are
> ignorant of what their own low level activations and weights are doing.
> Minds, being high level phenomena, will generally lack access and
> introspection to the low levels supporting them.*


*Yes.*


> *> self reference (and it's limitations) answer Nagel's question of why he
> cannot know what it is like to be a bat, unless he is a bat, but then he is
> no longer Nagel. So Nagel (the system) is trapped in itself, and thus
> cannot access bat-like knowledge.*
>

*Exactly.  *

*> When we speak of a person wanting a glass of milk, and going to get it,
> this is still meaningful, even though everything could fundamentally be
> explained in terms of interacting quantum fields, and no laws of physics
> are violated by a person's mind or will,*
>

*Yes, although theoretically possible it would be silly to use quantum
field theory to explain why a person drinks a glass of milk.  *



> * > without this insight, we face a dilemma in explaining what the point
> of consciousness is*
>

*Darwinian Evolution gave us intelligence and if there was a "point" for
doing so it was to increase the number of genes that get into the next
generation, but unlike intelligence natural selection can't detect
consciousness so there is no "point" to it, it must exist because it's a
brute fact that consciousness is the way data feels when it is being
processed intelligently.    *


*> "There was a young man who said, “Though,*
> *it seems that I know that I know,*
> *what I would like to see*
> *is the ‘I’ that knows ‘me’*
> *when I know that I know that I know.”"*
> *— Alan Wattsin “The Book On the Taboo Against Knowing Who You Are” (1966)*
>


"I wonder why. I wonder why.
I wonder why I wonder.
I wonder *why* I wonder why
I wonder why I wonder!”
*― Richard P. Feynman, "Surely You're Joking, Mr. Feynman!" (1985)*

*John K Clark*



>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260902/bcd45c5c/attachment.htm>