[ExI] Douglas Hofstadter, LLMs and self-referentiality

[John Clark]

*Gödel Escher Bach by Douglas Hofstadter is one of my all-time favorite
books, it and Drexler's Engines Of Creation have done the most in shaping
my current worldview, but the huge advance in AI made during the last 3 to
4 years have made it clear that many of the ideas expressed in Hofstetter's
book are just wrong, and I think even Hofstetter would now admit that.
Quantum computing expert Scott Aaronson talks about this in a recent post:*

*"**A central thesis that many readers, including me, took from Douglas
Hofstadter’s Gödel Escher Bach when young was that the secret of
intelligence (and therefore, of AI) was going to have a lot to do with
self-referentiality and “strange loops.” [...] LLMs’ ability to talk about
themselves popped out as a byproduct of their ability to talk about
anything in the discourse universe they were trained on. The big, old ideas
about intelligence that ended up basically vindicated were the ideas about
how intelligence is about prediction, and prediction is about compression,
and compression is about finding better and better upper bounds on
Kolmogorov complexity. Not the self-reference stuff. [...] The idea that
you’d need explicit self-referentiality before you could get convincing and
world-changing conversational intelligence? Let it be buried in a
Westminster Abbey or Arlington National Cemetery for the most important
wrong ideas in human history — geocentrism, Aristotle’s teleological
physics, aether, phlogiston, Freud’s psychology, Marx’s prediction of a
workers’ uprising followed by a classless utopia, etc. But buried it needs
to be."*



*LLMs and self-referentiality <https://scottaaronson.blog/>*

*John K Clark    See what's on my list at  Extropolis
<https://groups.google.com/g/extropolis>*

*7ui*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20260901/6246cc89/attachment.htm>